.. _custom-metrics: ================================== How to add custom metrics in Fleio ================================== If you want to be able to bill custom metrics that are sent over by Gnocchi you will have to add a new billing resource in the Django admin. First of all you will have to enable Django admin. Please see :doc:`/configuring/django-admin` in order to enable the Django admin interface. After that, you will have to log in the Django admin and access the billing resources table (it should show in the billing app sub-menu). From top-right corner you will have to click on the "ADD BILLING RESOURCE" button. Configure as the following: .. code-block:: bash Display name: This is the resource type. Our example will be CUSTOM Type: metric Name: resource_type_name (should be a valid resource type name from Gnocchi) Definition: Should contain a list of metric definitions (see example below) .. code-block:: python { "metrics":[ { "name":"custom-metric-1", "aggregation":"sum", "granularity":300, "reaggregation":"sum" }, { "name":"custom-metric-2", "aggregation":"sum", "granularity":300, "reaggregation":"sum" } ] } Metric definition ================= A metric definition may have the following fields: - `name` - a valid metric name from Gnocchi - `aggregation` - aggregation function used to retrieve measures from Gnocchi (see below for valid values) - `reaggregation` - reaggregation function used to re-aggregate retrieved measures in Fleio (see below for valid values) - `granularity` - granularity used to retrieve measures from Gnocchi (valid values should be retrieved from the aggregation policy used for the metric) - `metric_query` - query used to filter retrieved measures (this will be passed to Gnocchi and must use the format described here https://gnocchi.osci.io/rest.html#search) - `refresh` - set this to `true` to instruct Gnocchi to force aggregation of all known measures - `resample` - new granularity to resample measure to - `fill` - value used to back fill missing data points when retrieving measures - `billing_based_on_existence` - flag used for billing per second/hour for resources that are not using the mean, min or max re-aggregations - `unit` - unit for the retrieved measures (see below for valid values) - `rescale_to` - unit to rescale to (see below for valid values) The following fields are optional: `reaggregation`, `metric_query`, `refresh`, `resample`, `fill`, `billing_based_on_existence`, `unit` and `rescale_to`. **Aggregation and reaggregation** Valid values for aggregation and reaggregation are `mean`, `sum`, `min`, `max`, `count`, `last` and `first`. Note that in order to use any of these values in aggregation they have to be defined in aggregation policy used for the metric in Gnocchi. For reaggregation any of the valid values can be used regardless of policies since reaggregation in performed in Fleio. If reaggregation field is not defined the value from aggregation field will be used for reaggregation. **Units and rescaling** Fleio can assign a custom unit to retrieved measures (e.g. b for bytes, k for kilobytes) in order to be able to rescale the measures to a more user-friendly value (e.g. display gigabytes instead of bytes). Supported values for rescaling (in ascending order) are: 'b', 'k', 'm', 'g', 't', 'p' and 'e'. Fleio will rescale by multiplying or dividing with 1024 as needed. **Display names** You can define custom display names for metric if you add a new dictionary in the settings.py (:ref:`How to edit settings.py file`) file like in the example below. These display names are used when viewing and editing pricing rules in Fleio. .. code-block:: python METRICS_DISPLAY_NAME = { 'custom-metric-1': 'Custom metric 1', 'custom-metric-2': 'Custom metric 2' } After editing the ``settings.py`` file, you must restart all the ``fleio`` services (see :ref:`restart-fleio`). If everything was done correctly, you should be able to add pricing rules based on the resource type that you previously defined. Custom metrics based on external usage ====================================== In order to have pricing rules based on external usage (collected 3rd party apps), you will first have to add the data in Gnocchi. The manual steps are the following. You will have to automate this using your preferred method. The following information must be sent to all regions that you're using. 1. Create the resource in Gnocchi. For this you will need two things: * resource type: example: custom_1 * project ID: taken from Fleio using the api (/staffapi/openstack/projects) Then use the Gnocchi resource create command. In our example, the resource name will be custom_1, and we will use the resource type custom_1 and project_id project_1: .. code-block:: python openstack metric resource create --type custom_1 -a project_id:project_1 custom_1 +-----------------------+-------------------------------------------------------------------+ | Field | Value | +-----------------------+-------------------------------------------------------------------+ | created_by_project_id | c0be1ea4c2b94c68b42f2f43e68e49e5 | | created_by_user_id | 2b107eeb74224a17a7fe108384c4c044 | | creator | 2b107eeb74224a17a7fe108384c4c044:c0be1ea4c2b94c68b42f2f43e68e49e5 | | ended_at | None | | id | 03580e05-031f-587c-b599-ad67d3e9838f | | metrics | | | original_resource_id | custom_1 | | project_id | project_1 | | revision_end | None | | revision_start | 2019-10-17T09:21:54.396570+00:00 | | started_at | 2019-10-17T09:21:54.396552+00:00 | | type | custom_1 | | user_id | None | +-----------------------+-------------------------------------------------------------------+ 2. Create metrics for your resource: .. code-block:: python openstack metric create --resource-id custom_1 custom-metric-1 +---------------------+-------------------------------------------------------------------+ | Field | Value | +---------------------+-------------------------------------------------------------------+ | archive_policy/name | fleio_policy | | creator | 2b107eeb74224a17a7fe108384c4c044:c0be1ea4c2b94c68b42f2f43e68e49e5 | | id | e6bdbf4e-af88-4503-8aeb-d455819b7eb0 | | name | custom-metric-1 | | resource_id | 03580e05-031f-587c-b599-ad67d3e9838f | | unit | None | +---------------------+-------------------------------------------------------------------+ 2.1 Check the resource: .. code-block:: python openstack metric resource show 03580e05-031f-587c-b599-ad67d3e9838f +-----------------------+-------------------------------------------------------------------+ | Field | Value | +-----------------------+-------------------------------------------------------------------+ | created_by_project_id | c0be1ea4c2b94c68b42f2f43e68e49e5 | | created_by_user_id | 2b107eeb74224a17a7fe108384c4c044 | | creator | 2b107eeb74224a17a7fe108384c4c044:c0be1ea4c2b94c68b42f2f43e68e49e5 | | ended_at | None | | id | 03580e05-031f-587c-b599-ad67d3e9838f | | metrics | custom-metric-1: e6bdbf4e-af88-4503-8aeb-d455819b7eb0 | | original_resource_id | custom_1 | | project_id | project_1 | | revision_end | None | | revision_start | 2019-10-17T09:21:54.396570+00:00 | | started_at | 2019-10-17T09:21:54.396552+00:00 | | type | custom_1 | | user_id | None | +-----------------------+-------------------------------------------------------------------+ As you can see it has the correct project_id, and the correct metrics. 3. Add the measures to your previously created metric (metric_1) .. code-block:: python openstack metric measures add -m 2019-10-17T11:00:00+03:00@100.0 e6bdbf4e-af88-4503-8aeb-d455819b7eb0 3.1 Check the added measures using gnocchi measures show: .. code-block:: python openstack metric measures show e6bdbf4e-af88-4503-8aeb-d455819b7eb0 +---------------------------+-------------+-------+ | timestamp | granularity | value | +---------------------------+-------------+-------+ | 2019-10-17T11:00:00+03:00 | 3600.0 | 100.0 | | 2019-10-17T11:00:00+03:00 | 1800.0 | 100.0 | | 2019-10-17T11:00:00+03:00 | 300.0 | 100.0 | | 2019-10-17T11:10:00+03:00 | 300.0 | 100.0 | +---------------------------+-------------+-------+ After that please follow our guide described in the first part of the page. Please see :ref:`custom-metrics` I have used the same names / metrics as in the guide, so it will be easier for you to understand the workflow. Project IDs can be obtained by using the fleio api. The general workflow to get the project id is the following: .. code-block:: python {{ your-fleio-backend-url }}/staffapi/clients - get the client id {{ your-fleio-backend-url }}/staffapi/billing/services - get the service id based on client id {{ your-fleio-backend-url }}/staffapi/openstack/projects - get the project id based on service id