Billing, Metering and Rating
Billing, Metering and Rating
Resources usage (vcpu, ram, disk, IP,... ) are accounted by the Openstack projects Ceilometer, Gnocchi and CloudKitty. They are stored into timeseries databases. These metrics can later be used for billing, alerting and/or auto-scaling. First, Ceilometer retrieve data on compute or controller side. For example instance has a flavor with 2 vcpus. Then, Gnocchi store data into a time series database. Finally, Cloudkitty bill the data, which means it will convert flavor to a certain price.
You will find below some typical use cases, please refer to the full documentation for other uses.
You can query these metrics at any time, to verify your resource consumption for example.
Billing
Billing is accounted with a virtual currency called ICU (Infomaniak Cloud Unit), you can find the currency converter here.
Example :
| Currency | Value | Infomaniak Cloud Unit (ICU) |
|---|---|---|
| CHF | 1 | 50 |
| EUR | 1 | 55.5 |
Installing The Clients
sudo apt install python3-gnocchiclient
sudo apt install python3-cloudkittyclient
Or using pip:
python3 -m pip install python-openstackclient gnocchiclient python-cloudkittyclient
Using The Clients
How much money have I been charged ?
This can be calculated by summing all the ICU ratings for the desired period and then dividing by the ICU's monetary equivalent in the wanted currency.
For example, using shell command subsitution to get the past date and some jq to make the calculation,
the following snippet calculates the CHF cost (50 ICUs equalling to CHF 1.-) of the last calculated hour :
openstack rating dataframes get -b $(date +'%Y-%m-%dT%H:00:00+00:00' -u --date="2 hours ago") -c Resources -f json \
| jq 'map(.Resources[].rating | tonumber) | add | . / 50'
Info
You may wonder why the command specify 2 hours ago instead of 1 hour. The reason to this is that the billing is calculated every hour, therefore the last hour has not been calculated yet. A more precise term would be last calculated hour.
Resources rated
To list all the resources rated you can use the command below.
$ openstack rating info metric list
+---------------------------------------+--------------+-----------------+
| Metric | Unit | Metadata |
+---------------------------------------+--------------+-----------------+
| instance_up | instance | ['flavor_name'] |
| instance_reserved | instance | ['flavor_name'] |
| image.size | GiB | [] |
| volume.size | GiB | ['volume_type'] |
| volume.snapshot.size | GiB | ['type'] |
| snapshot.size | GiB | ['type'] |
| ip.floating | ip | ['state'] |
| network.ports.ext-net1 | ip | [] |
| network.ports.router-gw-ext-floating1 | ip | [] |
| network.subnets.public | ip | [] |
| network.services.lb.loadbalancer | loadbalancer | [] |
| storage.objects.size | Gib | [] |
| storage.objects.outgoing.bytes | Gib | [] |
| volume.backup.size | Gib | [] |
+---------------------------------------+--------------+-----------------+
Get rating for a specific resource
The example below will display the amount of ICU consumed for the resource type storage.objects.outgoing.bytes.
Info
The 10TB free tier for storage.objects.outgoing.bytes isn't accounted here meaning you' ll start being charged if storage.objects.outgoing.bytes > 10000000000000.
$ openstack rating summary get -b $(date +'%Y-%m-%dT%H:00:00+00:00' -u --date="2022-01-01") -s storage.objects.outgoing.bytes
+----------------------------------+--------------------------------+-------------+---------------------+---------------------+
| Tenant ID | Resource Type | Rate | Begin Time | End Time |
+----------------------------------+--------------------------------+-------------+---------------------+---------------------+
| 638e9cb746924467a37b0bd1865ba3f5 | storage.objects.outgoing.bytes | 10349.48817 | 2022-01-01T00:00:00 | 2022-02-01T00:00:00 |
+----------------------------------+--------------------------------+-------------+---------------------+---------------------+
Get rating for all resources
The command below will display the amount of ICU consumed for each type of resources since 2022-01-01
$ for metric in $(openstack rating info metric list -f value -c Metric); do echo -n "$metric: "; openstack rating summary get -b $(date +'%Y-%m-%dT%H:00:00+00:00' -u --date="2022-01-01") -s $metric -f value -c Rate; done
instance_up: 874.49036
instance_reserved: 122.3492
image.size: 0.45206
volume.size: 3661.71077
volume.snapshot.size: 0
snapshot.size: 0
ip.floating: 483.33332
network.ports.ext-net1: 44.97717
network.ports.router-gw-ext-floating1: 146.80365
network.subnets.public: 0
network.services.lb.loadbalancer: 49.31507
storage.objects.size: 0.0
storage.objects.outgoing.bytes: 0.00291
volume.backup.size: 0
Get rating/hour
For example, command below shows the rating value per hour for the running VM instances for the period starting the 2021-11-01 :
$ openstack rating dataframes get -b 2021-11-01 -r instance_up --fit-width
+---------------------+---------------------+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
| Begin | End | Project ID | Resources |
+---------------------+---------------------+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
| 2021-11-01T00:00:00 | 2021-11-01T01:00:00 | 247f44b372c44205b59bfeacc2ea86bc | [{'rating': '0.402435', 'service': 'instance_up', 'desc': {'flavor_name': 'a2-ram4-disk20-perf1', 'id': |
| | | | '10736315-bd1a-406c-b90a-2f2fcce8674f', 'project_id': '247f44b372c44205b59bfeacc2ea86bc', 'tenant_id': |
| | | | '247f44b372c44205b59bfeacc2ea86bc'}, 'volume': '1', 'rate_value': '0.4024'}] |
| 2021-11-01T00:00:00 | 2021-11-01T01:00:00 | 247f44b372c44205b59bfeacc2ea86bc | [{'rating': '0.804869', 'service': 'instance_up', 'desc': {'flavor_name': 'a4-ram8-disk20-perf1', 'id': |
| | | | 'b8880e4e-9cb7-4419-b2cf-d20254ce53da', 'project_id': '247f44b372c44205b59bfeacc2ea86bc', 'tenant_id': |
| | | | '247f44b372c44205b59bfeacc2ea86bc'}, 'volume': '1', 'rate_value': '0.8049'}] |
| 2021-11-01T01:00:00 | 2021-11-01T02:00:00 | 247f44b372c44205b59bfeacc2ea86bc | [{'rating': '0.402435', 'service': 'instance_up', 'desc': {'flavor_name': 'a2-ram4-disk20-perf1', 'id': |
| | | | '10736315-bd1a-406c-b90a-2f2fcce8674f', 'project_id': '247f44b372c44205b59bfeacc2ea86bc', 'tenant_id': |
| | | | '247f44b372c44205b59bfeacc2ea86bc'}, 'volume': '1', 'rate_value': '0.4024'}] |
| 2021-11-01T01:00:00 | 2021-11-01T02:00:00 | 247f44b372c44205b59bfeacc2ea86bc | [{'rating': '0.804869', 'service': 'instance_up', 'desc': {'flavor_name': 'a4-ram8-disk20-perf1', 'id': |
| | | | 'b8880e4e-9cb7-4419-b2cf-d20254ce53da', 'project_id': '247f44b372c44205b59bfeacc2ea86bc', 'tenant_id': |
| | | | '247f44b372c44205b59bfeacc2ea86bc'}, 'volume': '1', 'rate_value': '0.8049'}] |
+---------------------+---------------------+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------
- Instance with id
10736315-bd1a-406c-b90a-2f2fcce8674fis/was rated 0.4024 ICU/hour - Instance with id
b8880e4e-9cb7-4419-b2cf-d20254ce53dais/was rated 0.8049 ICU/hour
Info
Please note the -r argument, which is used here to filter on the 'instance_up' resource type.
This only shows rating for the VM running cost (reserved CPU/RAM), but does not include HDD/GPU, which are also billed
when the instance is not running and are thus rated under the 'instance_reserved' resource type.
So, to have a full breakdown of your compute costs, you have to add up both resource types.
Apart from instance resources, several other resource types exist, each rating different metrics for your infrastructure. Here's a snippet to find out about all your project's used resource types, again using some jq :
$ openstack rating dataframes get -c Resources -f json | jq -r 'map(.Resources | map(.service)) | flatten | unique | .[]'
image.size
instance_reserved
instance_up
ip.floating
network.ports.ext-net1
network.ports.router-gw-ext-floating1
storage.objects.size
volume.size
Get resources usage for the past 30 days
$ openstack usage show
Usage from 2021-04-21 to 2021-05-20 on project 0c92a99218894f148b297bf539ce521a:
+---------------+------------+
| Field | Value |
+---------------+------------+
| CPU Hours | 999.54 |
| Disk GB-Hours | 13581.45 |
| RAM MB-Hours | 2047067.84 |
| Servers | 41 |
+---------------+------------+
Display Available Metrics
Metrics are available for all types of resources: VM instances, volumes, floating IP addresses, or anything else...
You can either display all available metrics in your project (and grep what you're looking for).
$ # The column "resource_id" corresponds to the id of your VM instance or volume or floating IP, etc...
$ # In this example we only want the metrics for the VM instance with ID "36886229-f6f0-4784-a389-e78feab9b42a"
$ openstack metric list | grep "36886229-f6f0-4784-a389-e78feab9b42a"
| 460f10c7-2d5c-44e9-bd8c-3c8f4a4fe15c | ik-medium-rate | disk.ephemeral.size | GB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 67ad4165-0e79-4dd8-8a8c-83c6c74e176b | ik-medium-rate | memory | MB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 687c8882-86a9-46bb-9f3c-229636a0a222 | ik-medium-rate | memory.usage | MB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 6fce8350-e606-4dbc-85dd-1a6df0edfb54 | ik-medium-rate | vcpus | vcpu | 36886229-f6f0-4784-a389-e78feab9b42a |
| dd88f6f3-400a-47eb-b633-69d55f356206 | ik-medium-rate | disk.root.size | GB | 36886229-f6f0-4784-a389-e78feab9b42a |
| e3aebc62-77d9-4516-a49a-2580686575e3 | ik-medium-rate | cpu | ns | 36886229-f6f0-4784-a389-e78feab9b42a |
| fe4f55e9-9603-4b66-9719-107b349f3f14 | ik-medium-rate | compute.instance.booting.time | sec | 36886229-f6f0-4784-a389-e78feab9b42a |
Or display available metrics for a specific resource:
$ openstack metric resource show 36886229-f6f0-4784-a389-e78feab9b42a -c metrics
+---------+---------------------------------------------------------------------+
| Field | Value |
+---------+---------------------------------------------------------------------+
| metrics | compute.instance.booting.time: fe4f55e9-9603-4b66-9719-107b349f3f14 |
| | cpu: e3aebc62-77d9-4516-a49a-2580686575e3 |
| | disk.ephemeral.size: 460f10c7-2d5c-44e9-bd8c-3c8f4a4fe15c |
| | disk.root.size: dd88f6f3-400a-47eb-b633-69d55f356206 |
| | memory.usage: 687c8882-86a9-46bb-9f3c-229636a0a222 |
| | memory: 67ad4165-0e79-4dd8-8a8c-83c6c74e176b |
| | vcpus: 6fce8350-e606-4dbc-85dd-1a6df0edfb54 |
+---------+---------------------------------------------------------------------+
Warning
Sometimes there are multiple resources for a given instance (for example a VM and its volume), so metrics can be linked to another resource than your instance id. In this case, you can use the search function:
$ # notice the "%" after the instance id in the search
$ openstack metric resource search "original_resource_id like '36886229-f6f0-4784-a389-e78feab9b42a%'"
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
| id | type | project_id | user_id | original_resource_id | started_at | ended_at | revision_start | revision_end | creator |
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
| 33d60f84-ce92-5dd8-a9bd-971bc221543d | instance_disk | d1440aa24a65411fb9bac2b842c8defa | 153855c4fa1b4415bdd768f88c559f72 | 36886229-f6f0-4784-a389-e78feab9b42a-vda | 2021-04-29T07:15:19.775934+00:00 | None | 2021-04-29T07:15:19.775950+00:00 | None | 35b89d95e2504576866fa01215247181:d1860cf28a264c6c8bbf0b5468c9b9c8 |
| 36886229-f6f0-4784-a389-e78feab9b42a | instance | d1440aa24a65411fb9bac2b842c8defa | 153855c4fa1b4415bdd768f88c559f72 | 36886229-f6f0-4784-a389-e78feab9b42a | 2021-04-29T07:12:06.362575+00:00 | None | 2021-04-29T08:00:26.396108+00:00 | None | 35b89d95e2504576866fa01215247181:d1860cf28a264c6c8bbf0b5468c9b9c8 |
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
Display Measures
Once you identified the metric you are interested in, you can display its measures. For example to display vcpus metric of our instance 36886229-f6f0-4784-a389-e78feab9b42a:
$ openstack metric measures show 6fce8350-e606-4dbc-85dd-1a6df0edfb54
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2021-04-29T06:00:00+00:00 | 21600.0 | 2.0 |
| 2021-04-29T12:00:00+00:00 | 21600.0 | 1.4 |
| 2021-04-29T18:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T00:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T06:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-29T07:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-29T08:00:00+00:00 | 3600.0 | 1.0 |
...
| 2021-04-30T11:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-29T09:00:00+00:00 | 300.0 | 1.0 |
| 2021-04-29T10:00:00+00:00 | 300.0 | 1.0 |
...
| 2021-04-30T11:00:00+00:00 | 300.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+
The granularity varies over time, according to the archive policy of the metric (see next section).
Tip
There is a shortcut if you don't know the metric's id:
$ openstack metric measures show --resource-id 36886229-f6f0-4784-a389-e78feab9b42a vcpus
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2021-04-29T06:00:00+00:00 | 21600.0 | 2.0 |
...
| 2021-04-30T12:00:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+
You can also resample measures if you prefer to get a daily usage for example:
$ openstack metric measures show --start 2021-03-08T00:00:00 --granularity 3600 --resample 86400 --resource-id 3fa780c7-3897-4fd2-a177-120bc2a94b65 vcpus
+---------------------------+-------------+--------------------+
| timestamp | granularity | value |
+---------------------------+-------------+--------------------+
| 2021-03-08T01:00:00+01:00 | 86400.0 | 1.2285714285714284 |
| 2021-03-09T01:00:00+01:00 | 86400.0 | 1.0 |
| 2021-03-10T01:00:00+01:00 | 86400.0 | 1.0 |
+---------------------------+-------------+--------------------+
Archive Policy
All metrics are stored according to the ik-medium-rate policy:
$ openstack metric archive-policy show ik-medium-rate
+---------------------+-------------------------------------------------------------------+
| Field | Value |
+---------------------+-------------------------------------------------------------------+
| aggregation_methods | mean, rate:mean, std, count, max, min, sum |
| back_window | 0 |
| definition | - timespan: 62 days, 0:00:00, granularity: 1:00:00, points: 1488 |
| | - timespan: 732 days, 0:00:00, granularity: 6:00:00, points: 2928 |
| | - timespan: 7 days, 0:00:00, granularity: 0:05:00, points: 2016 |
| name | ik-medium-rate |
+---------------------+-------------------------------------------------------------------+