Metrics
Using metrics for security monitoring
Every project resource exposes metrics through the standard OpenStack Telemetry API (Gnocchi), at up to 5-minute granularity. They provide an external view of resource usage — sustained CPU, unexpected network throughput, unusual disk activity — to complement or cross-check the monitoring deployed on the instances, which remains the responsibility of the project owner. Metrics are collected with an Application Credential whose access is scoped through project roles (see Policies). To be notified on a threshold crossing instead of polling, see Alarming.
Display Available Metrics
Metrics are exposed per resource: instances, their disks and network interfaces, floating IP addresses, router public IPs and object storage accounts.
You can either display all available metrics in your project (and grep what you're looking for).
$ # The column "resource_id" corresponds to the id of the resource (instance, floating IP, etc.)
$ # In this example we only want the metrics for the VM instance with ID "36886229-f6f0-4784-a389-e78feab9b42a"
$ openstack metric list | grep "36886229-f6f0-4784-a389-e78feab9b42a"
| 460f10c7-2d5c-44e9-bd8c-3c8f4a4fe15c | ik-medium-rate | disk.ephemeral.size | GB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 67ad4165-0e79-4dd8-8a8c-83c6c74e176b | ik-medium-rate | memory | MB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 687c8882-86a9-46bb-9f3c-229636a0a222 | ik-medium-rate | memory.usage | MB | 36886229-f6f0-4784-a389-e78feab9b42a |
| 6fce8350-e606-4dbc-85dd-1a6df0edfb54 | ik-medium-rate | vcpus | vcpu | 36886229-f6f0-4784-a389-e78feab9b42a |
| dd88f6f3-400a-47eb-b633-69d55f356206 | ik-medium-rate | disk.root.size | GB | 36886229-f6f0-4784-a389-e78feab9b42a |
| e3aebc62-77d9-4516-a49a-2580686575e3 | ik-medium-rate | cpu | ns | 36886229-f6f0-4784-a389-e78feab9b42a |
| fe4f55e9-9603-4b66-9719-107b349f3f14 | ik-medium-rate | compute.instance.booting.time | sec | 36886229-f6f0-4784-a389-e78feab9b42a |
Or display available metrics for a specific resource:
$ openstack metric resource show 36886229-f6f0-4784-a389-e78feab9b42a -c metrics
+---------+---------------------------------------------------------------------+
| Field | Value |
+---------+---------------------------------------------------------------------+
| metrics | compute.instance.booting.time: fe4f55e9-9603-4b66-9719-107b349f3f14 |
| | cpu: e3aebc62-77d9-4516-a49a-2580686575e3 |
| | disk.ephemeral.size: 460f10c7-2d5c-44e9-bd8c-3c8f4a4fe15c |
| | disk.root.size: dd88f6f3-400a-47eb-b633-69d55f356206 |
| | memory.usage: 687c8882-86a9-46bb-9f3c-229636a0a222 |
| | memory: 67ad4165-0e79-4dd8-8a8c-83c6c74e176b |
| | vcpus: 6fce8350-e606-4dbc-85dd-1a6df0edfb54 |
+---------+---------------------------------------------------------------------+
Warning
Sometimes there are multiple resources for a given instance (for example an instance and its disks), so metrics can be linked to another resource than your instance id. In this case, you can use the search function:
$ # notice the "%" after the instance id in the search
$ openstack metric resource search "original_resource_id like '36886229-f6f0-4784-a389-e78feab9b42a%'"
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
| id | type | project_id | user_id | original_resource_id | started_at | ended_at | revision_start | revision_end | creator |
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
| 33d60f84-ce92-5dd8-a9bd-971bc221543d | instance_disk | d1440aa24a65411fb9bac2b842c8defa | 153855c4fa1b4415bdd768f88c559f72 | 36886229-f6f0-4784-a389-e78feab9b42a-vda | 2021-04-29T07:15:19.775934+00:00 | None | 2021-04-29T07:15:19.775950+00:00 | None | 35b89d95e2504576866fa01215247181:d1860cf28a264c6c8bbf0b5468c9b9c8 |
| 36886229-f6f0-4784-a389-e78feab9b42a | instance | d1440aa24a65411fb9bac2b842c8defa | 153855c4fa1b4415bdd768f88c559f72 | 36886229-f6f0-4784-a389-e78feab9b42a | 2021-04-29T07:12:06.362575+00:00 | None | 2021-04-29T08:00:26.396108+00:00 | None | 35b89d95e2504576866fa01215247181:d1860cf28a264c6c8bbf0b5468c9b9c8 |
+--------------------------------------+---------------+----------------------------------+----------------------------------+------------------------------------------+----------------------------------+----------+----------------------------------+--------------+-------------------------------------------------------------------+
Display Measures
Once you identified the metric you are interested in, you can display its measures. For example to display vcpus metric of our instance 36886229-f6f0-4784-a389-e78feab9b42a:
$ openstack metric measures show 6fce8350-e606-4dbc-85dd-1a6df0edfb54
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2021-04-29T06:00:00+00:00 | 21600.0 | 2.0 |
| 2021-04-29T12:00:00+00:00 | 21600.0 | 1.4 |
| 2021-04-29T18:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T00:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T06:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 21600.0 | 1.0 |
| 2021-04-29T07:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-29T08:00:00+00:00 | 3600.0 | 1.0 |
...
| 2021-04-30T11:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 3600.0 | 1.0 |
| 2021-04-29T09:00:00+00:00 | 300.0 | 1.0 |
| 2021-04-29T10:00:00+00:00 | 300.0 | 1.0 |
...
| 2021-04-30T11:00:00+00:00 | 300.0 | 1.0 |
| 2021-04-30T12:00:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+
The granularity varies over time, according to the archive policy of the metric (see next section).
Tip
There is a shortcut if you don't know the metric's id:
$ openstack metric measures show --resource-id 36886229-f6f0-4784-a389-e78feab9b42a vcpus
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2021-04-29T06:00:00+00:00 | 21600.0 | 2.0 |
...
| 2021-04-30T12:00:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+
You can also resample measures if you prefer to get a daily usage for example:
$ openstack metric measures show --start 2021-03-08T00:00:00 --granularity 3600 --resample 86400 --resource-id 3fa780c7-3897-4fd2-a177-120bc2a94b65 vcpus
+---------------------------+-------------+--------------------+
| timestamp | granularity | value |
+---------------------------+-------------+--------------------+
| 2021-03-08T01:00:00+01:00 | 86400.0 | 1.2285714285714284 |
| 2021-03-09T01:00:00+01:00 | 86400.0 | 1.0 |
| 2021-03-10T01:00:00+01:00 | 86400.0 | 1.0 |
+---------------------------+-------------+--------------------+
Tip
Timestamps are displayed in the local time zone of the client. Set TZ=UTC in the environment to get them in UTC.
Archive Policy
Each metric is stored according to an archive policy, shown by openstack metric list. The instance metrics above use ik-medium-rate:
$ openstack metric archive-policy show ik-medium-rate
+---------------------+-------------------------------------------------------------------+
| Field | Value |
+---------------------+-------------------------------------------------------------------+
| aggregation_methods | mean, rate:mean, std, count, max, min, sum |
| back_window | 0 |
| definition | - timespan: 62 days, 0:00:00, granularity: 1:00:00, points: 1488 |
| | - timespan: 732 days, 0:00:00, granularity: 6:00:00, points: 2928 |
| | - timespan: 7 days, 0:00:00, granularity: 0:05:00, points: 2016 |
| name | ik-medium-rate |
+---------------------+-------------------------------------------------------------------+
Note
The min aggregation method is available in region dc3-a only.