On documentation for batch GET metric

[lijomonvarghese]

Before reporting an issue on Gnocchi, please be sure to provide all necessary
information.

Which version of Gnocchi are you using

gnocchi 3.3.1

How to reproduce your problem

It is not clear from document (https://gnocchi.xyz/rest.html) how to query to get measures of multiple metrics in a single query ( batch mode)

What is the result that you get

I can query for single metric. But the amount of measures /metrics to be fetched are quite large and not possible to use this method due to the time it takes for a query. I dont find a sample /syntax to be used for getting measures values in batch mode

What is result that you expected

As mentioned in #104
{'id1': [(timestamp, granularity, value)],
'id2': [(timestamp, granularity, value)]}

I should be able to issues a single query where i pass a set of metric IDs and i get response from GNOCCHI with the following format
{'id1': [(timestamp, granularity, value)],
'id2': [(timestamp, granularity, value)]}

[lijomonvarghese]

Further to the information added initially, i have the following comments as well, where i see that GET method for gathering measures for multiple metrics is not supported.?

I have found the following information
chungg@4d430f8
chungg@4d430f8#diff-6d3580ec72f1faa3da0d32d7e6129f92
features:

• |
  Support added to accept GET requests to /v1/batch/metrics/measures to
  retrieve multiple metrics in a single request. Metric ids should be passed
  in as query parameters in the format
  /v1/batch/metrics/measures?metric=uuid1&metric=uuid2. Additionally,
  start, stop, granularity and aggregation parameters may be
  passed in for additional filtering. All metrics must support the same
  granularity and aggregation if filtering on those attributes.

However when i tried to excute the bulk get , i get the message that GET method is not supported

curl -kH"X-Auth-Token: $token" "https://10.10.192.30:8041/v1/batch/metrics/measures/metric=e4774aca-25ac-4755-baf8-3e42596863e9&metric=eb4c5eaa-3246-4c6f-a629-bd67b0dd4aae"

<title>405 Method Not Allowed</title>

405 Method Not Allowed

The method GET is not allowed for this resource.

[chungg]

so that PR was reverted by c42848fc. it's way too long ago for me to remember why.

that said, you should probably be looking at the aggregates endpoint. something along the line of POST https://<host>:<port>/v1/aggregates with a payload similar like:

{"operations": ["metric", ["<metric_id1>" "mean"], ["<metric_id2>" "mean"], ... ]}

you can pass in start/stop as parameters to slice series.

if that works, feel free to contribute to the docs.

[chungg]

i should mention #441 as something to keep in mind. i don't think we ever looked much into it but it could be something you hit and if you do, it'd be good if you could provide details on when this becomes an issue.

[lijomonvarghese]

Many thanks for the reply and the suggestion to use aggregates .
But in my case , the aggregates query resulted in "The resource could not be found"

date

Fri Mar 13 02:31:14 TZINFO 2020

openstack metric measures show 03eb4735-2655-46cd-933f-ad6749d4b716 | tail -5

| 2020-03-12T12:00:00+00:00 | 600.0 | 6144.0 |
| 2020-03-12T13:00:00+00:00 | 600.0 | 6144.0 |
| 2020-03-12T14:00:00+00:00 | 600.0 | 6144.0 |
| 2020-03-12T15:00:00+00:00 | 600.0 | 6144.0 |
+---------------------------+-------------+--------+

openstack metric resource show 2be5b5ee-fa3f-4595-aac8-c21d8f4bcf76

+-----------------------+-------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------+
| created_by_project_id | be1dd9852fb7450a98ab04ebf636b41b |
| created_by_user_id | 95d8cde3321644119c0829385f0e9292 |
| creator | 95d8cde3321644119c0829385f0e9292:be1dd9852fb7450a98ab04ebf636b41b |
| ended_at | None |
| id | 2be5b5ee-fa3f-4595-aac8-c21d8f4bcf76 |
| metrics | cpu: 77717284-f4d3-4d28-b747-e2792ccea07b |
| | disk.allocation: 45e93fb9-e8a5-4225-9b47-648c835e2643 |
| | disk.capacity: 9f60d347-1f7e-4bc3-861a-1d4b49dec0ea |
| | disk.read.bytes: 343aafc5-8bc4-4edb-aa37-46d4cea5e3c2 |
| | disk.read.requests: 59ffded1-9346-4ba6-b1be-9fc1d9a3b26c |
| | disk.usage: e51b9d7c-659b-4f06-9f5f-a662bada0656 |
| | disk.write.bytes: b6315fe8-949b-43d9-974d-03476a89d19d |
| | disk.write.requests: 0ce7d3ae-2984-4966-8fa1-4c95264ecec9 |
| | memory.resident: 7e94e239-65cb-49ff-852b-f9f74fe3b87d |
| | memory.usage: 2b26e7c4-7310-4576-84aa-98d0eb0b95c6 |
| | memory: 03eb4735-2655-46cd-933f-ad6749d4b716 |
| original_resource_id | 2be5b5ee-fa3f-4595-aac8-c21d8f4bcf76 |
| project_id | b14666ac6c7a46d2bd91663d904021e0 |
| revision_end | None |
| revision_start | 2020-02-27T00:11:18.226963+00:00 |
| started_at | 2020-02-27T00:11:18.226947+00:00 |
| type | instance |
| user_id | 17a95d099df7441cb70f59bc5a1630e0 |
+-----------------------+-------------------------------------------------------------------+
curl -k -H "Content-Type: application/json" -H "X-Auth-Token: $token" -X POST "https://:8041/v1/aggregates?granularity=600" -d "{"operations":(metric "03eb4735-2655-46cd-933f-ad6749d4b716" "mean") }"

cat /tmp/se.json
{
"operations": "(metric memory mean)",
"resource_type": "instance",
"search": "original_resource_id='2be5b5ee-fa3f-4595-aac8-c21d8f4bcf76'"
}
root@pny9-nc-02-cic-1:~# curl -k -H "Content-Type: application/json" -H "X-Auth-Token: $token" -X POST "https://:8041/v1/aggregates?granularity=600" -d "@/tmp/se.json"

<title>404 Not Found</title>

404 Not Found

The resource could not be found.

Not sure if you may be able to help here. Thanks again.

[chungg]

i'll be honest, i'm not exactly sure what i'm looking at since the formatting is kind of wonky. are you saying both curl examples above are failing?

[lijomonvarghese]

Both the curl command examples are failing.
I am not sure the issue i having is due to an old version of GNOCCHI
#gnocchi --version
gnocchi 3.3.1

Could it be reason that "operations" are not available in 3.3.1?
How can i check if the v1/aggregates end point is available for the version of gnocchi i am using ?

[chungg]

that command is returning the client version, you'll need to check the packages you've installed for server version.

that said, it does seem like your usage of " would be wrong in the first call and your second call should be similar to https://gnocchi.xyz/rest.html#search-and-aggregate

[lijomonvarghese]

I did not have a luck by changing the payload of the HTTP post request to v1/aggregates.

I have searched on my installation and i find that v1/aggregates is not available in the client
#pwd
/usr/lib/python2.7/dist-packages/gnocchiclient/v1

egrep "v1/" *.py

archive_policy.py: url = "v1/archive_policy/"
archive_policy_rule.py: url = "v1/archive_policy_rule/"
capabilities.py: cap_url = "v1/capabilities/"
metric.py: metric_url = "v1/metric/"
metric.py: resource_url = "v1/resource/generic/%s/metric/"
metric.py: metric_batch_url = "v1/batch/metrics/measures"
metric.py: resources_batch_url = "v1/batch/resources/metrics/measures"
metric.py: return self._get("v1/aggregation/metric",
metric.py: "v1/aggregation/resource/%s/metric/%s?%s" % (
resource.py: url = "v1/resource/"
resource.py: url = "v1/search/resource/%s?%s" % (resource_type, qs)
resource_type.py: url = "v1/resource_type/"
status.py: url = "v1/status"

Also i tried to download the latest client from github(master) and extracted. I find a file called aggregates.py there, but it is not having the API end point v1/aggregates defined there ( like in other files metrics.py , resources.py etc)

Could this be a reason why the v1/aggregates query not working ?

[chungg]

you'll need to update to python-gnocchiclient>=6.0.0 if you want to use aggregates via client.

that said, if you're using curl and seeing errors you'll need to do some debugging on client/server to see what exactly you're doing wrong.

[lijomonvarghese]

Thanks for the suggestion. I did upgrade the gnocchiclient to latest version 7.0.5.
I ran the following operation
gnocchi.aggregates.fetch(operations="(metric ee6c0d11-208b-4275-bcff-b568d6cb5a08 mean)" ,details=True)
But i am still reeving HTTP response 404 ( resource not found)
I manually checked all the gnocchi end points against my gnocchi server and every other URL is working except the "v1/aggregates".

My package version are as below. Is my server gnocchi package version does not support v1/aggregates? From which version is it supported?

#apt list --installed | grep gnocchi

gnocchi-api/xenial,now 4.0.5-2u16.04+mcp2 all [installed]
gnocchi-common/xenial,now 4.0.5-2u16.04+mcp2 all [installed,automatic]
gnocchi-metricd/xenial,now 4.0.5-2u16.04+mcp2 all [installed]
gnocchi-statsd/xenial,now 4.0.5-2u16.04+mcp2 all [installed]
python-gnocchi/xenial,now 4.0.5-2u16.04+mcp2 all [installed,automatic]
python-gnocchiclient/xenial,now 3.3.1-2u16.04+mcp2 all [installed,automatic]

[lijomonvarghese]

I see from documents that , v1/aggregates is available in 4.1 and 4.2 only.
I see that my system has version 4.0 and i reckon that is the reason why , its failing for me.

IF i use the v1/aggregates to fetch measures for multiple metrics, how many metric can be passed at a time, like in the example below?
(metric (( ), ( ), ...))

[chungg]

yes, that would be why. :)

i don't think there's a limit. of course that request is sent to a single process so i wouldn't send it in one giant payload. you'll need to play around with minimising http requests but also parallelising requests.

[tobias-urdin]

Closing as there is no new information here.