Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

influxdb endpoint sometimes raise a 500 #716

Open
sileht opened this issue Feb 4, 2018 · 3 comments
Open

influxdb endpoint sometimes raise a 500 #716

sileht opened this issue Feb 4, 2018 · 3 comments
Labels
bug

Comments

@sileht
Copy link
Member

sileht commented Feb 4, 2018 •
edited
Loading 

DEBUG [pifpaf.drivers] gnocchi-api[26055] output: 2018-02-04 16:39:33,934 [26413] DEBUG    gnocchi.rest.influxdb: Getting metrics from resource `bob'
DEBUG [pifpaf.drivers] gnocchi-api[26055] output: 2018-02-04 16:39:34,503 [26413] DEBUG    gnocchi.rest.influxdb: Add measures batch for 367 metrics
DEBUG [pifpaf.drivers] gnocchi-api[26055] output: 2018-02-04 16:39:34,544 [26413] CRITICAL root: Traceback (most recent call last):
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/dec.py", line 131, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     resp = self.call_func(req, *args, **self.kwargs)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/dec.py", line 196, in call_func
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return self.func(req, *args, **kwargs)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     response = req.get_response(self.application)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/request.py", line 1327, in send
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     application, catch_exc_info=False)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/request.py", line 1291, in call_application
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     app_iter = application(self.environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/paste/urlmap.py", line 216, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return app(environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/dec.py", line 131, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     resp = self.call_func(req, *args, **self.kwargs)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/dec.py", line 196, in call_func
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return self.func(req, *args, **kwargs)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     response = req.get_response(self.application)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/request.py", line 1327, in send
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     application, catch_exc_info=False)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/request.py", line 1291, in call_application
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     app_iter = application(self.environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/webob/exc.py", line 1162, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return self.application(environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/rest/app.py", line 123, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return self.app(environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/pecan/middleware/recursive.py", line 56, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return self.application(environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/pecan/core.py", line 840, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return super(Pecan, self).__call__(environ, start_response)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/pecan/core.py", line 683, in __call__
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     self.invoke_controller(controller, args, kwargs, state)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/pecan/core.py", line 574, in invoke_controller
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     result = controller(*args, **kwargs)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/rest/influxdb.py", line 254, in post_write
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     pecan.request.incoming.add_measures_batch(measures_to_batch)
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/incoming/__init__.py", line 129, in add_measures_batch
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     in six.iteritems(metrics_and_measures)))
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/utils.py", line 312, in parallel_map
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     return list(executor.map(lambda args: fn(*args), list_of_args))
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/collectd/.tox/py27/local/lib/python2.7/site-packages/concurrent/futures/_base.py", line 630, in map
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     fs = [self.submit(fn, *args) for args in itertools.izip(*iterables)]
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/incoming/__init__.py", line 128, in <genexpr>
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     for metric_id, measures
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:   File "/workspace/sileht/gnocchi/gnocchi/gnocchi/incoming/__init__.py", line 108, in _encode_measures
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:     dtype=TIMESERIES_ARRAY_DTYPE).tobytes()
DEBUG [pifpaf.drivers] gnocchi-api[26055] output: ValueError: invalid literal for float(): 18:58
DEBUG [pifpaf.drivers] gnocchi-api[26055] output:
DEBUG [pifpaf.drivers] gnocchi-api[26055] output: [pid: 26415|app: 0|req: 7/48] 127.0.0.1 () {30 vars in 472 bytes} [Sun Feb  4 16:41:09 2018] POST /v1/influxdb/write?consistency=any&db=telegraf => generated 0 bytes in 1187 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 5)
@jd
Copy link
Member

jd commented Feb 6, 2018 •
edited
Loading

Do you have an idea of what could have pushed 18:58 as a value?

With having the original line sent it's hard to understand how this can happen.

@jd jd added the bug label Feb 6, 2018
@jd jd added this to the 4.2 milestone Feb 6, 2018
@jd jd self-assigned this Feb 6, 2018
@sileht
Copy link
Member Author

sileht commented Feb 7, 2018

I just "apt install telegraf" and put the gnocchi endpoint for my HTTP/1.1/keepalive fix and see this. I will try to catch the line.

@jd jd removed this from the 4.2 milestone Mar 26, 2018
@jd
Copy link
Member

jd commented Apr 17, 2018 

[17:04:53]  <seuf>	I'm trying to use the system input plugin and it send metrics like 'system,host=fedora-26 uptime_format="5 days,  6:52" 1523976940000000000'
[17:05:21]  <seuf>	but the uptime_format is not parsable by gnocchi-api :
[17:05:22]  <seuf>	  File "/usr/lib/python2.7/site-packages/gnocchi/incoming/__init__.py", line 108, in _encode_measures
[17:05:25]  <seuf>	    dtype=TIMESERIES_ARRAY_DTYPE).tobytes()
[17:05:28]  <seuf>	ValueError: invalid literal for float(): 5 days,  6:23

@jd jd removed their assignment Oct 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jd @sileht