Python driver fails with unexpected bytes

[ssut]

I'm running tasks using celery in Python with python rethinkdb driver. I followed Ten-minute guide with Rethinkdb and Python and it seems to be running fine for a while, but it fails with UnicodeDecodeError after some time.

[2017-06-18 16:24:00,102: WARNING/PoolWorker-6] b'{"deleted":0,"errors":0,"generated_keys":["cef00e30-fdb1-443e-8313-54ab6da8812d"],"inserted":1,"replaced":0,"skipped":0,"unchanged":0}]}\x02\x00\x00\x00\x00\x00\x00\x00\x94\x00\x00\x00'
[2017-06-18 16:24:00,105: ERROR/PoolWorker-6] Task tasks.crawl[ddaffbc4-e56d-4da8-8304-c5abd243be79] raised unexpected: UnicodeDecodeError('utf-8', b'{"deleted":0,"errors":0,"generated_keys":["cef00e30-fdb1-443e-8313-54ab6da8812d"],"inserted":1,"replaced":0,"skipped":0,"unchanged":0}]}\x02\x00\x00\x00\x00\x00\x00\x00\x94\x00\x00\x00', 144, 145, 'invalid start byte')
Traceback (most recent call last):
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/celery/app/trace.py", line 367, in trace_task
  R = retval = fun(*args, **kwargs)
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/celery/app/trace.py", line 622, in __protected_call__
  return self.run(*args, **kwargs)
  File "/home/ssut/dev/.../tasks.py", line 215, in crawl
  r.table('logs').insert(log).run(conn)
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/rethinkdb/ast.py", line 123, in run
  return c._start(self, **global_optargs)
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/rethinkdb/net.py", line 626, in _start
  return self._instance.run_query(q, global_optargs.get('noreply', False))
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/rethinkdb/net.py", line 459, in run_query
  res = self._read_response(query)
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/rethinkdb/net.py", line 507, in _read_response
  self._parent._get_json_decoder(query))
  File "/home/ssut/.local/share/virtualenvs/server-NO78Iaa1/lib/python3.6/site-packages/rethinkdb/net.py", line 74, in __init__
  json_str = json_str.decode('utf-8')
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 144: invalid start byte

The first line is a hint I modified to check where the error comes from:

I think the root cause of this issue is that the part of the socket receives response from the rethinkdb server does the wrong behavior.

[jhay06]

i'm getting this error when a changefeed or the .changes() method shares the same connection of the existing connection to rethinkdb that responsible to insert the data .

[lsabi]

@jhay06 do you have a reproducible example? The code for the python driver has changed a lot in almost 8 years

[jhay06]

i dont have access on my laptop now . but the current rethinkdb python drivers still working as off now.

to remove the error and plan to run the change feed and insert method on the same app .you should create new connection for change feed and run it on a thread . dont share it to the existing connection

[lsabi]

If you share the same connection, then it works? Sorry for the questions, but I need to completely understand the environment/setup in order to restrict the area of code where the error is triggered.

[jhay06]

no , changefeed must be on its own connection instance exclusive.

[lsabi]

Ok. On what version of the driver are you? What version of rdb are you using?