Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance and reduce latency of Transport.write by attempting to send data immediately if all write buffers are empty #619

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

tarasko
Copy link

@tarasko tarasko commented Aug 16, 2024

Current implementation of UVStream.write almost never sends data immediately. Instead the data is stored in the buffer and picked up later by uv_check callback.
This introduces unnecessary latency and CPU overhead.
Despite being a change only in UVStream it also directly benefit _SSLProtocolTransport.write latency since it uses UVStream.write to send ssl frames.

This PR increases RPS rate between echoclient and echoserver by roughly 10%.
echoclient --worker 1 --num 200000
echoserver --uvloop --proto

Because of this change a couple of test had to be tweaked.

  • get_write_buffer_size cannot reliably return how much data has been already buffered. It doesn't take into account the socket send buffer (which is in the kernel)
  • writing above limits does not necessarily cause pause_writing because again the data first goes to the socket send buffer

Please note that the current implementation of asyncio does the same thing. It tries to write directly to the socket, and, only if EWOULDBLOCK happens, the data is added to the buffer
https://github.com/python/cpython/blob/c13e7d98fb8581014a225b900b1b88ccbfc28097/Lib/asyncio/selector_events.py#L1065

@tarasko
Copy link
Author

tarasko commented Aug 20, 2024

This also increases aiohttp websocket echo performance by 20-25%

@tarasko tarasko changed the title Reduce Transport.write latency by attempting to send data immediately if all write buffers are empty Improve performance and reduce latency of Transport.write latency by attempting to send data immediately if all write buffers are empty Aug 29, 2024
@tarasko tarasko changed the title Improve performance and reduce latency of Transport.write latency by attempting to send data immediately if all write buffers are empty Improve performance and reduce latency of Transport.write by attempting to send data immediately if all write buffers are empty Aug 29, 2024
@tarasko
Copy link
Author

tarasko commented Aug 29, 2024

Hi @fantix, I saw you recently commited something to uvloop. Do you know if someone could take a look at this PR and maybe give a feedback?

@fantix
Copy link
Member

fantix commented Sep 3, 2024

Yeah I'll go through all issues/PRs again and include some in the final 0.21 release, it's just all taking some time unfortunately, thanks for the PR and your kind patience!

@tarasko tarasko closed this Sep 5, 2024
@tarasko tarasko reopened this Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants