Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrap outbound client thread stream exceptions #210

Merged
merged 7 commits into from
Aug 30, 2024
Merged

Conversation

FinleyMcIlwaine
Copy link
Collaborator

Unlike the server inbound/outbound and client outbound threads, http2 is aware of the client outbound thread. If the server disconnects, there is a race between the http2-thrown exception and the exception that will come from grapesy attempting to read. No matter who wins that race, we want to mark the exception with ServerDisconnected.

Also add sanity check tests for handling of exceptions/disconnects of client/server while there are open connections with ongoing concurrent calls.

Resolves #102

Unlike the server inbound/outbound and client outbound threads, `http2` is aware
of the client outbound thread. If the server disconnects, there is a race
between the `http2`-thrown exception and the exception that will come from
`grapesy` attempting to read. No matter who wins that race, we want to mark the
exception with `ServerDisconnected`.
@FinleyMcIlwaine FinleyMcIlwaine force-pushed the finley/102 branch 2 times, most recently from a49fdfa to 11abc7e Compare August 6, 2024 23:56
See module headers of `Test.Sanity.Disconnect` and `Test.Sanity.Exception`
test-grapesy/Test/Sanity/Disconnect.hs Show resolved Hide resolved
test-grapesy/Test/Sanity/Disconnect.hs Outdated Show resolved Hide resolved
test-grapesy/Test/Sanity/Disconnect.hs Outdated Show resolved Hide resolved
test-grapesy/Test/Util.hs Show resolved Hide resolved
test-grapesy/Test/Sanity/Disconnect.hs Outdated Show resolved Hide resolved
test-grapesy/Test/Sanity/Disconnect.hs Show resolved Hide resolved
test-grapesy/Test/Sanity/Exception.hs Outdated Show resolved Hide resolved
test-grapesy/Test/Sanity/Exception.hs Outdated Show resolved Hide resolved
test-grapesy/Test/Sanity/Exception.hs Show resolved Hide resolved
util/Network/GRPC/Util/Session/Channel.hs Show resolved Hide resolved
The 'ReconnectAfter' constructor of reconnect policy now holds an optional
'Server' argument, allowing reconnect policies to specify new server addresses
to attempt reconnection to. This makes it possible to fall back to redundant
servers, without needing to completely throw away a connection on the client.
We had a few spots where we were defining a `RawRPC "trivial" "trivial"` RPC
with `NoMetadata`, so just abstracted to deduplicate.
`rawTestServer` was doing nothing special anymore, and `forkServer` allows us to
query the server port as well. We keep `respondWith` around because it is a
useful abstraction for modeling broken servers.
No more of that "predicate" nonsense. Client steps are now described by a little
DSL and interpreted by the client.

Also enable `-Wmissing-export-lists`
Copy link
Collaborator

@edsko edsko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, almost there I think :)

src/Network/GRPC/Client/Connection.hs Outdated Show resolved Hide resolved
src/Network/GRPC/Client/Connection.hs Show resolved Hide resolved
loop :: ReconnectPolicy -> IO ()
loop remainingReconnectPolicy = do
loop :: Server -> ReconnectPolicy -> IO ()
loop server remainingReconnectPolicy = do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see that the actual implementation was as clean as this :)

-------------------------------------------------------------------------------}

foreign import ccall unsafe "kill" c_kill :: CInt -> CInt -> IO ()
foreign import ccall unsafe "exit" c_exit :: CInt -> IO ()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see, fair enough. Then let's add least add a comment here, because exiting the Haskell process this way also doesn't allow the RTS to shutdown cleanly. Of course, that's kind of the point here: this is modelling the server crashing for some reason.

--
-- We shouldn't need to handle this case, since our client never
-- explicitly sends 'NoMoreElems'. However, see discussion in the
-- ticket above.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the http2 stuff is merged, this ticket can be closed right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do that independently from this PR, of course. Remind me, do we rely on any other recent http2/http-semantics changes for this PR to go through?

Copy link
Collaborator

@edsko edsko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving in anticipation of the changes discussed above.

Reconnect policies can now specify whether they want to attempt reconnection
with the original server given to `withConnection`, the last server we attempted
connection with, or a new server specified by the policy itself.
@FinleyMcIlwaine FinleyMcIlwaine merged commit dec0e10 into main Aug 30, 2024
12 checks passed
@edsko edsko deleted the finley/102 branch September 3, 2024 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

An exception on an RPC call causes the whole connection to fail
2 participants