Well-Typed are delighted to announce the release of grapesy
(Hackage, GitHub), an industial strength
Haskell library providing support for gRPC, a modern open source
high performance Remote Procedure Call (RPC) framework developed by Google. The
library has the following features:
Parametric in the choice of message format; Protobuf is the
most common choice for gRPC and is of course supported, as is JSON. There is
also support for general binary (“raw”) messages, and adding additional
formats is easy and can be done without modifying grapesy itself.
Client-side and server-side support for the Protobuf common communication
patterns: non-streaming, client-side streaming,
server-side streaming, and bidirectional streaming. Use of these patterns is
independent of the choice of message encoding, and is strictly optional.
For the specific case of Protobuf, support for
Protobuf rich errors.
Support for metadata: request initial metadata,
response initial metadata, and response trailing metadata.
Support for all the common gRPC compression
algorithms: gzip and zlib (both through the zlib package),
as well as snappy (through a new package snappy-c,
developed for this purpose). Bespoke compression algorithms can also be used,
and compression can be disabled on a per-message
basis.
Support for both unencrypted and encrypted connections (TLS).
Support for cancellation, both through
deadlines/timeouts (server-side cancellation) as well as through
terminating a RPC early (client-side cancellation).
Flow control: we are careful to use back-pressure
to limit traffic, ultimately relying on HTTP2 flow control (which can be
adjusted through the HTTP2Settings, primarily
the stream window size and the connection window size).
Support for Wait-for-Ready, where the
connection to a server can be (re)established in the background, rather than
an RPC failing if the server is not immediately available. Note that this
must be enabled explicitly (as per the
spec).
Asynchronous design: operations happen in the background whenever possible
(opening a connection, initiating an RPC, sending a message), and exceptions
are only raised when directly interacting with those background processes.
For example, when a client disconnects from the server, the corresponding
handler will only get an exception if it attempts any further communication
with that client. This is particularly important in RPC servers, which may
need to complete certain operations even if the client that requested those
operations did not stick around to wait for them.
Type safety: the types of inputs (messages sent from the client to the server)
and outputs (messages from the server to the client), as well as the types of
the request and response metadata, are all determined from the choice of a
specific RPC. In addition, for Protobuf servers we can guarantee at the
type-level that all methods are handled (or explicitly declared as
unsupported).
Extensive documentation: this blog post contains a number of tutorials that
highlight the various parts of grapesy, and the Haddock documentation is
comprehensive.
The library is designed to be robust:
Exception safety: all exceptions, in both client and in server contexts, are
caught and handled in context appropriate ways; they are never simply “lost”.
Server-side exceptions are reported as gRPC errors on the client; handlers can
also throw any of the standard gRPC
errors.
Deals correctly with broken deployments (clients or servers that do not
conform to the gRPC specification). This includes things such as dealing with
non-200 HTTP status codes, correctly responding to
unsupported content types (for which the gRPC spec mandates a different
resolution on servers and clients), dealing with servers that don’t respect
timeouts, etc.
Native Haskell library (does not bind to any C or C++ libraries).
Comes with a comprehensive test suite, which has been instrumental in
achieving high reliability, as well as finding problems elsewhere in the
network stack; as part of the development of grapesy we have also made
numerous improvements to http2 and related libraries. Many thanks to Kazu Yamamoto for being so receptive to all our PRs and
willing to discuss all the issues we found, as well as his hard work on these
core infrastructure libraries!
No memory leaks: even under stress conditions, memory usage is completely
flat in both the client and the server.
Good performance, on par with the official Java
implementation.
Developing a library of this nature is a significant investment, and so
Well-Typed is thankful to Anduril for sponsoring the work.
Quickstart
In this section we explain how to get started, in the style of the
official Quickstart guide. You can also use the
Quickstart example as a basic template for your own gRPC applications.
Neither gRPC nor grapesy requires the use of Protobuf, but it is the most
common way of using gRPC, and it is used by both the Quickstart tutorial as well
as the Basics tutorial. You will therefore need to install the
protobuf buffer compiler protoc, which can usually be done using your system’s
package manager; see Protobuf Buffer Compiler
Installation for details.
Download the example
If you want to work through this quick start, you will need to clone the
grapesy repository:
$ git clone https://github.com/well-typed/grapesy.git
$ cd grapesy/tutorials/quickstart
Run a gRPC application
From the grapesy/tutorials/quickstart directory, run the server
$ cabal run greeter_server
From another terminal, run the client:
$ cabal run greeter_client
If all went well, you should see the server responding to the client with
Proto {message: "Hello, you"}
Update the gRPC service
Now let’s try to add another method to the Greeter service. This service is
defined using protocol buffers; for an introduction to gRPC in general and
Protobuf specifically, you may wish to read the official Introduction to
gRPC; we will also see more examples of Protobuf below in the
Basics tutorial. You can find the definition for the quickstart
tutorial in tutorials/quickstart/proto/helloworld.proto:
syntax = "proto3";
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
Let’s add another method to this service, with the same request and response
types:
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
// Sends another greeting
rpc SayHelloAgain (HelloRequest) returns (HelloReply) {}
}
Generate gRPC code
The example is set up to use a custom Cabal setup script to automatically
compile the proto definitions; see
proto-lens-setup for a detailed discussion on how to
do this. If you prefer not to use custom setup scripts in your own projects, it
is also possible to run the Protobuf compiler manually; see section Manually
running the protocol compiler of the
proto-lens-protoc documentation.
This means that to re-run the Protobuf compiler, it suffices to build either
the client or the server; let’s attempt to build the server:
$ cabal build greeter_server
You should see a type error:
app/Server.hs:13:7: error: [GHC-83865]
• Couldn't match type: '[]
with: '[Protobuf Greeter "sayHelloAgain"]
This is telling you that the server is incomplete: we are missing a handler
for the new sayHelloAgain method.
Update the server
To update the server, edit Server.hs and add:
sayHelloAgain :: Proto HelloRequest -> IO (Proto HelloReply)
sayHelloAgain req = do
let resp = defMessage & #message .~ "Hello again, " <> req ^. #name
return resp
Then update methods to list the new handler:
methods :: Methods IO (ProtobufMethodsOf Greeter)
methods =
Method (mkNonStreaming sayHello)
$ Method (mkNonStreaming sayHelloAgain)
$ NoMoreMethods
Update the client
Unlike the server, the change to the service definition does not require
changes to the client. The server must implement the new method, but the client
does not have to call it. Of course, it is more interesting when it does, so
let’s add another call to Client.hs:
withConnection def server $ \conn -> do
let req = defMessage & #name .~ "you"
resp <- nonStreaming conn (rpc @(Protobuf Greeter "sayHello")) req
print resp
resp2 <- nonStreaming conn (rpc @(Protobuf Greeter "sayHelloAgain")) req
print resp2
Run
After restarting greeter_server, running greeter_client should now output
Proto {message: "Hello, you"}
Proto {message: "Hello again, you"}
Basics
In this section we delve a little deeper, following the official Basics
tutorial, which introduces the RouteGuide service.
From the official docs:
Our example is a simple route mapping application that lets clients get
information about features on their route, create a summary of their route,
and exchange route information such as traffic updates with the server and
other clients.
You can find the example in the tutorials/basics directory of the
grapesy repo.
Defining the service
The RouteGuide example illustrates the four different kinds of communication
patterns that Protobuf services can have. You can find the full service
definition in tutorials/basics/proto/route_guide.proto:
Non-streaming: client sends a single input, server replies with a single
output:
// Obtains the feature at a given position.
rpc GetFeature(Point) returns (Feature) {}
Server-side streaming: client sends a single input, server can respond with
any number of outputs:
// Obtains the Features available within the given Rectangle.
rpc ListFeatures(Rectangle) returns (stream Feature) {}
Client-side streaming: client can send any number of inputs, after which
the server responds with a single output:
// Accepts a stream of Points on a route being traversed, returning a
// RouteSummary when traversal is completed.
rpc RecordRoute(stream Point) returns (RouteSummary) {}
Bidirectional streaming: the client and the server can exchange messages
at will:
// Accepts a stream of RouteNotes sent while a route is being traversed,
// while receiving other RouteNotes (e.g. from other users).
rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
There is explicit support in grapesy for these four communication patterns,
both for defining servers and for defining clients. In addition, there is a
lower-level API which provides more control over the communication; we will
see some examples in Beyond the basics.
Generated code
As in the Quickstart, we have set things up in the
example to automatically generate Haskell code from the .proto definition.
There is however one more thing that we need to take care of, which we glossed
over previously. The .proto definition is sufficient to determine the types
of the methods of the service, their arguments, and their results. But it does
not say anything about the type of any metadata. We don’t need any metadata
in this example, so we can declare the following module:
module Proto.API.RouteGuide (
module Proto.RouteGuide
) where
import Network.GRPC.Common
import Network.GRPC.Common.Protobuf
import Proto.RouteGuide
type instance RequestMetadata (Protobuf RouteGuide meth) = NoMetadata
type instance ResponseInitialMetadata (Protobuf RouteGuide meth) = NoMetadata
type instance ResponseTrailingMetadata (Protobuf RouteGuide meth) = NoMetadata
This re-exports module Proto.RouteGuide (which was generated), along with
three type family instances that indicate that none of the methods of the
RouteGuide require metadata. We will see an example of using
metadata later.
Proto wrapper
In the repository you will find an implementation of the logic of the
RouteGuide example as a collection of pure functions; see
tutorials/basics/src/RouteGuide.hs. For example, the type of the function that
looks up which feature exists at a particular point, given the example database
of features, is given by:
featureAt :: DB -> Proto Point -> Maybe (Proto Feature)
The precise implementation is not very important for our purposes here, but we
should discuss that Proto wrapper. This is a type-level
marker that explicitly identifies Protobuf values. Such values don’t behave like
regular Haskell values; for example, record fields always have defaults, enums
might have unknown values, etc. The idiomatic way of accessing fields of a
Proto value is using a lens access and an
(overloaded) label; for example, the following
expression extracts a field #location from a feature (f :: Proto Feature):
To construct a Proto value you first create an empty value using
defMessage, and then update individual fields with a lens
update. For example, here is how we might construct a
Proto RouteSummary:
defMessage
& #pointCount .~ ..
& #featureCount .~ ..
& #distance .~ ..
& #elapsedTime .~ ..
Everything required to work with Protobuf values is (re-)exported from
Network.GRPC.Common.Protobuf. In addition,
Network.GRPC.Common.Protobuf.Any provides
functionality for working with the Protobuf Any type.
Implementing the server
We can use the type checker to help us in the development of the server. We
know that we want to implement the methods of the RouteGuide service; if we
define
methods :: DB -> Methods IO (ProtobufMethodsOf RouteGuide)
methods db = _
the type checker will tell us that it expects something of this type:
_ :: Methods IO [
Protobuf RouteGuide "getFeature"
, Protobuf RouteGuide "listFeatures"
, Protobuf RouteGuide "recordRoute"
, Protobuf RouteGuide "routeChat"
]
We can therefore refine methods to
methods :: DB -> Methods IO (ProtobufMethodsOf RouteGuide)
methods db =
Method _getFeature
$ Method _listFeatures
$ Method _recordRoute
$ Method _routeChat
$ NoMoreMethods
at which point the type checker informs us:
_getFeature :: ServerHandler' NonStreaming IO (Protobuf RouteGuide "getFeature")
_listFeatures :: ServerHandler' ServerStreaming IO (Protobuf RouteGuide "listFeatures")
_recordRoute :: ServerHandler' ClientStreaming IO (Protobuf RouteGuide "recordRoute")
_routeChat :: ServerHandler' BiDiStreaming IO (Protobuf RouteGuide "routeChat")
We can therefore refine once more to
methods :: DB -> Methods IO (ProtobufMethodsOf RouteGuide)
methods db =
Method (mkNonStreaming $ _getFeature)
$ Method (mkServerStreaming $ _listFeatures)
$ Method (mkClientStreaming $ _recordRoute)
$ Method (mkBiDiStreaming $ _routeChat)
$ NoMoreMethods
The resulting types will depend on the communication pattern (non-streaming,
client-side streaming, etc.). We will discuss them one by one.
Non-streaming RPC
The first method is a non-streaming RPC, for which the type checker infers:
_getFeature :: Proto Point -> IO (Proto Feature)
That is, we are given a point of interest, and must return “the” feature at
that point. We will also need the database of features. The implementation is
straight-forward, and essentially just wraps the pure function featureAt:
getFeature :: DB -> Proto Point -> IO (Proto Feature)
getFeature db p = return $ fromMaybe (defMessage & #location .~ p) (featureAt db p)
The only minor complication here is that we need to construct some kind of
default location for when there is no feature found at point p.
Server-side streaming
For server-side streaming we are given the input from the client, along with
a function that we can use to send outputs back to the client:
_listFeatures :: Proto Rectangle -> (NextElem (Proto Feature) -> IO ()) -> IO ()
NextElem is similar to Maybe:
data NextElem a = NoNextElem | NextElem !a
but with a more specialized API. For example, it
offers
forM_ :: Monad m => [a] -> (NextElem a -> m ()) -> m ()
which will invoke the specified callback NextElem x for all x in the list,
and then invoke the callback once more with NoNextElem. We can use this
to implement listFeatures:
listFeatures :: DB -> Proto Rectangle -> (NextElem (Proto Feature) -> IO ()) -> IO ()
listFeatures db r send = NextElem.forM_ (featuresIn db r) send
Client-side streaming
For client-side streaming we are given a function to receive inputs from the
client, and must produce a single output to be sent back to the client:
_recordRoute :: IO (NextElem (Proto Point)) -> IO (Proto RouteSummary)
To implement it, we can use another function from the
NextElem API:
collect :: Monad m => m (NextElem a) -> m [a]
The only other complication is that the function which constructs the
RouteSummary also wants to know how long it took to collect all points:
recordRoute :: DB -> IO (NextElem (Proto Point)) -> IO (Proto RouteSummary)
recordRoute db recv = do
start <- getCurrentTime
ps <- NextElem.collect recv
stop <- getCurrentTime
return $ summary db (stop `diffUTCTime` start) ps
Bidirectional streaming
For bidirectional streaming finally we get two functions: one to receive
inputs from the client, and one to send outputs back to the client:
_routeChat ::
IO (NextElem (Proto RouteNote))
-> (NextElem (Proto RouteNote) -> IO ())
-> IO ()
The implementation is straight-forward and does not require any new grapesy
features; you can find it in tutorials/basics/app/Server.hs.
Top-level application
The main server application then looks like this:
main :: IO ()
main = do
db <- getDB
runServerWithHandlers def config $ fromMethods (methods db)
where
config :: ServerConfig
config = ServerConfig {
serverInsecure = Just (InsecureConfig Nothing defaultInsecurePort)
, serverSecure = Nothing
}
The first parameter to
runServerWithHandlers are the server
parameters. The most important parameters to
consider are serverTopLevel and
serverExceptionToClient. These two
are related, and describe how to deal with exceptions:
serverTopLevel says what to do with
exceptions server-side; by default it simply prints them to stderrserverExceptionToClient says what
information to include in the error sent to the client; by default it calls
displayException. You may wish to override this if you are concerned about
leaking security sensitive information.
Implementing the client
You can find the complete client in tutorials/basics/app/Client.hs.
Connecting to the server
Before we can make any RPCs, we have to connect to the
server:
main :: IO ()
main =
withConnection def server $ \conn -> do
..
where
server :: Server
server = ServerInsecure $ Address "127.0.0.1" defaultInsecurePort Nothing
The first argument are the connection parameters, the
most important of which is probably the reconnection
policy which (amongst other things) is used
to enable Wait-for-Ready semantics.
Simple RPC
We already saw how to make a simple non-streaming RPC in the quickstart:
getFeature :: Connection -> IO ()
getFeature conn = do
let req = defMessage
& #latitude .~ 409146138
& #longitude .~ -746188906
resp <- nonStreaming conn (rpc @(Protobuf RouteGuide "getFeature")) req
print resp
We construct a request, do the RPC, and print the response.
Server-side streaming
When we make a server-side streaming RPC, we are given a function we can call
to get all of the server outputs:
listFeatures :: Connection -> IO ()
listFeatures conn = do
let req = ..
serverStreaming conn (rpc @(Protobuf RouteGuide "listFeatures")) req $ \recv ->
NextElem.whileNext_ recv print
Here we are using another function from the
NextElem API, in a sense dual to the one we used
server-side; for comparison, both types:
forM_ :: Monad m => [a] -> (NextElem a -> m ()) -> m ()
whileNext_ :: Monad m => m (NextElem a) -> (a -> m b) -> m ()
Client-side streaming
To make a client-side streaming RPC, we are given a function that we can use
to send inputs to the server; once we are done sending all inputs, we then
receive the final (and only) output from the server:
recordRoute :: Connection -> IO ()
recordRoute conn = do
resp <- clientStreaming_ conn (rpc @(Protobuf RouteGuide "recordRoute")) $ \send -> do
replicateM_ 10 $ do
let p = (db !! i) ^. #location
send $ NextElem p
threadDelay 500_000 -- 0.5 seconds
send NoNextElem
print resp
Bidirectional streaming
Finally, for bidirectional streaming we are given two functions, one to send,
and one to receive. In this particular case, we can first send all inputs and
then receive all outputs, but in general these can be interleaved in any order:
routeChat :: Connection -> IO ()
routeChat conn = do
biDiStreaming conn (rpc @(Protobuf RouteGuide "routeChat")) $ \send recv -> do
NextElem.forM_ messages send
NextElem.whileNext_ recv print
where
messages = ..
See also The Haskell Unfolder, episode 27: duality for a
more in-depth look into the duality between these various communication
patterns.
End of output
When we discussed the client-side implementation of a client-side streaming
RPC, we used function
clientStreaming_:
clientStreaming_ ::
..
-> ( (NextElem (Input rpc) -> m ())
-> m ()
)
-> m (Output rpc)
The callback is given a function (which we called send) to send outputs to the
server. The problem with this approach is that it’s possible to forget to call
this function; in particular, it’s quite easy to forget the final
to indicate to the server that there is no further input coming. In some cases
iteration functions such as NextElem.forM_ can
take care of this, but this could also result in the opposite problem, calling
send on a NextElem after NoNextElem has already been sent.
In short: make sure to send NoNextElem in clients or servers that
stream values to their peer:
If you forget to do this in a server handler, grapesy will assume this is a
bug and throw a HandlerTerminated exception, which will be reported as a
gRPC exception with an unknown error on the client.
If you forget to do this in a client, grapesy will assume that you intend to
cancel the RPC. The server will see call closed
suddenly, and on the client this will
result in a gRPC exception with “cancelled” as the error.
Sending more elements after sending NoNextElem will result in SendAfterFinal
exception.
Side note. In principle it is possible to give clientStreaming_ a
different type:
-- Alternative definition, not actually used in grapesy
clientStreaming_ ::
..
-> m (NextElem (Input rpc))
-> m (Output rpc)
In this style there is no callback at all; instead, we must provide an action
that produces the next element one by one, and the library will ensure that the
function is called repeatedly until it returns NoNextElem. This amounts to
inversion of control: you don’t call a function to send each value, but the
library calls you to ask what the next value to be sent is. This provides
stronger guarantees that the communication pattern is implemented correctly, but
we deemed the cost too high: it results in quite an awkward programming model.
Of course, if you want to, nothing stops you from defining such an API on top
of the API offered by grapesy.
Beyond the basics
In this section we describe some of the more advanced features of grapesy.
Using the low-level API
Both the Quickstart and the Basics tutorial used the
StreamType API, which captures the four different communication patterns (aka
streaming types) used in Protobuf, both on the
server and on the
client: non-streaming, server-side streaming,
client-side streaming, and bidirectional streaming. Although these four
communication patterns originate in Protobuf, in grapesy they are not
Protobuf specific and can also be used with other message encodings.
The high-level API will probably suffice for the vast majority of gRPC
applications, but not quite all, and grapesy also offers a low-level API. The
most important reasons to use the low-level API instead are:
Making sure that the final message is marked as final; we discuss this in more
detail in this section in Final elements.
Sending and receiving metadata; we will discuss this in detail in
the next section Using metadata.
Preference: some people may simpler prefer the style of the low-level API
over the high-level API.
Although the use of the low-level API does come with some responsibilities that
are taken care of for you in the high-level API, it is not significantly
more difficult to use.
Final elements
When we discussed the high-level API, we saw the
NextElem type. The low-level API uses
StreamElem instead; here they are side by side:
data NextElem a = NoNextElem | NextElem !a
data StreamElem b a = NoMoreElems !b | StreamElem !a | FinalElem !a !b
There are two differences here:
When there are no more elements, we record an additional value. This is the
metadata to be sent or received after the final element. We will see an
example of this below; for RouteGuide this metadata will
always be NoMetadata, which is a trivial type isomorphic to ():
data NoMetadata = NoMetadata
The final element can be marked as final, rather than requiring a separate
NoMoreElems value. This may feel like an insignificant difference, but
although it is a technicality, in some cases it’s an important
technicality.
To understand the need for marking the final element, we need to understand that
gRPC messages are transferred over HTTP2 DATA frames.
It’s not necessarily true that one frame corresponds to one message, but let’s
for the sake of simplicity assume that it is. Then in order to send 3 messages,
we have two options:
| frame 1: message 1 |
frame 1: message 1 |
| frame 2: message 2 |
frame 2: message 2 |
| frame 3: message 3 |
frame 3: message 3, marked END_STREAM |
frame 4: empty, marked END_STREAM |
|
corresponding to
[StreamElem 1, StreamElem 2, StreamElem 3, NoMoreElems NoMetadata]
and [StreamElem 1, StreamElem 2, FinalElem 3 NoMetadata]
respectively. This matters because some servers report an error if they receive
a message that they expect will be the final message, but the corresponding
HTTP2 DATA frame is not marked END_STREAM. This is not completely
unreasonable: after all, waiting to receive the next DATA frame might be a
blocking operation.
This is particularly important in cases where a server (or client) only expects
a single message (non-streaming, client-side streaming, expecting a single
output from the server, or server-side streaming, expecting a single input from
the client). It is much less critical in other situations, which is why the
high-level API gets away with using NextElem instead of StreamElem (which it
uses only when multiple messages are expected).
On the server
To use the low-level API on the server, you can either use RawMethod to use
the low-level API for some (or all) of the methods of an API, or you avoid the
use of fromMethods altogether. The latter option is primarily useful if you
don’t have a type-level description of your service available. If you do,
the first option is safer:
methods :: DB -> Methods IO (ProtobufMethodsOf RouteGuide)
methods db =
RawMethod (mkRpcHandler $ getFeature db)
$ RawMethod (mkRpcHandler $ listFeatures db)
$ RawMethod (mkRpcHandler $ recordRoute db)
$ RawMethod (mkRpcHandler $ routeChat )
$ NoMoreMethods
It is also possible to use the high-level API for most methods, and escape to
the low-level API for those methods that need it.
Unlike with the high-level API, the signature of all handlers that use the
low-level API is the same:
getFeature :: DB -> Call (Protobuf RouteGuide "getFeature") -> IO ()
listFeatures :: DB -> Call (Protobuf RouteGuide "listFeatures") -> IO ()
recordRoute :: DB -> Call (Protobuf RouteGuide "recordRoute") -> IO ()
routeChat :: Call (Protobuf RouteGuide "routeChat") -> IO ()
The most important two functions for communication on
the server are recvInput and
sendOutput:
recvInput :: Call rpc -> IO (StreamElem NoMetadata (Input rpc))
sendOutput :: Call rpc -> StreamElem (ResponseTrailingMetadata rpc) (Output rpc) -> IO ()
For convenience there are also some derived functions available; for example,
here is getFeature again, now using the low-level API:
getFeature :: DB -> Call (Protobuf RouteGuide "getFeature") -> IO ()
getFeature db call = do
p <- recvFinalInput call
sendFinalOutput call (
fromMaybe (defMessage & #location .~ p) (featureAt db p)
, NoMetadata
)
The StreamElem API also offers some iteration
functions similar to the ones offered by NextElem;
for example, here is listFeatures:
listFeatures :: DB -> Call (Protobuf RouteGuide "listFeatures") -> IO ()
listFeatures db call = do
r <- recvFinalInput call
StreamElem.forM_ (featuresIn db r) NoMetadata (sendOutput call)
The full server definition is available in tutorials/lowlevel/app/Server.hs.
On the client
The main function to make an RPC using the low-level API is
withRPC. For example, here is getFeature:
getFeature :: Connection -> IO ()
getFeature conn = do
let req = ..
withRPC conn def (Proxy @(Protobuf RouteGuide "getFeature")) $ \call -> do
sendFinalInput call req
resp <- recvFinalOutput call
print resp
The second argument to withRPC are the call
parameters, of which there are two important ones:
the timeout for this RPC, and the request
metadata. (When using the high-level API the
only way to set a timeout is to specify the default RPC
timeout for the connection.)
End of output, revisited
At the end of the basics tutorial, we emphasized the importance of indicating
end of output for streaming clients and server handlers. The
discussion there is relevant when using the low-level API as well, with some
additional caveats:
In the high-level API, the library can take care of marking the (only) value
for non-streaming output as final; in the low-level API, this is your own
responsibility, either through calling
sendFinalInput /
sendFinalOutput
or through calling
sendInput /
sendOutput
and constructing the StreamElem manually.
For streaming outputs, you can use
sendEndOfInput (clients) or
sendTrailers (servers) to indicate end of output
after the fact (like NoNextElem does), or use
sendFinalInput /
sendFinalOutput to mark the final element as
final when you send it. This should be preferred whenever possible.
As an example of using metadata, let’s construct a simple file server which
tells the client the size of the file to be downloaded as the initial response
metadata, then streams the contents of the file as a series of chunks, and
finally reports a SHA256 hash of the file contents in the trailing response
metadata. The client can use the initial file size metadata to show a progress
bar, and the hash in the trailing metadata to verify that everything went well.
You can find the full example in tutorials/metadata.
Service definition
The .proto file is straight-forward:
syntax = "proto3";
package fileserver;
service Fileserver {
rpc Download (File) returns (stream Partial) {}
}
message File {
string name = 1;
}
message Partial {
bytes chunk = 1;
}
As mentioned above, however, the .proto definition does not
tell us the type of the metadata. We need to do this in Haskell:
type instance RequestMetadata (Protobuf Fileserver "download") = NoMetadata
type instance ResponseInitialMetadata (Protobuf Fileserver "download") = DownloadStart
type instance ResponseTrailingMetadata (Protobuf Fileserver "download") = DownloadDone
data DownloadStart = DownloadStart {
downloadSize :: Integer
}
deriving stock (Show)
data DownloadDone = DownloadDone {
downloadHash :: ByteString
}
deriving stock (Show)
(In this example we make no use of request metadata; see
callRequestMetadata for the main entry
point for setting request metadata.)
Serialization
In order for the server to be able to send the metadata to the client, we need
to be able serialize it as one (or more, or zero) headers/trailers. This means
we must give an instance of BuildMetadata:
instance BuildMetadata DownloadStart where
buildMetadata DownloadStart{downloadSize} = [
CustomMetadata "download-size" $ C8.pack (show downloadSize)
]
instance BuildMetadata DownloadDone where
buildMetadata DownloadDone{downloadHash} = [
CustomMetadata "download-hash-bin" downloadHash
]
Note the use of the -bin suffix for the name of the download-hash-bin
trailer: this indicates that this is metadata containing binary data, and that
it must be Base64-encoded; grapesy will automatically take care of encoding
and decoding for binary metadata.
We need to take care of one more thing. The HTTP2 spec mandates that clients
must be informed up-front which trailing headers they can
expect. In grapesy this comes down to giving an instance
of StaticMetadata:
instance StaticMetadata DownloadDone where
metadataHeaderNames _ = ["download-hash-bin"]
This can be an over-approximation but not an under-approximation; if you return
a trailer in BuildMetadata that was not declared in StaticMetadata, then
grapesy will throw an exception.
Deserialization
For deserialization we must provide an instance of
ParseMetadata, which is given all metadata
headers to parse. In our example this is relatively simple because our metadata
uses only a single header:
instance ParseMetadata DownloadStart where
parseMetadata md =
case md of
[CustomMetadata "download-size" value]
| Just downloadSize <- readMaybe (C8.unpack value)
-> return $ DownloadStart{downloadSize}
_otherwise
-> throwM $ UnexpectedMetadata md
instance ParseMetadata DownloadDone where
parseMetadata md =
case md of
[CustomMetadata "download-hash-bin" downloadHash]
-> return $ DownloadDone{downloadHash}
_otherwise
-> throwM $ UnexpectedMetadata md
These particular instances will throw an error if additional metadata is
present. This is a choice, and instead we could simply ignore any additional
headers. There is no single right answer here: ignoring additional metadata runs
the risk of not realizing that the peer is trying to tell you something
important, but throwing an error runs the risk of unnecessarily aborting an RPC.
The metadata that is sent to the client with the response headers can be
overridden with
setResponseInitialMetadata. This
can be done at any point before initiating the request, either explicitly using
initiateResponse or implicitly by sending the
first output to the client using sendOutput and
related functions.
Most server handlers however don’t care about metadata, and prefer not to have
to call to setResponseInitialMetadata at all. For this reason
mkRpcHandler has type
mkRpcHandler :: Default (ResponseInitialMetadata rpc) => ..
This constraint is inherited by the high-level API, which doesn’t support
metadata at all:
Method :: (Default (ResponseInitialMetadata rpc), Default (ResponseTrailingMetadata rpc)) => ..
Crucially, there is a Default instance for
NoMetadata:
instance Default NoMetadata where
def = NoMetadata
In our case however we cannot provide a Default instance, because the initial
metadata depends on the file size. We therefore use mkRpcHandlerNoDefMetadata instead:
methods :: Methods IO (ProtobufMethodsOf Fileserver)
methods =
RawMethod (mkRpcHandlerNoDefMetadata download)
$ NoMoreMethods
This means we must call setResponseInitialMetadata in the handler; if we
don’t, an exception will be raised when the response is initiated.
Server handler
Since we are using the low-level API (we must, if we want to deal with metadata),
the server handler has this signature:
download :: Call (Protobuf Fileserver "download") -> IO ()
download call = do
We wait for the request from the client, get the file size, set the response
initial metadata, and initiate the response. Explicitly initiating the response
in this manner is not essential, but it means that the file size is sent to the
client (along with the rest of the response headers) before the first chunk is
sent; in some cases this may be important:
req :: Proto File <- recvFinalInput call
let fp :: FilePath
fp = Text.unpack (req ^. #name)
fileSize <- getFileSize fp
setResponseInitialMetadata call $ DownloadStart fileSize
initiateResponse call
We then open the file the client requested, and keep reading chunks until we
have reached end of file. Although it is probably not particularly critical in
this case, we follow the recommendations from End of output,
revisited and mark the final chunk as the final
output to the client, as opposed to telling the client that no more outputs are
available after the fact.
withFile fp ReadMode $ \h -> do
let loop :: SHA256.Ctx -> IO ()
loop ctx = do
chunk <- BS.hGet h defaultChunkSize
eof <- hIsEOF h
let resp :: Proto Partial
resp = defMessage & #chunk .~ chunk
ctx' :: SHA256.Ctx
ctx' = SHA256.update ctx chunk
if eof then
sendFinalOutput call (resp, DownloadDone $ SHA256.finalize ctx')
else do
sendNextOutput call resp
loop ctx'
loop SHA256.init
When we send the final output, we must also include the hash that we computed
as we were streaming the file to the client.
Client
Let’s first consider how to process the individual chunks that we get from the
server. We do this in an auxiliary function processPartial:
processPartial ::
Handle
-> Proto Partial
-> ProgressT (StateT SHA256.Ctx IO) ()
processPartial h partial = do
liftIO $ BS.hPut h chunk
modify $ flip SHA256.update chunk
updateProgressBar $ BS.length chunk
where
chunk :: ByteString
chunk = partial ^. #chunk
We do three things in this function: write the chunk to disk, update the hash,
and update the progress bar; this uses StateT to keep track of the partially
computed hash, and ProgressT for a simple progress bar (ProgressT is defined
in tutorials/metadata/app/ProgressT.hs; its details are not important here).
This in hand, we can now define the main client function. We are given some file
inp that we are interested in downloading, and a path out where we want to
store it locally. Like in the server, here too we must use the low-level API, so
the client starts like this:
download :: Connection -> String -> String -> IO ()
download conn inp out = do
withRPC conn def (Proxy @(Protobuf Fileserver "download")) $ \call -> do
sendFinalInput call $ defMessage & #name .~ Text.pack inp
We then wait for the initial response metadata, telling us how big the file is:
DownloadStart{downloadSize} <- recvResponseInitialMetadata call
We then use StreamElem.whileNext_ again
to process all the chunks using processPartial that we already discussed,
unwrap the monad stack, and finally do a hash comparison:
(DownloadDone{downloadHash = theirHash}, ourHash) <-
withFile out WriteMode $ \h ->
flip runStateT SHA256.init . runProgressT downloadSize $
StreamElem.whileNext_ (recvOutput call) (processPartial h)
putStrLn $ "Hash match: " ++ show (theirHash == SHA256.finalize ourHash)
Custom monad stack
In this section we will briefly discuss how to use custom monad stacks.
You can find the full tutorial in tutorials/monadstack; it is a variant on
Basics tutorial.
On the server
Most of the server handlers in for the RouteGuide service need to take the
DB as an argument:
getFeature :: DB -> Proto Point -> IO (Proto Feature)
listFeatures :: DB -> Proto Rectangle -> (NextElem (Proto Feature) -> IO ()) -> IO ()
recordRoute :: DB -> IO (NextElem (Proto Point)) -> IO (Proto RouteSummary)
It might be more convenient to define a custom Handler monad stack in which
we have access to the DB at all times:
newtype Handler a = WrapHandler {
unwrapHandler :: ReaderT DB IO a
}
deriving newtype (Functor, Applicative, Monad, MonadIO, MonadReader DB)
runHandler :: DB -> Handler a -> IO a
runHandler db = flip runReaderT db . unwrapHandler
The types of our handlers then becomes
getFeature :: Proto Point -> Handler (Proto Feature)
listFeatures :: Proto Rectangle -> (NextElem (Proto Feature) -> IO ()) -> Handler ()
recordRoute :: IO (NextElem (Proto Point)) -> Handler (Proto RouteSummary)
Note that the callbacks to send or receive values still live in IO. The DB
argument now disappears from methods also:
methods :: Methods Handler (ProtobufMethodsOf RouteGuide)
methods =
Method (mkNonStreaming getFeature )
$ Method (mkServerStreaming listFeatures)
$ Method (mkClientStreaming recordRoute )
$ Method (mkBiDiStreaming routeChat )
$ NoMoreMethods
The only requirement from grapesy is that at the top-level we can hoist this
monad stack into IO, using hoistMethods:
hoistMethods :: (forall a. m a -> n a) -> Methods m rpcs -> Methods n rpcs
Here’s how we can run the server:
runServerWithHandlers def config $ fromMethods $
hoistMethods (runHandler db) methods
On the client
For the high-level API there is support for custom monad stacks also. One reason
why you might want to do this is to avoid having to pass the Connection object
around all the time. In the Basics tutorial our client functions had these
signatures:
getFeature :: Connection -> IO ()
listFeatures :: Connection -> IO ()
recordRoute :: Connection -> IO ()
routeChat :: Connection -> IO ()
Like on the server, we can define a custom monad stack to reduce syntactic
overhead:
newtype Client a = WrapClient {
unwrapClient :: ReaderT ClientEnv IO a
}
deriving newtype (Functor, Applicative, Monad, MonadIO, MonadCatch, MonadThrow, MonadMask)
data ClientEnv = ClientEnv {
conn :: Connection
}
In order for such a monad stack to be useable, it needs to implement MonadIO
and MonadMask, as well as CanCallRPC; it’s this last class that tells
grapesy to get get access to the Connection object:
instance CanCallRPC Client where
getConnection = WrapClient $ conn <$> ask
With this defined, we can now avoid having to pass the connection around at all.
Instead of importing from
Network.GRPC.Client.StreamType.IO we
import from
Network.GRPC.Client.StreamType.CanCallRPC
instead, which gives us a different definition of nonStreaming and friends.
For example, here is getFeature:
getFeature :: Client ()
getFeature = do
let req = ..
resp <- nonStreaming (rpc @(Protobuf RouteGuide "getFeature")) req
liftIO $ print resp
As for the server handlers, the callbacks provided to send and receive messages
still live in IO; this means that we’ll need to liftIO them where
appropriate:
listFeatures :: Client ()
listFeatures = do
let req = ..
serverStreaming (rpc @(Protobuf RouteGuide "listFeatures")) req $ \recv -> liftIO $
NextElem.whileNext_ recv print
Using conduits
We discussed the simplest form of serverStreaming and co when we discussed the
implementation of the client in the Basics tutorial,
and we have also seen the generalized form to arbitrary monad
stacks. There is one more form, provided in
Network.GRPC.Client.StreamType.Conduit,
which provides an API using conduits. You can find this
example in tutorials/conduit; there is currently no conduit support on the
server side.
The main idea is that serverStreaming provides a source to stream from,
and clientStreaming_ provides a sink to stream to:
listFeatures :: Connection -> IO ()
listFeatures conn = do
let req = ..
let sink :: ConduitT (Proto Feature) Void IO ()
sink = ..
serverStreaming conn (rpc @(Protobuf RouteGuide "listFeatures")) req $ \source ->
runConduit $ source .| sink
recordRoute :: Connection -> IO ()
recordRoute conn = do
let source :: ConduitT () (Proto Point) IO ()
source = ..
resp <- clientStreaming_ conn (rpc @(Protobuf RouteGuide "recordRoute")) $ \sink ->
runConduit $ source .| sink
print resp
In bidirectional streaming finally we get two conduits, one in each direction
(that is, one source and one sink).
(Ab)using Trailers-Only
For this final section we need to consider some more low-level details about how
gRPC is sent over HTTP2. When we discussed final elements, we
mentioned that gRPC messages are sent using HTTP2 DATA frames, but we didn’t
talk about headers. In general, a gRPC request looks like this:
One or more HEADERS frames, containing the request
headers. One of the most important headers here is the :path
(pseudo) header, which indicates which RPC we
want to invoke; for example, this might be
/routeguide.RouteGuide/ListFeatures.
One or more DATA frames, the last of which is marked END_STREAM. We
discussed these before.
This is probably as expected, but the structure of the response may look
a bit more surprising:
Just like the request, we first get one or more HEADERS. An important
example here is the content-type response header, which indicates what kind
of message encoding is being used (for example, application/grpc+proto for
Protobuf).
One or more DATA frames, the last of which is marked END_STREAM.
Finally, another set of headers, also known as trailers. This set of
trailers provides some concluding information about how the RPC went;
for example, if the RPC failed, then the trailers will include a
grpc-status header with a non-zero value. Any application specific
response trailing metadata (such as the checksum we discussed in the
file server example) is included here as well.
There is however a special case, known as Trailers-Only: if there is no data
to be sent at all, it is possible to send only HEADERS frames, the last of
which is marked END_STREAM, and no DATA frames at all. Put another way, the
two sets of headers (headers and trailers) are combined, and the data frames are
omitted entirely.
The gRPC specification is very explicit about the use of
Trailers-Only, and states that it can be used only in RPCs that result in an
error:
Most responses are expected to have both headers and trailers but
Trailers-Only is permitted for calls that produce an immediate error.
In grapesy this will happen automatically: if a server handler raises an
error, and no outputs have as yet been sent to the client, then grapesy will
automatically take advantage of Trailers-Only and only send a single set of
headers.
However, some gRPC servers also make use of Trailers-Only in non-error
cases, when there is no output (e.g. for server-side streaming). Since this does
not conform to the gRPC specification, grapesy will not do this automatically,
but it is possible if really needed. In tutorials/trailers-only you can find
an example RouteGuide server which will take advantage of Trailers-Only in
the listFeatures method, when there are no features to return:
listFeatures :: DB -> Call (Protobuf RouteGuide "listFeatures") -> IO ()
listFeatures db call = do
r <- recvFinalInput call
case featuresIn db r of
[] -> sendTrailersOnly call NoMetadata
ps -> StreamElem.forM_ ps NoMetadata (sendOutput call)
The difference between this implementation and the previous one can only be
observed when we look at the raw network traffic; the difference is not visible
at the gRPC level. Since this violates the specification, however, it’s possible
(though perhaps unlikely) that some clients will be confused by this server.
Future work
The gRPC specification is only the core of the gRPC ecosystem.
There are additional features that are defined on top, some of which are
supported by grapesy (see list of features at the start of this post), but not
all; the features that are not yet supported are listed below. Note that these
are optional features, which have various degrees of support in the official
clients and servers. If you or your company needs any of these features, we’d be
happy to discuss options; please contact us at
info@well-typed.com.
Authentication. The gRPC Authentication
Guide mentions three ways to authenticate: SSL/TLS,
Application Layer Transport Security (ALTS) and token-based
authentication, possibly through OAuth2. Of these three only SSL/TLS is
currently supported by grapesy.
Interceptors are essentially a form of middleware
that are applied to every request, and can be used for things like metrics
(see below).
Custom Backend Metrics. There is support in
grapesy for parsing or including an Open Request Cost Aggregation (ORCA)
load report in the response trailers through the endpoint-load-metrics-bin
trailer, but there is otherwise no support for ORCA or custom backend metrics
in general.
Load balancing. There is very limited
support for load balancing in the
ReconnectPolicy, but we have no support for
load balancing as described in the Custom Load Balancing
Policies Guide.
Custom name resolution.
Automatic deadline propagation.
There is of course support for setting timeouts,
but there is no support for automatic propagation from one server to another,
adjusting for clock skew. See the section “Deadline Propagation” in
Deadlines Guide for server.
Introspection, services that allow to query the state of the server:
True binary metadata. There is support in
grapesy for sending binary metadata (in -bin headers/trailers), using
base64 encoding (as per the spec). True binary
metadata is about avoiding this encoding
overhead.
Sending keep-alive pings (this will require
adding this feature to the http2 library).
Retry policies. The gRPC documentation
currently identifies two such policies: request
hedging, which sends the same request to a number
of servers, waiting for the first response it receives; and automatic
retries of failed requests. There is support in grapesy
for the grpc-previous-rpc-attempts request header as well as the
grpc-retry-pushback-ms response trailer, necessary to support these
features.
can be used to engrave any of the
three syllables 송 승 숭 the operator has acutally used it
to engrave the first of these.
</noscript>
</noscript>
</noscript>
</noscript>
</noscript>
Submission Deadline: 9 February 2025
Never spoken before? No worries! We’re committed to supporting speakers from all backgrounds, especially those from underrepresented groups in tech.
