Getting Started

Installation

First, install the Cap'n Proto tools. Then, install the Go support by running the following:

# Be sure you have $GOPATH set.
$ go get -u zombiezen.com/go/capnproto2/...

What is Cap'n Proto?

Cap'n Proto is a language-agnostic serialization format. Given a simple schema like this:

using Go = import "zombiezen.com/go/capnproto2/go.capnp";
@0x85d3acc39d94e0f8;
$Go.package("books");
$Go.import("foo/books");

struct Book {
	title @0 :Text;
	# Title of the book.

	pageCount @1 :Int32;
	# Number of pages in the book.
}

You can use it to write data structures to byte streams, like so:

package main

import (
	"os"

	"foo/books"
	"zombiezen.com/go/capnproto2"
)

func main() {
	// Make a brand new empty message.  A Message allocates Cap'n Proto structs.
	msg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
	if err != nil {
		panic(err)
	}

	// Create a new Book struct.  Every message must have a root struct.
	book, err := books.NewRootBook(seg)
	if err != nil {
		panic(err)
	}
	book.SetTitle("War and Peace")
	book.SetPageCount(1440)

	// Write the message to stdout.
	err = capnp.NewEncoder(os.Stdout).Encode(msg)
	if err != nil {
		panic(err)
	}
}

And then read it back like so:

package main

import (
	"fmt"
	"os"

	"foo/books"
	"zombiezen.com/go/capnproto2"
)

func main() {
	// Read the message from stdin.
	msg, err := capnp.NewDecoder(os.Stdin).Decode()
	if err != nil {
		panic(err)
	}

	// Extract the root struct from the message.
	book, err := books.ReadRootBook(msg)
	if err != nil {
		panic(err)
	}

	// Access fields from the struct.
	title, err := book.Title()
	if err != nil {
		panic(err)
	}
	pageCount := book.PageCount()
	fmt.Printf("%q has %d pages\n", title, pageCount)
}

Cap'n Proto works by generating code: your schema can be converted to code for any language that Cap'n Proto supports. For Go, you would use:

$ capnp compile -ogo foo/books.capnp

This would generate a file foo/books.capnp.go.

Remote calls using interfaces

Serializing data structures is useful, but the real power of Cap'n Proto is the RPC Protocol. RPC (remote procedure call) is a form of network communication that allows you to make function calls between processes, often on different machines. Just by providing a byte stream -- like a pipe or a network connection -- you can make method calls on objects in other processes.

The first step is to define an interface inside your schema:

using Go = import "zombiezen.com/go/capnproto2/go.capnp";
@0xdb8274f9144abc7e;
$Go.package("hashes");
$Go.import("foo/hashes");

interface HashFactory {
	newSha1 @0 () -> (hash :Hash);
}

interface Hash {
	write @0 (data :Data) -> ();
	sum @1 () -> (hash :Data);
}

Implementing the interface is straightforward:

package main

import (
	"crypto/sha1"
	"fmt"
	"hash"
	"net"

	"foo/hashes"
	"golang.org/x/net/context"
	"zombiezen.com/go/capnproto2/rpc"
)

// hashFactory is a local implementation of HashFactory.
type hashFactory struct{}

func (hf hashFactory) NewSha1(call hashes.HashFactory_newSha1) error {
	// Create a new locally implemented Hash capability.
	hs := hashes.Hash_ServerToClient(hashServer{sha1.New()})
	// Notice that methods can return other interfaces.
	return call.Results.SetHash(hs)
}

// hashServer is a local implementation of Hash.
type hashServer struct {
	h hash.Hash
}

func (hs hashServer) Write(call hashes.Hash_write) error {
	data, err := call.Params.Data()
	if err != nil {
		return err
	}
	_, err = hs.h.Write(data)
	if err != nil {
		return err
	}
	return nil
}

func (hs hashServer) Sum(call hashes.Hash_sum) error {
	s := hs.h.Sum(nil)
	return call.Results.SetHash(s)
}

func server(c net.Conn) error {
	// Create a new locally implemented HashFactory.
	main := hashes.HashFactory_ServerToClient(hashFactory{})
	// Listen for calls, using the HashFactory as the bootstrap interface.
	conn := rpc.NewConn(rpc.StreamTransport(c), rpc.MainInterface(main.Client))
	// Wait for connection to abort.
	err := conn.Wait()
	return err
}

And then using it in a client:

func client(ctx context.Context, c net.Conn) error {
	// Create a connection that we can use to get the HashFactory.
	conn := rpc.NewConn(rpc.StreamTransport(c))
	defer conn.Close()
	// Get the "bootstrap" interface.  This is the capability set with
	// rpc.MainInterface on the remote side.
	hf := hashes.HashFactory{Client: conn.Bootstrap(ctx)}

	// Now we can call methods on hf, and they will be sent over c.
	s := hf.NewSha1(ctx, func(p hashes.HashFactory_newSha1_Params) error {
		return nil
	}).Hash()
	// s refers to a remote Hash.  Method calls are delivered in order.
	s.Write(ctx, func(p hashes.Hash_write_Params) error {
		err := p.SetData([]byte("Hello, "))
		return err
	})
	s.Write(ctx, func(p hashes.Hash_write_Params) error {
		err := p.SetData([]byte("World!"))
		return err
	})
	// Get the sum, waiting for the result.
	result, err := s.Sum(ctx, func(p hashes.Hash_sum_Params) error {
		return nil
	}).Struct()
	if err != nil {
		return err
	}

	// Display the result.
	sha1Val, err := result.Hash()
	if err != nil {
		return err
	}
	fmt.Printf("sha1: %x\n", sha1Val)
	return nil
}

func main() {
	c1, c2 := net.Pipe()
	go server(c1)
	client(context.Background(), c2)
}

For the sake of simplicity, this example uses an in-memory pipe, but you can use TCP connections, Unix pipes, or any other type that implements io.ReadWriteCloser.

The return type for a client call is a promise, not an immediate value. It isn't until the Struct() method is called on a method that the client function blocks on the remote side. This relies on a feature of the Cap'n Proto RPC protocol called promise pipelining. The upshot is that this function only requires one network round trip to receive its results, even though there were multiple chained calls. This is one of Cap'n Proto's key advantages.

Further Reading

For more details on writing schemas, see the Cap'n Proto language reference. The capnp package docs detail the encoding and client API, whereas the rpc package docs detail how to initiate an RPC connection.