Skip to content

Latest commit

 

History

History
708 lines (584 loc) · 26.9 KB

API.md

File metadata and controls

708 lines (584 loc) · 26.9 KB
Raw

API Overview

This document explains how you use the code generated by the SwiftProtobuf plugin.

The generated code relies very heavily on the associated SwiftProtobuf library. Apart from the actual properties themselves, most of the methods described below are not explicit in the generated code. Rather, most of them appear in common shared protocols in the runtime library. They're collected here to make it easier to understand.

Message API

Messages in the input proto file generate Swift structs in the result. These structs conform to SwiftProtobuf.Message and provide Swift properties for every field, basic information about the message, standard initializers, and serialization and deserialization methods.

Here is a simple proto3 input file to motivate the example below:

syntax = "proto3";
message Example {
   enum E {
      DEFAULT = 0;
   }
   int32 field1 = 1;
   repeated string field2 = 2;
}

Here is the API for the struct generated from the above. (As mentioned above, this is not what you'll see if you open up the generated Swift code in your editor. This includes a lot of methods from extensions located in the library, and omits many details of the generated code that are intended purely for internal use by the library.)

public struct Example: SwiftProtobuf.Message {
  // The generated struct carries constant properties reflecting
  // basic information about the message:
  public var protoMessageName: String {return "Example"}

  // Nested enum and message types are nested in the generated Swift
  public enum E: SwiftProtobuf.Enum { ... }

  // A public property is created for each field in the proto.
  public var field1: Int32 { get set }
  public var field2: [String] { get set }

  // Default Initializer
  public init()

  // A convenience factory method for constructing
  // immutable objects. You can use it like this:
  //
  // let e = Example.with {
  //    $0.field1 = 7
  //    $0.field2 = ["foo", "bar"]
  // }
  public static with(_ configurator: (inout Example) -> ());

  // Messages can be serialized or deserialized to Data objects
  // using protobuf binary format.
  // Setting `partial` to `true` will suppress checks for required fields.
  // An extension map may be needed when decoding nested
  // proto2-format messages that utilize extensions.
  // See below for more details.
  func serializedData() throws -> Data
  init<Bytes: SwiftProtobufContiguousBytes>(serializedBytes: Bytes) throws {
  init<Bytes: SwiftProtobufContiguousBytes>(serializedBytes: Bytes, extensions: ExtensionMap? = nil, partial: Bool = false) throws

  // Messages can be serialized or deserialized to JSON format
  // as either UTF8-encoded ``SwiftProtobufContiguousBytes``-conforming objects or as Strings.
  func jsonUTF8Data() throws -> Data
  init<Bytes: SwiftProtobufContiguousBytes>(jsonUTF8Bytes: Bytes) throws
  func jsonString() throws -> String
  init(jsonString: String) throws

  // Messages can be serialized or deserialized to Protobuf TextFormat:
  func textFormatString() -> String
  init(textFormatString: String) throws

  // These are the generated methods used internally by the
  // serialization and deserialization mechanisms.
  // You should generally not call them directly.
  public func decodeMessage<D: Decoder>(decoder: inout D) throws
  public func traverse<V: Visitor>(visitor: inout V) throws
}

func ==(lhs: Example, rhs: Example) -> Bool

Generated struct name

The name of generated struct is based on the name of the message in the proto file.

For top-level messages, the name is prefixed with the proto package name as specified in any package statements. The name is converted to camel case with underscore separators to preserve the structure.

For example,

   syntax = "proto3";
   package my_company.cool_project;
   message FooBar {
      ...
      message Baz {
         ...
      }
   }

will by default generate a struct named MyCompany_CoolProject_FooBar with another Baz struct nested inside it. Note that Baz is not prefixed because it will be scoped to the parent type.

You can change the prefix with the option swift_prefix statement in your proto file:

   syntax = "proto3";
   package my_company.cool_project;
   option swift_prefix="My";
   message FooBar {
      ...
   }

will generate a struct named MyFooBar. (Note: swift_prefix is only supported by protoc 3.2 or later.)

⚠️ The swift_prefix option has proven problematic in practice. Because it ignores the package directive, it can easily lead to name conflicts and other confusion as your shared proto definitions evolve over time. For example, say you have a file that defines "User" and/or "Settings", that will work great without the package prefix until you use a second proto file that defined a different "User" and/or "Settings". Protocol buffers solved this by having the package in the first place, so by overriding that with a custom Swift prefix makes you that much more likely to have collisions in the future. If you are considering a prefix just to make the type names shorter/nicer, then instead consider using a Swift typealias within your source to remap the names locally where they are used, but keeping the richer name for the full build to thus avoid the conflicts.

If the resulting name would collide with a Swift reserved word or would otherwise cause problems in the generated code, then the word Message is appended to the name. For example, a message Int in the proto file will cause the generator to emit a struct IntMessage to the generated Swift file.

Enum API

Proto enums are translated to Swift enums in a fairly straightforward manner. The resulting Swift enums conform to the SwiftProtobuf.Enum protocol which extends RawRepresentable with a RawValue of Int. The generated Swift enum will have a case for each enum value in the proto file.

Proto3 enums have an additional UNRECOGNIZED(Int) case that is used whenever an unrecognized value is parsed from protobuf serialization or from other serializations that store integer enum values. Proto2 enums lack this extra case.

If deserialization encounters an unknown value:

  • For JSON, if the value was in string form, it causes a parsing error as it can't be mapped to a value. If the value was an integer value, then a proto3 syntax enum can still capture it via the UNRECOGNIZED(Int) case.
  • For protobuf binary, the value is handled as an unknown field.
public enum MyEnum: SwiftProtobuf.Enum {
    public typealias RawValue = Int

    // Case for each value
    // Names are translated to a lowerCamelCase convention from
    // the UPPER_CASE convention in the proto file:
    case default
    case other
    case andMore
    case UNRECOGNIZED(Int) // Only in proto3 enums

    // Initializer selects the default value (see proto2 and proto3
    // language guides for details).
    public init()

    public init?(rawValue: Int)
    public var rawValue: Int
    public var hashValue: Int
    public var debugDescription: String
}

Enum and enum case naming

The name of the Swift enum is copied directly from the name in the proto file, prefixed with the package name or the name from option swift_prefix as documented above for messages. If that name would conflict with a Swift reserved word or otherwise cause problems for the generated code, the word Enum will be appended to the name.

Enum case names are converted from UPPER_SNAKE_CASE conventions in the proto file to lowerCamelCase in the Swift code.

If the enum case name includes the enum name as a prefix (ignoring case and underscore characters), that prefix is stripped. If the stripped name would conflict with another entry in the same enum, the conflicting cases will have their respective numeric values appended to ensure the results are unique. For example:

syntax = "proto3";
enum TestEnum {
    TEST_ENUM_FOO = 0;
    TESTENUM_BAR = 1;
    BAZ = 2;
    BAR = -3;
}

becomes

enum TestEnum {
  case foo = 0
  case bar_1 = 1
  case baz = 2
  case bar_n3 = -3 // 'n' for "negative"
}

Note #1: Enum aliases can potentially result in conflicting names even after appending the case numeric value. Since aliases are only supported to provide alternate names for the same underlying numeric value, SwiftProtobuf simply drops the alias in such cases. See the protobuf documentation for allow_alias for more information about enum case aliases.

Note #2: In most cases where an enum case name might conflict with a Swift reserved word, or otherwise cause problems, the code generator will protect the enum case name by surrounding it with backticks. In the few cases where this is insufficient, the code generator will append an additional underscore _ to the converted name.

Message Fields

Each message field in the proto file is compiled into a corresponding property on the generated struct. Field names are converted from snake_case conventions in the proto file to lowerCamelCase property names in the Swift file.

Note: In many cases where the resulting name would cause a problem in the generated Swift, the code generator will protect the field name by surrounding it with backticks. Sometimes, this is insufficient and the code generator will append a _p to the converted name.

Types in the proto file are mapped to Swift types as follows:

Basic types

Proto type Swift Type
int32 Int32
sint32 Int32
sfixed32 Int32
uint32 UInt32
fixed32 UInt32
int64 Int64
sint64 Int64
sfixed64 Int64
uint64 UInt64
fixed64 UInt64
bool Bool
float Float
double Double
string String
bytes Data

Generated Types

Enums in the proto file generate Int-valued enums in the Swift code.

Groups in the proto file generate Swift structs that conform to SwiftProtobuf.Message.

Messages in the proto file generate Swift structs that conform to SwiftProtobuf.Message.

Note: There is also a SwiftProtobuf._MessageImplementationBase protocol. You should not refer to that directly; use SwiftProtobuf.Message when you need to work with arbitrary groups or messages.

Type modifiers

Proto3 singular fields generate properties of the corresponding type above. These properties are initialized to the appropriate default value as specified in the proto3 specification:

  • Numeric fields are initialized to zero.
  • Boolean fields are initialize to false.
  • String fields are initialized to the empty string.
  • Bytes fields are initialized to an empty Data() object.
  • Enum fields are initialized to the default value (the value corresponding to zero, which must be the first item in the enum).
  • Message fields are initialized to an empty message of the appropriate type.

Notes: For performance, the field may be initialized lazily, but this is invisible to the user. The property will be serialized if it has a non-default value.

Proto2 optional fields generate properties of the corresponding type above. It also generates has and clear methods that can be used to test whether the field has a value or to reset it to it's default. If a default value was specified in the proto file, the field will be initialized to that value, and will be reset to that value when you invoke the clear method. If no default value was specified, the default value is the same as for proto3 singular fields above.

Proto2 required fields Required fields behave the same as optional fields, except that serialization or deserialization may fail if the field is not provided.

To illustrate the handling of proto2 fields, consider the following short example:

syntax = "proto2";
message ExampleProto2 {
   optional int32 item_count = 1 [default = 12];
   optional string item_label = 2;
}

This will generate the following field structure in the Swift code:

public struct ExampleProto2 {
    public var itemCount: Int32 = 12
    public var hasItemCount: Bool
    public mutating func clearItemCount()

    public var itemLabel: String = "";
    public var hasItemLabel: Bool
    public mutating func clearItemLabel()
}

Compound types

Singular message fields generate simple properties of the corresponding Swift struct type. The fields are initialized with default instances of the struct. (This initialization is usually done lazily the first time you read such a field.) Message fields generate has and clear methods as above for both proto2 and proto3.

Proto2 groups act exactly like messages in all respects, except that they are serialized differently when they appear as a field value.

Proto repeated fields generate simple properties of type Array<T> where T is the base type from above. Repeated fields are always initialized to an empty array.

Proto map fields generate simple properties of type Dictionary<T,U> where T and U are the respective key and value types from above. Map fields are always initialized to an empty map.

Oneof fields

Oneof fields generate an enum with a case for each associated field. These enums conform to ProtobufOneofEnum. Every case has an associated value corresponding to the declared field.

The message will have a read/write property named after the enum which contains the enum value; this property has an optional type and will be nil if no oneof field is set.

It also will contain a separate read/write computed property for each member field of the enum.

Here is a simple example of a message with a oneof structure:

syntax = "proto3";
message ExampleOneOf {
   int32 field1 = 1;
   oneof alternatives {
       int64 id = 2;
       string name = 3;
   }
}

And here is the corresponding generated Swift code. Note that the two fields id and name above share storage, thanks to the generated OneOf_Alternatives enum type. Also note that you can access the alternatives property here directly if you want to use a switch construct to analyze the fields contained in the oneof:

public struct ExampleOneOf: SwiftProtobuf.Message {
   enum OneOf_Alternatives {
   case id(Int32)
   case name(String)
   }

   var field1: Int32 = 0
   var alternatives: OneOf_Alternatives?

   var id: Int32 {
      get {
         if case .id(let v)? = alternatives {return v}
         else {return 0}
      }
      set {
         alternatives = .id(newValue)
      }
   }
   var name: String {
      get {
         if case .name(let v)? = alternatives {return v}
         else {return ""}
      }
      set {
         alternatives = .name(newValue)
      }
   }
}

Well-Known Types

For most of the proto3 well-known types, the Swift API is exactly what you would expect from the corresponding proto definitions. (In fact, the runtime library version for most of these is simply generated.) For convenience, most of these also have hand-written extensions that expand the functionality with various convenience methods. The variations from the default generated behavior are described below.

Proto Type Swift Type
google.protobuf.Any Google_Protobuf_Any
google.protobuf.Api Google_Protobuf_Api
google.protobuf.BoolValue Google_Protobuf_BoolValue
google.protobuf.BytesValue Google_Protobuf_BytesValue
google.protobuf.DoubleValue Google_Protobuf_DoubleValue
google.protobuf.Duration Google_Protobuf_Duration
google.protobuf.Empty Google_Protobuf_Empty
google.protobuf.FieldMask Google_Protobuf_FieldMask
google.protobuf.FloatValue Google_Protobuf_FloatValue
google.protobuf.Int64Value Google_Protobuf_Int64Value
google.protobuf.ListValue Google_Protobuf_ListValue
google.protobuf.StringValue Google_Protobuf_StringValue
google.protobuf.Struct Google_Protobuf_Struct
google.protobuf.Timestamp Google_Protobuf_Timestamp
google.protobuf.Type Google_Protobuf_Type
google.protobuf.UInt32Value Google_Protobuf_UInt32Value
google.protobuf.UInt64Value Google_Protobuf_UInt64Value
google.protobuf.Value Google_Protobuf_Value

For most of these types, you should refer to Google's documentation. Details are provided here to explain details of how these are implemented by SwiftProtobuf.

Google_Protobuf_Value, Google_Protobuf_Struct, Google_Protobuf_ArrayValue

These types can be used for ad hoc encoding and decoding of arbitrary JSON structures.

They are particularly useful when dealing with legacy JSON formats where the bulk of the structure is well-defined and maps cleanly into protobuf JSON conventions but there are occasional fields that may contain arbitrary data.

syntax = "proto3";
message ExampleAdHocJSON {
   int32 id = 1;
   string name = 2;
   google.protobuf.Struct jsonObject = 3;
}

Google_Protobuf_NullValue is a simple single-value enum that corresponds to null in JSON syntax. In particular, NullValue and Value are the only ways to determine if a null appeared in JSON. (In all other circumstances, protobuf JSON decoders treat JSON null as either illegal or as a default value for the field.)

Google_Protobuf_Struct contains a single fields dictionary mapping strings to Google_Protobuf_Value objects. It also conforms to ExpressibleByDictionaryLiteral and provides a subscript for directly accessing the values by name.

Google_Protobuf_ArrayValue is similar, it conforms to ExpressibleByArrayLiteral and provides an integer-keyed subscript for accessing values by index.

The Google_Protobuf_Value type can support any JSON type and provides a oneof view of the contents.

Google_Protobuf_Any

The google.protobuf.Any proto type is provided as Google_Protobuf_Any. This type serves as a general container that can store any protobuf message type.

For example, suppose you have a message that contains such a field, as defined in the following proto file:

syntax = "proto3";
import "google/protobuf/any.proto";
message ExampleAny {
   string message = 1;
   google.protobuf.Any detail = 2;
}

If you have some other (separately-defined) message type Foo, you can store one of those objects in the ExampleAny struct by wrapping it in a Google_Protobuf_Any as follows:

    let foo = Foo()
    var exampleAny = ExampleAny()
    exampleAny.detail = Google_Protobuf_Any(message: foo)

You can then encode or decode the exampleAny as usual, even on systems that do not have the definition for Foo.

Of course, after decoding an ExampleAny, you need to inspect the detail field and then extract the inner message yourself:

    let anyObject = decodedExampleAny.detail
    if anyObject.isA(Foo.self) {
        let foo = try Foo(unpackingAny: anyObject)
    }

Caveat: The inner object is not actually decoded until you call the unpackingAny initializer. In particular, it is possible for the outer object to decode successfully even when the inner object is malformed.

You can also, of course, have repeated Any fields or use them in other more complex structures.

When coded to JSON format, the Any field will be written in a verbose form that expands the JSON encoding of the contained object. This makes the result easier to read and easier to interoperate with non-protobuf JSON implementations, but means that you cannot translate between binary and JSON encodings without having the type information available. If you need to translate objects between binary and JSON encodings, you should carefully read the documentation comments for Google_Protobuf_Any.register() which explains how to make your custom types available to the decode/encode machinery for this purpose.

Note: Google's C++ implementation will not decode JSON unless it understands the types of all inner objects. SwiftProtobuf can decode JSON in this case and can re-encode back to JSON. It only needs the types when translating between dissimilar encodings.

Caveat: SwiftProtobuf's Text format decoding will currently ignore Any fields if the types are not registered.

Google_Protobuf_Duration, Google_Protobuf_Timestamp

The Google_Protobuf_Duration and Google_Protobuf_Timestamp structs provide standard ways to exchange durations and timestamps between systems.

Following Google's specification, serializing one of these objects to JSON will throw an error if the duration is greater than 315576000000 seconds or if the timestamp is before 0001-01-01T00:00:00Z or after 9999-12-31T23:59:59.999999999Z in the Gregorian proleptic calendar.

The Google_Protobuf_Duration type conforms to ExpressibleByFloatLiteral; it can be initialized with a double representing the number of seconds.

A Google_Protobuf_Duration can be converted to and from a Foundation TimeInterval:

extension Google_Protobuf_Duration {
   public init(timeInterval: TimeInterval)
   public var timeInterval: TimeInterval {get}
}

A Google_Protobuf_Timestamp can be converted to and from common Foundation timestamp representations:

extension Google_Protobuf_Timestamp {
   /// To/From a Foundation `Date` object
   public init(date: Date)
   public var date: Date
   /// Relative to POSIX epoch of 00:00:00UTC 1 Jan 1970
   public init(timeIntervalSince1970: TimeInterval)
   public var timeIntervalSince1970: TimeInterval {get}
   /// Relative to Foundation's "reference date" of 00:00:00UTC 1 Jan 2001
   public init(timeIntervalSinceReferenceDate: TimeInterval)
   public var timeIntervalSinceReferenceDate: TimeInterval {get}
}

There are also overrides for simple arithmetic with durations and timestamps:

func -(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Timestamp) -> Google_Protobuf_Duration
func -(lhs: Google_Protobuf_Duration, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Duration
public func +(lhs: Google_Protobuf_Duration, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Duration
public func -(operand: Google_Protobuf_Duration) -> Google_Protobuf_Duration
public func -(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Timestamp
public func +(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Timestamp

Extensions

Extensions are used to add additional properties to messages defined elsewhere. They are fully supported in proto2 files.

They are supported in proto3 only when extending the standard Descriptor type.

Extensions are ignored when serializing or deserializing to JSON.

They are defined in proto2 files as follows:

/// File sample.proto
syntax="proto2";
message CanBeExtended {
   extensions 100 to 200;
}

extend CanBeExtended {
   optional int32 extensionField = 100;
}

There are several pieces to the extension support:

  • Extensible Messages (such as CanBeExtended above) conform to ExtensibleMessage and define some additional methods needed by the other components. You should not need to use these methods directly.

  • Extension objects are opaque objects that define the extension itself, including storage and serialization details. Because proto allows extension names to be reused in different scopes, these objects appear in the scope corresponding to the context where the proto extension was defined (file level or within the message that wrapped the extend directive); generally it does not correspond to that of the message being extended. In the above example, the extension object would be Extensions_extensionField at the file scope. Most common Swift code accessing Extensions won't have to access these directly.

  • Extension properties use Swift's extension capability to add properties to the message that is being extended. In most cases, you can simply use the extension properties without understanding any of the other extension machinery. The above example creates a Swift extension of CanBeExtended that defines a new property extensionField of type Int32.

  • Extension maps are collections of extension objects indexed by the target message and field number. An extension map is generated for every file that defined proto extensions and included as a static global variable. It is named based on the proto package, filename, and then ends in _Extensions, so the above file would be Sample_Extensions. These maps are then used by the Message apis for parsing/merging extension fields in the binary data; if a mapping isn't found, the extension field ends up in the unknownFields on the message.

    If you need to handle extensions defined in multiple files, you can build up your own ExtensionMap will all the data by using SimpleExtensionMap. The easiest way is to create a new SimpleExtensionMap passing in a list of the generated *_Extensions ExtensionMaps that were generated for you in each file (i.e. - let myMap = SimpleExtensionMap(Sample_Extensions, …)).

Descriptors

Some other languages expose Descriptor objects for messages, enums, fields, and oneof, but not all languages. The .proto language also allows developers to add options to messages, fields, etc. that can be looked up at runtime in those descriptors.

Support for descriptors ends up requiring some amount of code, but more importantly it requires capturing a large binary blob of data for every message, enum, oneof, etc. That data has two potenial issues, it bloats the binaries, and it is something that can be extracted from the binary to help reverse engineer details about the binary.

For these reasons, SwiftProtobuf does not current support anything like the Descriptor objects. It is something that could get revisited in the future, but will need careful consideration; the bloat/size issues is of the most concern because of Swift's common use for mobile applications.

Aside: proto2 vs. proto3

The terms proto2 and proto3 refer to two different dialects of the proto language. The older proto2 language dates back to 2008, the proto3 language was introduced in 2015. These should not be confused with versions of the protobuf project or the protoc program. In particular, the protoc 3.0 program has solid support for both proto2 and proto3 language dialects. Many people continue to use the proto2 language with protoc 3.0 because they have existing systems that depend on particular features of the proto2 language that were changed in proto3.