Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serializer system doesn't support Protobufs without unnecessary pointer dereference #104

Open
jsimnz opened this issue Jun 2, 2023 · 0 comments

Comments

@jsimnz
Copy link

jsimnz commented Jun 2, 2023

Theres a bug in how the serializer is used within the table that doubles the pointers for types, and prevents the usage of Protobufs as Table types without weird/unnecessary reflection and pointer dereferencing.

ie:

/* defined proto message:

message MyMessage {
  ...
}

*/

// Generated from protobuf
// with pointer recievers on all methds
type MyMessage struct { ... }

func main() {
  // explicit parameter type declaration for clarity
  opts := bond.DefaultOptions()
  opts.Serializer = protoSerializer{}
  bond = bond.Open(opts)
  
  myTable = bond.NewTable[*MyMessage](bond.TableOptions[*MyMessage]{
    ...
  })

  myTable.Exists(&MyMessage{...}) // this will error
}

With the following serializer implementation:

// Protobuf serializer
type protoSerializer struct{}

var (
	ErrNotProtobuf = fmt.Errorf("repo: type doesn't implement proto.Message")
)

func (ps protoSerializer) Serialize(i interface{}) ([]byte, error) {
	p, ok := i.(proto.Message)
	if !ok {
		fmt.Printf("%v (%T)\n", i, i)
		return nil, ErrNotProtobuf
	}

	return proto.Marshal(p)
}

func (ps protoSerializer) Deserialize(b []byte, i interface{}) error {
	p, ok := i.(proto.Message)
	if !ok {
		return ErrNotProtobuf
	}

	return proto.Unmarshal(b, p)
}

The issue stems from the fact that values passed to the Serializer within the Table code, have an additional pointer to them.
Like right here (there are a couple of spots, this is just one):

bond/table.go

Line 403 in 6192574

data, err := serialize(&tr)

This adds an additional pointer, but doesn't actually provide any additional value as far as I can tell. Testing locally on my fork, you can remove the additional & pointer reference, and everything still works (passes all tests).

I suspect it works because the builtin serializers (cbor, json, and msgpack) all do their own recursive pointer indirection, when they call <codec>.Marshal(i). So an additional pointer doesn't cause any issues.

However, in the protobuf serializer code above, it needs a proto.Message type, not an interface{} type, unlike the other codec marshallers. So when the p, ok := i.(proto.Message) runs, it won't pass the type assertion, since the underlying type of i isn't *MyMessage (which does implement proto.Message) but **MyMessage (double pointer, which doesn't implement proto.Message)

The protoSerializer code can be modified to work by checking the reflect.TypeOf(i), and if its a pointer, dereference, and recusively calling Serialize, but obviously I'd like to avoid that.

The other alternative is to not use bond.Table[*MyMessage] as the table parameter, but when using protobufs, all the generated code is with pointer recievers, so the unnecessary dereference would just happen elsewhere in my code. Ideally the extra & in the table code can be removed, and this kind of usecase can be cleanly implemented.

--
ps. Enjoying bond 👍 - thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant