Feature: ToMap() utility

[alarbada]

Feature description

In all the connectors I make I always have to either hardcode the map[string]string configuration, which is very susceptible of subtle connector misconfiguration bugs, or manually implement a toMap() method that returns the configuration as a map like this:

type SourceConfig struct {
	// StreamARN is the Kinesis stream's Amazon Resource Name
	StreamARN string `json:"streamARN" validate:"required"`
	// StreamName is the name of the Kinesis Data Stream
	StreamName string `json:"streamName"`
}

func (s SourceConfig) toMap() map[string]string {
	return map[string]string{
		"streamARN":  s.StreamARN,
		"streamName": s.StreamName,
	}
}

This works, but is also error prone, as I still need to manually check that the config parameter keys correspond to the ones on the source config. I'd rather do this:

func TestSomething(t *testing.T) {
	sourceCfg := SourceConfig{
		StreamARN:  "...",
		StreamName: "...",
	}.toMap()

	/// ...
}

And have the toMap() automatically generated by paramgen, instead of me implementing it. This way I've got a bit more "type safety", I make it impossible to misspell a property key and value.

[lovromazgon]

I briefly played around with this, it's not trivial. Paramgen currently works in two steps. Step 1 is to extract the parameters map, step 2 is to generate code based on the map using a Go template. I thought at first it would be as easy as updating the Go template in step 2, but that's not the case, because the extracted parameters don't have the information about the original field names. The whole thing gets much more complicated if you think about nested fields and wildcards (maps).

I also tried playing around with an approach where the config would get marshalled into JSON, unmarshalled into a map and then flattened, in case it contains nested fields. This approach also breaks down when fields are not tagged using a json tag, because paramgen is using a different way to determine the default field name than json does, and the json lib doesn't allow you to configure it other than with field tags.

So even though this seems easy, and for the basic case it probably is, it's not trivial to implement in the general case and support for nested structs, wildcards and fields without an explicit json tag.

---

However, what's easy to do is to generate constants for fields: #60

Example:

conduit-commons/paramgen/internal/testdata/basic/want.go

Lines 10 to 35 in 58f3e14

	const (
	SourceConfigFoo = "foo"
	SourceConfigMyBool = "myBool"
	SourceConfigMyByte = "myByte"
	SourceConfigMyDurSlice = "myDurSlice"
	SourceConfigMyDuration = "myDuration"
	SourceConfigMyFloat32 = "myFloat32"
	SourceConfigMyFloat64 = "myFloat64"
	SourceConfigMyFloatSlice = "myFloatSlice"
	SourceConfigMyInt = "myInt"
	SourceConfigMyInt16 = "myInt16"
	SourceConfigMyInt32 = "myInt32"
	SourceConfigMyInt64 = "myInt64"
	SourceConfigMyInt8 = "myInt8"
	SourceConfigMyIntSlice = "myIntSlice"
	SourceConfigMyRune = "myRune"
	SourceConfigMyString = "myString"
	SourceConfigMyStringMap = "myStringMap.*"
	SourceConfigMyStructMapMyInt = "myStructMap.*.myInt"
	SourceConfigMyStructMapMyString = "myStructMap.*.myString"
	SourceConfigMyUint = "myUint"
	SourceConfigMyUint16 = "myUint16"
	SourceConfigMyUint32 = "myUint32"
	SourceConfigMyUint64 = "myUint64"
	SourceConfigMyUint8 = "myUint8"
	)

Do you think this would be helpful @alarbada?

[alarbada]

@lovromazgon it is much better indeed, it statically prevents mistakes when updating / deleting properties. Still, when adding a new one you've got to be a bit careful. But for my tests #60 is more than enough. Thanks a lot!

[lovromazgon]

Ok merged #60 and regarding this as done for now.