Add Scan and Value to pgtype.FlatArray, pgtype.Array, pgtype.Range, and pgtype.Multirange

[jackc]

This allows using the types directly without *Map.SQLScanner.

These methods were previously not implemented because they all require a *Map to encode or decode their underlying elements. The new implementations use an internal default *Map. This means that they may not be able to directly support custom types, but normal types like pgtype.FlatArray[string] will work as expected.

[stephenafamo]

Amazing work @jackc

Aside from normal types like pgtype.FlatArray[string], will this also work with types that implement database/sql.Scanner and driver.Valuer themselves?

[jackc]

@stephenafamo Normal types that implement Scanner and Valuer were already supported. The issue this works around is the difficulty of implementing them on generic container types like arrays and ranges.

[stephenafamo]

To be clear, what I meant is an array (or other container) of types that implement scanner/valuer

[gspeicher]

@stephenafamo I have not tested this branch directly, but the code examples in #1779, upon which this changeset appears to be based, does indeed work for FlatArray[T], Multirange[T], and Range[T] where T is a type supported by pgx such as pgtype.Date.

[muhlemmer]

Looks good to me 👍

[jackc]

To be clear, what I meant is an array (or other container) of types that implement scanner/valuer

@stephenafamo I'd assumed it would work. But I added a test:

func TestPGTypeFlatArrayOfScanner(t *testing.T) {
	testWithAllQueryExecModes(t, func(t *testing.T, db *sql.DB) {
		var names pgtype.FlatArray[sql.NullString]

		err := db.QueryRow("select array['John', 'Jane']::text[]").Scan(&names)
		require.NoError(t, err)
		require.Equal(t, pgtype.FlatArray[sql.NullString]{{String: "John", Valid: true}, {String: "Jane", Valid: true}}, names)

		var n int
		err = db.QueryRow("select cardinality($1::text[])", names).Scan(&n)
		require.NoError(t, err)
		require.EqualValues(t, 2, n)

		err = db.QueryRow("select null::text[]").Scan(&names)
		require.NoError(t, err)
		require.Nil(t, names)
	})
}

And it fails pretty horribly. Stack overflow / infinite recursion. All types must be registered on the connection for it to work. It just so happens all the tests I originally wrote were of types that are registered by default. 🤦

In the PostgreSQL protocol, the OID / data type of each result column is always available. The entire pgx scanning system is built around having the data type of what is being decoded. But the database/sql.Scanner interface does not provide underlying type information. In our case all we get is a string. While a concrete type can parse the string because it knows its own type, a generic type cannot.

I thought I had worked around that issue, but apparently I have not. I'm beginning to think it might not be possible, at least without an excessive amount of complexity or special cases.

[jackc]

I'm tired of fighting with database/sql's scanning system. Instead, I've made a proposal to allow drivers to override Scan entirely. If this is accepted there would be complete parity of type support between pgx and database/sql. It also could improve database/sql performance.

The proposal is here: golang/go#67546. I think it would be simple to implement in Go and simple for drivers to support.

[stephenafamo]

Thanks for all the work @jackc

I hope the proposal moves forward quickly, perhaps it can be in go1.24

[hiendaovinh]

The proposal is accepted. Looking for it to be shipped soon. I'm an user of pgx and bob and I would love to see proper pgx array support with bob.