Adding multiple description sources

[VoLKyyyOG]

Hi all,

I'm currently working on a more efficient way to add several description sources via the TableMetadata class. Right now, a naive method to add multiple descriptions may look like this (example BasePostgresMetadataExtractor).

for key, group in groupby(self._get_raw_extract_iter(), self._get_table_key):
    columns = []
    for row in group:
        last_row = row
        columns.append(
            ColumnMetadata(
                row['col_name'],
                row['col_description'],
                row['col_type'],
                row['col_sort_order']
            )
        )

    yield TableMetadata(
        database=self._database,
        cluster=last_row['cluster'],
        schema=last_row['schema'],
        name=last_row['name'], # table name (object_name)
        columns=columns,
        tags=last_row['tags'], # Python native list OR comma separated string
        description_source="DESCRIPTION  SOURCE 1",
        description="DESCRIPTION 1"
    )
    yield TableMetadata(
        database=self._database,
        cluster=last_row['cluster'],
        schema=last_row['schema'],
        name=last_row['name'], # table name (object_name)
        columns=columns,
        tags=last_row['tags'], # Python native list OR comma separated string
        description_source="DESCRIPTION  SOURCE 2",
        description="DESCRIPTION 2"
    )

However, this would be quite inefficient from what I can see as its duplicating the other fields (notably columns). Is there a cleaner or more concise way of adding multiple description sources at once? Perhaps a dictionary or something can be passed through.

[VoLKyyyOG]

Playing around models/table_metadata to accept a dictionary of programmatic descriptions. Will update soon.

[VoLKyyyOG]

PR: #1903