bug: stream_maps "*" doesn't work. Appears to do nothing at all

[gruckion]

Singer SDK Version

How do I find this?

Is this a regression?

• Yes

Python Version

3.12

Bug scope

Mapping (stream maps, flattening, etc.)

Operating System

Mac OSX

Description

Expected: As per the documentation to be able to rename a field. Using "*" in the loaders stream_maps for meltanolabs-target-postgres.

https://sdk.meltano.com/en/latest/stream_maps.html#applying-a-mapping-across-two-or-more-streams

Code

I tried to apply a change based on the docs. But I do not see acc_num in the outputted model 🤷 


        stream_maps:
          "*":
            acc_num: account_number
            account_number: __NULL__


What I actually want is.

```yaml
stream_maps:
  "*":
    name: first_name
    first_name: __NULL__

  loaders:
    - name: target-postgres
      variant: meltanolabs
      pip_url: meltanolabs-target-postgres
      config:
        host: 127.0.0.1
        port: 5432
        user: postgres
        dbname: postgres
        database: postgres
        stream_maps:
          mydatabase-bank_accounts:
            created_at:
              record['created_at'] if record.get('created_at') and '0000'
              not in record['created_at'] and '-00' not in record['created_at'] else
              '1970-01-01 00:00:00'
            updated_at:
              record['updated_at'] if record.get('updated_at') and '0000'
              not in record['updated_at'] and '-00' not in record['updated_at'] else
              '1970-01-01 00:00:00'

But this doesn't work. I'm doing a tap mysql -> postgres. The source mysql db has a load of invalid postgres date / timestamps that I need to accomidate.

Disgustingly I have to put

        stream_maps:
          mydatabase-my_table_name:
            created_at:
              record['created_at'] if record.get('created_at') and '0000'
              not in record['created_at'] and '-00' not in record['created_at'] else
              '1970-01-01 00:00:00'
            updated_at:
              record['updated_at'] if record.get('updated_at') and '0000'
              not in record['updated_at'] and '-00' not in record['updated_at'] else
              '1970-01-01 00:00:00'

Manually for each and every table 😢 .

[edgarrmondragon]

@gruckion See my comment in #1605 (comment). I can confirm the glob syntax is working .

[gruckion]

@gruckion See my comment in #1605 (comment). I can confirm the glob syntax is working .

I see, your using plugin. That works, but the stream maps that's built in doesn't work.

[edgarrmondragon]

That works, but the stream maps that's built in doesn't work.

Hey @gruckion. I don't understand the distinction you're making. In the linked comment, I'm using stream_maps as part of the target-postgres config, just like in the meltano.yml example you shared.

Am I missing something? In which situations does the glob syntax not work? Can you share an example?