Add Generated TMC TIM Topic

[mwodahl]

PR Details

Description

This PR adds a new Kafka topic that contains TIMs generated by the TMC. This includes TIMs deposited via the TimDepositController, offloaded files, and those deposited through the TimReceiver.

Motivation and Context

This update allows users to access a Kafka topic that only contains TMC-generated TIMs.

How Has This Been Tested?

This has been tested locally with docker-compose and Postman. TIMs were verified to be deposited to the new topic in the event they were generated by the TMC using kcat.

Types of changes

• [ X ] Defect fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that cause existing functionality to change)

Checklist:

• [ X ] I have added any new packages to the sonar-scanner.properties file
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• [ X ] I have read the CONTRIBUTING document.
  ODE Contributing Guide
• [ X ] I have added tests to cover my changes.
• [ X ] All new and existing tests passed.

[dmccoystephenson]

Looks good & the unit tests pass!

[Michael7371]

Looks good. This extra topic will be nice to have!

[payneBrandon]

Discussed offline, but we'll want to tweak this to push the same TIM data as the topic.OdeTimJson, but with the additional asn1 field filled out. Since this is the encoded path, getting the combo is more challenging. One option we talked about is adding a key to the metadata (which is maintained through encoding) and using KTable on top of the ode tim topic to find the corresponding json and combine.

[mwodahl]

Updated this to include an additional topic, KeyedOdeTimJson, that has a unique identifier attached to the TIM JSON that allows for later querying in the Asn1EncodedDataRouter. I added a new topic instead of reusing the OdeTimJson topic as it prevents unnecessary information from being included in the OdeTimJson topic and allows for pushing a null record with the generated TIM UUID to erase the TIM record from TimJson-Store after it's been retrieved. I've also updated the diagrams to reflect these changes.

[dmccoystephenson]

Looks good & the unit tests pass, I just had one question/suggestion

[payneBrandon]

@mwodahl looks like we have a merge conflict from Matt's properties changes

[Michael7371]

From my local testing, it looks like the topic.OdeTimJson still is missing the asn1 field. Is this the expected behavior?

[mwodahl]

From my local testing, it looks like the topic.OdeTimJson still is missing the asn1 field. Is this the expected behavior?

Yep, that's expected. The asn1 field will only be present in the topic.OdeTimJsonTMCFiltered topic.

[payneBrandon]

lgtm!

[dmccoystephenson]

Just the one comment

[Michael7371]

LGTM!