Added conversion between JSON and ASN.1 C-struct for SDSM with Kafka updates #562

willjohnsonk · 2023-09-07T22:14:36Z

PR Details

Description

Added functionality to convert SDSMs from inbound json strings to outbound ASN.1 styled C-structs to facilitate communication over the CARMA Streets plugin in both directions.

Related Issue

CDAR-308
#589

Motivation and Context

How Has This Been Tested?

Tested locally in VSCode with new unit tests for the conversion between jsons and the ASN.1 C-struct messages.

Types of changes

Defect fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that cause existing functionality to change)

Checklist:

I have added any new packages to the sonar-scanner.properties file
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
V2XHUB Contributing Guide
I have added tests to cover my changes.
All new and existing tests passed.

…o previous converter

willjohnsonk · 2023-09-08T13:49:22Z

Needs fixes to sourceID and lights fields - in progress

May consider checking required fields with "isMember" before setting data

Some of the json access statements are very long, I find this more readable from a hierarchy standpoint but may be obtrusive. Can use variables to simply access if needed

paulbourelly999 · 2023-09-08T20:40:11Z

src/v2i-hub/CARMAStreetsPlugin/src/CARMAStreetsPlugin.cpp

@@ -49,6 +49,8 @@ void CARMAStreetsPlugin::UpdateConfigSettings() {
 	GetConfigValue<string>("MapTopic", _transmitMAPTopic);
 	GetConfigValue<string>("SRMTopic", _transmitSRMTopic); 
 	GetConfigValue<string>("SimSensorDetectedObjTopic", _transmitSimSensorDetectedObjTopic); 
+	GetConfigValue<string>("SdsmTopic", _subscribeToSdsmTopic);


I think you will need to add this as configuration parameter to the manifest.json for this to work.

paulbourelly999 · 2023-09-08T20:40:20Z

src/v2i-hub/CARMAStreetsPlugin/src/CARMAStreetsPlugin.cpp

@@ -49,6 +49,8 @@ void CARMAStreetsPlugin::UpdateConfigSettings() {
 	GetConfigValue<string>("MapTopic", _transmitMAPTopic);
 	GetConfigValue<string>("SRMTopic", _transmitSRMTopic); 
 	GetConfigValue<string>("SimSensorDetectedObjTopic", _transmitSimSensorDetectedObjTopic); 
+	GetConfigValue<string>("SdsmTopic", _subscribeToSdsmTopic);
+	GetConfigValue<string>("SdsmTopic", _transmitSDSMTopic);


Yep figured I missed something minor. I meant to ask- can/should these use the same name? The other configs seem to use [x]Topic interchangeably for subscribe/transmit but this one does both

paulbourelly999 · 2023-09-08T20:44:45Z

src/v2i-hub/CARMAStreetsPlugin/src/CARMAStreetsPlugin.h

@@ -67,6 +69,12 @@ class CARMAStreetsPlugin: public PluginClientClockAware {
 	 * @param routeableMsg 
 	 */
 	void HandleMapMessage(MapDataMessage &msg, routeable_message &routeableMsg);
+	/**
+	 * @brief Subscribe to SDSM broadcast.


Lets have a better description here. This method handles incoming SDSMs received by RSU and forwards them to CARMA Streets as JSON string messages.

paulbourelly999 · 2023-09-11T15:55:08Z

src/v2i-hub/CARMAStreetsPlugin/src/CARMAStreetsPlugin.cpp

+					SetStatus<uint>(Key_SDSMMessageSkipped, ++_sdsmMessageSkipped);
+					continue;
+				}
+				//Convert the SSM JSON string into J3224 SDSM message and encode it.


This is still referencing SSM. Minor fix comment.

paulbourelly999 · 2023-09-11T15:55:25Z

src/v2i-hub/CARMAStreetsPlugin/src/CARMAStreetsPlugin.cpp

+				ASN_STRUCT_FREE_CONTENTS_ONLY(asn_DEF_SensorDataSharingMessage, sdsm_ptr.get()); // same as above
+				PLOG(logDEBUG) << "sdsmEncodedMsg: "  << sdsmEncodedMsg;
+
+				//Broadcast the encoded SSM message


paulbourelly999 · 2023-09-11T16:01:37Z

src/v2i-hub/CARMAStreetsPlugin/src/J3224ToSDSMJsonConverter.cpp

+            }
+        }
+
+        //std::cout << SDSMDataJson.toStyledString() << std::endl;


Does this method still exist. Any reason why we would keep this commented out line in the final commit?

paulbourelly999 · 2023-09-11T16:04:17Z

src/v2i-hub/CARMAStreetsPlugin/src/J3224ToSDSMJsonConverter.h

+namespace CARMAStreetsPlugin
+{
+
+    // Template to use when created shared pointer objects for optional data


Can we create a TODO comment to move this these functions to a more reusable place since they may be helpful in general for manipulating asn1c generated structs.

paulbourelly999 · 2023-09-11T16:05:34Z

src/v2i-hub/CARMAStreetsPlugin/src/JsonToJ3224SDSMConverter.cpp

+{
+
+    // Template to use when created shared pointer objects for optional data
+    template <typename T>


Wait why are we redefining functions we already defined in the previous header file?

paulbourelly999 · 2023-09-11T16:09:03Z

src/v2i-hub/CARMAStreetsPlugin/src/JsonToJ3224SDSMConverter.cpp

+
+    bool JsonToJ3224SDSMConverter::parseJsonString(const std::string &consumedMsg, Json::Value &sdsmJsonOut) const
+    {
+        const auto jsonLen = static_cast<int>(consumedMsg.length());


Is this method at all unique to sdsms or is it just parsing strings into JSON values? If it is not unique maybe we can add a TODO comment to consolidate all the places parse strings into JSON values and try to reuse a single method. Maybe atleast for CARMA-Streets Plugin.

paulbourelly999 · 2023-09-11T16:26:46Z

src/v2i-hub/CARMAStreetsPlugin/src/JsonToJ3224SDSMConverter.cpp

+                    // lights
+                    auto lights_ptr = CARMAStreetsPlugin::create_store_shared<ExteriorLights_t>(shared_ptrs);
+                    auto lights = static_cast<int16_t>((*itr)["detected_object_data"]["detected_object_optional_data"]["detected_vehicle_data"]["lights"].asInt());
+                    lights_ptr->buf = (uint8_t *)calloc(2, sizeof(uint8_t)); // TODO: find calloc alternative if possible, causes a memory leak


Comment out and leave a TODO comment to fix this optional fields.

…ot-fhwa-OPS/V2X-Hub into feature/sdsm-json-to-cstruct

paulbourelly999 · 2023-09-11T19:19:21Z

src/v2i-hub/CARMAStreetsPlugin/test/test_J3224ToSDSMJsonConverter.cpp

@@ -381,7 +381,7 @@ namespace unit_test
        ASSERT_EQ(5, sdsmJson["objects"][0]["detected_object_data"]["detected_object_common_data"]["speed_confidence_z"].asInt());
        ASSERT_EQ(15000, sdsmJson["objects"][0]["detected_object_data"]["detected_object_common_data"]["heading"].asInt());
        ASSERT_EQ(5, sdsmJson["objects"][0]["detected_object_data"]["detected_object_common_data"]["heading_conf"].asInt());
-        ASSERT_EQ(200, sdsmJson["objects"][0]["detected_object_data"]["detected_object_common_data"]["accel_4_way"]["long"].asInt());
+        ASSERT_EQ(200, sdsmJson["objects"][0]["detected_object_data"]["detected_object_common_data"]["accel_4_way"]["Long"].asInt());


Why is this Long instead of long?

paulbourelly999

Looks good.

willjohnsonk added 2 commits August 25, 2023 10:23

Added conversion between ASN.1 and JSON for SDSM

db06ff8

Added SDSM conversion to C-struct, updated main plugin, minor fixes t…

794866a

…o previous converter

willjohnsonk requested a review from paulbourelly999 September 7, 2023 22:14

paulbourelly999 requested changes Sep 8, 2023

View reviewed changes

Added sourceID/lights, plugin manifest changes

39550fe

paulbourelly999 reviewed Sep 11, 2023

View reviewed changes

willjohnsonk and others added 3 commits September 11, 2023 13:10

Merge branch 'develop' into feature/sdsm-json-to-cstruct

2e5eb8d

Naming fixes, removal of lights, TODO additions

a86cc09

Merge branch 'feature/sdsm-json-to-cstruct' of https://github.com/usd…

35d1e25

…ot-fhwa-OPS/V2X-Hub into feature/sdsm-json-to-cstruct

paulbourelly999 reviewed Sep 11, 2023

View reviewed changes

Changed JSON Long fields to be named long

eff13fc

paulbourelly999 approved these changes Sep 11, 2023

View reviewed changes

paulbourelly999 merged commit 5328f89 into develop Sep 11, 2023
7 of 8 checks passed

paulbourelly999 deleted the feature/sdsm-json-to-cstruct branch September 11, 2023 20:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added conversion between JSON and ASN.1 C-struct for SDSM with Kafka updates #562

Added conversion between JSON and ASN.1 C-struct for SDSM with Kafka updates #562

willjohnsonk commented Sep 7, 2023 •

edited by paulbourelly999

Loading

willjohnsonk commented Sep 8, 2023

paulbourelly999 Sep 8, 2023

paulbourelly999 Sep 8, 2023

willjohnsonk Sep 8, 2023

paulbourelly999 Sep 8, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 Sep 11, 2023

paulbourelly999 left a comment

Added conversion between JSON and ASN.1 C-struct for SDSM with Kafka updates #562

Added conversion between JSON and ASN.1 C-struct for SDSM with Kafka updates #562

Conversation

willjohnsonk commented Sep 7, 2023 • edited by paulbourelly999 Loading

PR Details

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

willjohnsonk commented Sep 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulbourelly999 left a comment

Choose a reason for hiding this comment

willjohnsonk commented Sep 7, 2023 •

edited by paulbourelly999

Loading