Fermi #199

shb46 · 2024-09-20T17:24:56Z

Fermi GBM Schema for a single schema, "ALERT" - Initial Version

gcn/notices/fermi/gbm/FermiGBMAlert.schema.json

Vidushi-GitHub · 2024-10-02T13:46:15Z

gcn/notices/fermi/gbm/strict/FermiGBMAlert.schema.json

+      "type": "string",
+      "description": "Date and time of notice creation [UTC, ISO 8601], ex YYYY-MM-DDTHH:MM:SS.ssssssZ"
+    },
+    "alert_tense": {


It's already present in core schema, you just have to ref the schema.
Same for most of the properties below, rate_snr, trigger_time etc.

Vidushi-GitHub · 2024-10-02T13:48:47Z

We have planned a mission page, where you can keep the content present at readme file.

Vidushi-GitHub · 2024-10-02T13:58:36Z

Hi @shb46, sorry for the delayed response due to travel and vacation. Most of the stuff looks fine.
My main confusion about schema is why strict schema there and why you are re-writing core properties, instead of ref?

shb46 · 2024-10-02T21:27:59Z

Hi @Vidushi-GitHub,

Hope everything is well! Here are my replies:

We have planned a mission page, where you can keep the content present at readme file.

Can we have an instrument page as well? the lat and the GBM instruments are run by separate teams.
Can notice type have its own page?

Hi @shb46, sorry for the delayed response due to travel and vacation. Most of the stuff looks fine. My main confusion about schema is why strict schema there and why you are re-writing core properties, instead of ref?

Those are two independent schemas intended to describe the same notice type:

The "Kafka" one: fermi/gbm/FermiGBMAlert.schema.json
- Makes $ref-s to the core schema.
- Doesn't $ref the strict schema.
- Validates new notices received by the Kafka server.
- More permissive compared to the strict schema.
The "strict" schema: fermi/gbm/strict/FermiGBMAlert.schema.json:
- Doesn't $ref the core schema, nor the "Kafka" schema.
- Is less permissive than the Kafka schema.
- Clarifies what to expect in a notice, incl. which core fields are not used.
- Used by the producer, in addition to the Kafka schema.

[Edit:] I should rename the strict schema file so it has a different name.

Vidushi-GitHub · 2024-10-10T19:46:31Z

Hi Boyan,
Could you please change the filename?
fermi/gcn/notices/fermi/gbm/FermiGBMAlert.schema.json to fermi/gcn/notices/fermi/gbm/alert.schema.json?

As it's already in fermi/gbm folder, again fermiGBM is redundant.

Vidushi-GitHub · 2024-10-10T19:56:26Z

We have planned a mission page, where you can keep the content present at readme file.

Can we have an instrument page as well? the lat and the GBM instruments are run by separate teams.

Can notice type have its own page?

For Fermi mission page, we have information for GCN Classic Notices at https://gcn.nasa.gov/missions/fermi, and would move forward with updating it.
@jracusin shall we separate out GBM, LAT content into two mission pages as available at https://gcn.nasa.gov/missions/fermi Or, just update the existing one as single page?

Vidushi-GitHub · 2024-10-10T20:12:04Z

Those are two independent schemas intended to describe the same notice type:

The "Kafka" one: fermi/gbm/FermiGBMAlert.schema.json

Makes $ref-s to the core schema.

Doesn't $ref the strict schema.

Validates new notices received by the Kafka server.

More permissive compared to the strict schema.

The "strict" schema: fermi/gbm/strict/FermiGBMAlert.schema.json:

Doesn't $ref the core schema, nor the "Kafka" schema.

Is less permissive than the Kafka schema.

Clarifies what to expect in a notice, incl. which core fields are not used.

Used by the producer, in addition to the Kafka schema.

[Edit:] I should rename the strict schema file so it has a different name.

I am trying to understand why do you wanna create two schemas for same notice types? What is advantage of not referring $ref or kafka schema? Would you produce same notices from two schemas?
Definitions/clarifications shouldn't be problem, that can be added in alert schema as well.

@lpsinger, @jracusin do you have comments on the same notices from different schemas, alert ($ref to core) and strict (own definitions)?

shb46 · 2024-10-10T21:17:20Z

@jracusin:
Re. separate GBM/LAT pages.
Please let me know if it is possible. If yes, we should get the consensus of the GBM and LAT teams before we make a change. I think it will work with or without a "mission" page, i.e. either one of:

Missions

Fermi

GBM

LAT

or

Missions

FermiGBM

FermiLAT

shb46 · 2024-10-10T21:17:31Z

@Vidushi-GitHub: File renamed. The JPG and MD files removed.

shb46 · 2024-10-11T01:17:54Z

@jracusin: P.S.: I'm not planning to change the mission page at this time.

dakota002 · 2024-10-11T14:45:42Z

Hi @shb46 , I have a couple additional requests for this update:

Can you rename the strict schema and examples as well:

gcn/notices/fermi/gbm/strict/FermiGBMStrictAlert.schema.json -> gcn/notices/fermi/gbm/strict/alert.schema.json
gcn/notices/fermi/gbm/strict/FermiGBMAlert1.example.json -> gcn/notices/fermi/gbm/strict/alert.FermiGBMAlert1.example.json
gcn/notices/fermi/gbm/strict/FermiGBMAlert2.example.json -> gcn/notices/fermi/gbm/strict/alert.FermiGBMAlert2.example.json
gcn/notices/fermi/gbm/strict/FermiGBMStrictAlert1.example.json -> gcn/notices/fermi/gbm/strict/alert.FermiGBMStrictAlert1.example.json
gcn/notices/fermi/gbm/strict/FermiGBMStrictAlert2.example.json -> gcn/notices/fermi/gbm/strict/alert.FermiGBMStrictAlert2.example.json

The schema name change reason is the same as the other schema as @Vidushi-GitHub mentioned, as the path already gives the name. As for the examples, the pattern of: alert.[any specific name for the example].example.json is recognized in our schema browser and parsers, and will help automate our docs.

Once this is all set, could you then squash your commits down to 1 single commit? I can assist in this if you have questions before doing so.

shb46 · 2024-10-11T15:50:20Z

Hi, @dakota002,

Please check the new file names before squashing.

dakota002 · 2024-10-11T17:36:11Z

Hi, @dakota002,

Please check the new file names before squashing.

The names look good to me!

shb46 · 2024-10-11T18:48:21Z

@dakota002: Did I squash it enough?

dakota002 · 2024-10-11T19:50:59Z

@dakota002: Did I squash it enough?

I don't believe so, I see 52 commits now. You may have pulled the history back into your branch before pushing it out. You should just run the command git push -f to assert that the new history is correct

shb46 · 2024-10-14T14:59:40Z

@dakota002: Can't figure it out. I'm going to need your help with this.

Vidushi-GitHub · 2024-10-28T13:15:49Z

gcn/notices/fermi/gbm/alert.schema.json

+      "type": "string",
+      "description": "Notice title"
+    },
+    "hitl": {


Would you like to rename it as "automatic", rather than providing a short form?
This field can be common in other missions as well and may create different name schemes.

@jracusin this field might be common for other missions, shall we add it in core schema?

Greetings, @Vidushi-GitHub and @jracusin:

To have it in the Core schema makes much more sense to me. (See: #202) In such case, we can simply drop it from this schema (GBM Alert) and survive until you decide on a name and update the Core. Some of our other notices use it regularly, but that is for the future.

So far, Fermi GBM has had it as "HITL" or "Human-in-the-Loop". I prefer to keep it this way. IMHO, "Automatic" inverts what people are used to receive and, secondly, the label is more ambiguous than "HITL". You should probably see if any other missions have an analog.

HITL was used in GCN Classic notices, so I'm okay with keeping it, as long as it's well described. It's not automatic, it is at least partially manual.

Could this instead be an enum with different types of localizations rather than a confusing boolean? e.g. flight, ground, HITL?

@jracusin: Then we would be more stright forward to name it "human localization". HITL Is more general. I don't mind dropping it from the Alert. It should be in the Core anyway.

After discussions "HILT" specific to GBM is fine.
Style looks inconsistent to other schema fields, if possible make the property name explicit like, human_in_loop or human_in_the_loop.

@shb46 are you considering to put it as enum?

Let's move this forward to some conclusion.

@Vidushi-GitHub: I renamed the field and adjusted the description accordingly. I'm keeping it as a boolean.

Vidushi-GitHub

It looks good to me.

The only issue we need to address is the strict schema being used.
Would you like to create it for validation purposes only, or do you plan to have an additional topic for strict schema for the team's use?

shb46 · 2024-10-28T15:32:08Z

Hi, @Vidushi-GitHub:

Regarding your latest comment.

It looks good to me.

Let's not forget the "hitl" field before approving, in case we decide to drop it.

The only issue we need to address is the strict schema being used. Would you like to create it for validation purposes only, or do you plan to have an additional topic for strict schema for the team's use?

The producer will validate against both the non-strict and the strict schemas at its end. Kafka can use only the non-strict. It also serves as a source-as-documentation.

jracusin · 2024-10-31T15:34:34Z

We have planned a mission page, where you can keep the content present at readme file.

Can we have an instrument page as well? the lat and the GBM instruments are run by separate teams.

Can notice type have its own page?

For Fermi mission page, we have information for GCN Classic Notices at https://gcn.nasa.gov/missions/fermi, and would move forward with updating it. @jracusin shall we separate out GBM, LAT content into two mission pages as available at https://gcn.nasa.gov/missions/fermi Or, just update the existing one as single page?

No, we should modify the existing Fermi mission page, and can have sections for GBM & LAT.

jracusin · 2024-10-31T15:38:04Z

Hi, @Vidushi-GitHub:

Regarding your latest comment.

It looks good to me.

Let's not forget the "hitl" field before approving, in case we decide to drop it.

The only issue we need to address is the strict schema being used. Would you like to create it for validation purposes only, or do you plan to have an additional topic for strict schema for the team's use?

The producer will validate against both the non-strict and the strict schemas at its end. Kafka can use only the non-strict. It also serves as a source-as-documentation.

Each schema is for a topic, so you cannot produce multiple schema to the same topic.

gcn/notices/fermi/gbm/alert.schema.json

gcn/notices/fermi/gbm/strict/alert.schema.json

Vidushi-GitHub · 2024-11-05T18:25:31Z

gcn/notices/fermi/gbm/detectors.schema.json

+      },
+      "minItems": 14,
+      "maxItems": 14,
+      "description": "Triggered detectors"


Please expand it

@Vidushi-GitHub: Description improved.

gcn/notices/fermi/gbm/strict/alert.schema.json

Vidushi-GitHub · 2024-11-05T18:34:46Z

gcn/notices/fermi/gbm/strict/alert.schema.json

+    },
+    "trigger_algorithm_number": {
+      "type": "number",
+      "description": "The criterion number that caused the flight software to trigger. Not to be confused with flight software or any other version number."


please make it consistent with the above definition.

gcn/notices/fermi/gbm/strict/alert.schema.json

gcn/notices/fermi/gbm/strict/alert.2strict.example.json

gcn/notices/fermi/gbm/strict/alert.2.example.json

…mber with which the units go together."

Vidushi-GitHub · 2024-11-06T03:12:20Z

gcn/notices/fermi/gbm/strict/alert.1.example.json

+    "on"
+  ],
+  "event_name": ["GRB240824A"],
+  "hitl": false,


Please make the example field name consistent with schema.
Additionally, validate the schema against examples, in case any change is left out.

@Vidushi-GitHub: Changed. Thank you for the reminder!

shb46 requested a review from Vidushi-GitHub September 25, 2024 14:56

shb46 marked this pull request as draft September 25, 2024 14:56

shb46 self-assigned this Sep 25, 2024

shb46 requested a review from jracusin October 1, 2024 00:56

Vidushi-GitHub reviewed Oct 2, 2024

View reviewed changes

gcn/notices/fermi/gbm/FermiGBMAlert.schema.json Outdated Show resolved Hide resolved

Vidushi-GitHub reviewed Oct 2, 2024

View reviewed changes

shb46 requested a review from Vidushi-GitHub October 2, 2024 14:16

dakota002 mentioned this pull request Oct 9, 2024

Fermi-GBM JSON Notices nasa-gcn/gcn.nasa.gov#2605

Open

5 tasks

shb46 force-pushed the fermi branch from 0a8ef0e to 890537f Compare October 15, 2024 17:32

Fermi gbm

dc47b17

dakota002 force-pushed the fermi branch from 890537f to dc47b17 Compare October 15, 2024 19:35

dakota002 marked this pull request as ready for review October 15, 2024 19:42

shb46 added 2 commits October 17, 2024 12:05

commit

42ec420

Corrections to examples

96bf975

shb46 force-pushed the fermi branch from 53c2cd0 to 96bf975 Compare October 17, 2024 17:16

Vidushi-GitHub reviewed Oct 28, 2024

View reviewed changes

Vidushi-GitHub requested changes Oct 28, 2024

View reviewed changes

jracusin reviewed Oct 31, 2024

View reviewed changes

gcn/notices/fermi/gbm/alert.schema.json Show resolved Hide resolved

jracusin reviewed Oct 31, 2024

View reviewed changes

gcn/notices/fermi/gbm/strict/alert.schema.json Outdated Show resolved Hide resolved

$schema property fix

896bf94

Vidushi-GitHub reviewed Nov 5, 2024

View reviewed changes

gcn/notices/fermi/gbm/strict/alert.schema.json Outdated Show resolved Hide resolved

Vidushi-GitHub reviewed Nov 5, 2024

View reviewed changes

gcn/notices/fermi/gbm/strict/alert.schema.json Outdated Show resolved Hide resolved

Vidushi-GitHub reviewed Nov 5, 2024

View reviewed changes

gcn/notices/fermi/gbm/strict/alert.2strict.example.json Outdated Show resolved Hide resolved

Vidushi-GitHub reviewed Nov 5, 2024

View reviewed changes

gcn/notices/fermi/gbm/strict/alert.2.example.json Outdated Show resolved Hide resolved

shb46 added 5 commits November 5, 2024 20:16

Issue nasa-gcn#199: hitl -> human_in_th_loop

c3bb0a8

Issue nasa-gcn#199: Elaborate on "detectors_status" description.

5d7ca02

Issue nasa-gcn#199: Typo in the rate_duration description

c5576ab

Issue nasa-gcn#199: Specify explicitly "rate_energy_range", as the nu…

6bf5331

…mber with which the units go together."

Issue nasa-gcn#199: "real-life" -> "streamed/real".

cb904dd

Vidushi-GitHub reviewed Nov 6, 2024

View reviewed changes

Issue nasa-gcn#199: Renamed fields in examples to reflect schema

07f1cbc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fermi #199

Fermi #199

shb46 commented Sep 20, 2024

Vidushi-GitHub Oct 2, 2024 •

edited

Loading

Vidushi-GitHub commented Oct 2, 2024

Vidushi-GitHub commented Oct 2, 2024

shb46 commented Oct 2, 2024 •

edited

Loading

Vidushi-GitHub commented Oct 10, 2024

Vidushi-GitHub commented Oct 10, 2024

Vidushi-GitHub commented Oct 10, 2024

shb46 commented Oct 10, 2024

shb46 commented Oct 10, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 14, 2024

Vidushi-GitHub Oct 28, 2024

shb46 Oct 28, 2024

jracusin Oct 31, 2024

jracusin Oct 31, 2024

shb46 Oct 31, 2024

Vidushi-GitHub Nov 5, 2024 •

edited

Loading

shb46 Nov 6, 2024

Vidushi-GitHub left a comment

shb46 commented Oct 28, 2024

jracusin commented Oct 31, 2024

jracusin commented Oct 31, 2024

Vidushi-GitHub Nov 5, 2024

shb46 Nov 6, 2024

Vidushi-GitHub Nov 5, 2024 •

edited

Loading

Vidushi-GitHub Nov 6, 2024

shb46 Nov 6, 2024

Fermi #199

Are you sure you want to change the base?

Fermi #199

Conversation

shb46 commented Sep 20, 2024

Vidushi-GitHub Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

Vidushi-GitHub commented Oct 2, 2024

Vidushi-GitHub commented Oct 2, 2024

shb46 commented Oct 2, 2024 • edited Loading

Vidushi-GitHub commented Oct 10, 2024

Vidushi-GitHub commented Oct 10, 2024

Vidushi-GitHub commented Oct 10, 2024

shb46 commented Oct 10, 2024

shb46 commented Oct 10, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 11, 2024

dakota002 commented Oct 11, 2024

shb46 commented Oct 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vidushi-GitHub Nov 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vidushi-GitHub left a comment

Choose a reason for hiding this comment

shb46 commented Oct 28, 2024

jracusin commented Oct 31, 2024

jracusin commented Oct 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vidushi-GitHub Nov 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vidushi-GitHub Oct 2, 2024 •

edited

Loading

shb46 commented Oct 2, 2024 •

edited

Loading

Vidushi-GitHub Nov 5, 2024 •

edited

Loading

Vidushi-GitHub Nov 5, 2024 •

edited

Loading