Functionality of ROI readout (in TD) in MLT

[MRiganSUSX]

No description provided.

[MRiganSUSX]

There are 4 associated PRs:

• trigger 242
• daqconf 398
• fddaqconf 6
• nddaqconf 7

This is one of the deliverables mentioned in daq-deliverables 109.

---

Overview: The goal is/was to be able to have an ROI selection in the trigger's MLT - ie to be able to read out a subsection of subdetectors (fragment producers). Until now, trigger decisions were made to read out everything. A 'working' solution for this was developed (mrigan/mlt_roi branches in trigger, daqconf, fddaqconf, triggeralgs) but was too 'rough' and (a) involved changing the data objects (TAs, TCs), as well as (b) modified all existing makers, and finally (c) used the concept of regions that were not clearly/permanently defined. The decision was made to postpone this to rethink how to best do this, even if it could involve a partial overhaul of the daq/trigger.

---

The request was then changed to having the ability to readout a 'random' selection of links for a trigger decision (not necessarily one associated with the TA/TC/TD). This would make use of part of the initial solution (specifically MLT's ability to modify requested components for a TD), and would be a useful functionality to test mlt-readout connection, as well as overall capabilities of handling high rates of data. This is what this deliverable is aimed at.

---

The logic:

CONFIG LINKS MAP:

• previously, daqconf provided an array of 'links' to the MLT. Entries in this array were SourceID of fragment producers that can be read out. This includes: TP,TA,TC buffers, HSI interface, Detector Readouts...
• Now, the map is split into:
  • "mandatory_links": there are links that are always going to be read out for each TD: HSI interface, TC buffer
  • "groups_links": there are other links, grouped according to how they link within the trigger 'flow', representing a single tplink source (and the related DLHs). Example would be a single tplink source + associated TP DLH + DLHs + buffers (TP, TA).
  • (the map has the same number of links as before, they are just arranged differently now)

DAQCONF:

• the whole ROI is optional behind a flag "mlt_use_roi_readout" which is false by default. Ie, unless specified MLT will create TDs that request all components, just as it was until now. Now change should be observed.
• There is additional field that can be passed to the MLT that specifies the probabilities of reading out a number of channels (default values shown below):

"mlt_roi_conf": [
            {
                "groups_selection_mode": "kRandom",
                "number_of_link_groups": 1,
                "probability": 0.8,
                "time_window": 1000
            },

• "mlt_roi_conf" is the daqconf name for a table/map that can be passed to the MLT
• "number_of_link_groups": represents the number of the aforementioned groups (think of it as single APA) to be readout
• "probability": this is the likelihood/weight of this option to be selected (1 is total)
• "time_window": the pre/post time to be associated with the requests (clock ticks)
• "groups_selection_mode": this has options of "kRandom" / "kSequential". Random will select groups ('APAs') at random, Sequential will read them by ID.
• The idea is that many such group selection options can be passed. Then one is selected for each TD, given the weights. Once it is selected, this TD will only request this configuration's number of links, for given time; and the groups are either selected at random or by ID.
• for example, if we had this configuration:

"mlt_roi_conf":[
    {
    "number_of_link_groups": 1,<br>
    "probability": 0.8,
    "time_window": 1000,
    "groups_selection_mode": "kRandom"},
    {"number_of_link_groups": 2,
    "probability": 0.2,
    "time_window": 1000,
    "groups_selection_mode": "kSequential"}]

we have 80% chance that 2 APAs are readout and 20% chance 1 APA is readout; for each TD. The coin is tossed when new TD is made.

• There are some addition checks for this in the trigger gen:
  • the number of requested links cannot be greater than actual available links
  • the probabilities should not be >1 (although trigger can actually handle that)

TRIGGER:
Changes in trigger:

• associated changes in jsonnet to allow new daqconf variables:
  • "use_roi_readout": flag to enable/disable ROI readout
  • "roi_conf": the configuration table for ROI readout (explained above in daqconf)
• New objects / functions (some non-essentials are omitted):
  • std::vector<dfmessages::SourceID> m_mandatory_links: holds array of links to be always read out
  • std::map<int, std::vector<dfmessages::SourceID>> m_group_links: holds map of groups of links that can potentially be read out
  • void parse_group_links(const nlohmann::json& data): parses complicated json object into nice map
  • dfmessages::ComponentRequest create_request_for_link(...): creates request object (links + times) for a provided link
  • std::vector<dfmessages::ComponentRequest> create_all_decision_requests(...): uses above for each link from provided vector of links (ie group or multiple groups)
  • void add_requests_to_decision(...): adds a request for component (link) to TD
  • bool m_use_roi_readout: the flag for ROI readout, affects the whether we read out everything or pick groups
  • std::map<int, roi_group> m_roi_conf: a map holding the daqconf blocks specifying probabilities for links to be readout (see daqconf above, this represents "mlt_roi_conf" table)
  • '''struct roi_group
    {
    int n_links;
    float prob;
    triggeralgs::timestamp_t time_window;
    std::string mode;
    }
    '''
  • above is a new struct created to specifically hold blocks (configurations) from "mlt_roi_conf"
  • std::map<int, roi_group> m_roi_conf: map of blocks described above, if we have multiple
  • void parse_roi_conf(const nlohmann::json& data): similar to other parse functions, links jsonnet objects to c++ objects
  • std::vector<int> m_roi_conf_ids: vector holding IDs for sequential readouts
  • std::vector<float> m_roi_conf_probs: vector for probabilities
  • std::vector<float> m_roi_conf_probs_c: vector for probabilities (cumulative)
  • float get_random_num_float(float limit): picks a roi group (weighted by probabilities from daqconf)
  • int get_random_num_int(): random int up to number of group links
  • int pick_roi_group_conf(): uses above to pick link group to be read out (remember ROI configuration gives number of group links to be readout, this function picks the groups for Random mode. Only unique ones)
  • void roi_readout_make_requests(dfmessages::TriggerDecision& decision): does most of ROI readout logic (bring things together):
    • picks ROI configuration (how many groups, mode, time, mode)
    • creates vector of links (given number of groups above, and the mode)
    • create requests for all these links
    • adds all there request to a TD
• Logic:
  • MLT loads whether we are using ROI readout or not. If not, eevry single TD will request all links.
  • If we are in ROI, when a TD is formed:
    • toses a coin to pick roi configuration (these have probabilities)
    • for picked configuration, picks groups of links
    • creates requests for all these selected links
    • adds to TD
    • continues as normal

Test configs:

• "mlt_use_roi_readout" is False by default, so no change to TDs and readout.
• very 'safe' ROI configration is set as default:

"mlt_roi_conf": [
            {
                "groups_selection_mode": "kRandom",
                "number_of_link_groups": 1,
                "probability": 0.8,
                "time_window": 1000
            },

• this means 1 links will be requested.
• many other blocks like this can be added, for example:

    "mlt_roi_conf":[
      {"number_of_link_groups": 1,
       "probability": 0.5,
       "time_window": 1000,
       "groups_selection_mode": "kRandom"
      },
      {"number_of_link_groups": 2,
       "probability": 0.2,
       "time_window": 1000,
       "groups_selection_mode": "kRandom"
      },
      {"number_of_link_groups": 10,
       "probability": 0.2,
       "time_window": 1000,
       "groups_selection_mode": "kSequential"
      },
      {"number_of_link_groups": 150,
       "probability": 0.1,
       "time_window": 1000,
       "groups_selection_mode": "kSequential"
      }
    ]

• in the example above:
  • 50% of time (10% of TDs), 1 group of links (~APA) is read out, groups picked at random
  • 20% of time, 2 groups of links are readout, groups picked at random
  • 20% of time, 10 groups are read, sequentially (by IDs)
  • 10% of time, ALL (if we assume FD with 150) groups are read

Minimal config to have ROI:

  "trigger": {
    "mlt_merge_overlapping_tcs": true,
    "mlt_use_roi_readout": true,
}

In reality (when defaults are filled in, this corresponds to:

  "trigger": {
    "mlt_use_roi_readout": true,
    "mlt_roi_conf":[
            {
                "groups_selection_mode": "kRandom",
                "number_of_link_groups": 1,
                "probability": 0.8,
                "time_window": 1000
            }]

More blocks can be added to "mlt_roi_conf".

Tests and results:
Passes:

• default configs (first and second) from Instructions to set up page
• daqsystem integration tests: minimal_system_quick_test.py, fake_data_producer_test.py
• custom tests with 0, 2, 4 tpset links with 1-5 roi configurations

Notes
In case of using this AND merging TCs (time-wise), need to decide what times to assign to trigger decisions. Currently will use the ones from ROI config provided by daqconf.

Logs
By default:

Use readout map: 0  <- ROI off by default
...
Sending a decision with triggernumber 2 timestamp 79554162207824014 number of links 4 based on TC of type 1   <- all links requested

Using ROI, 2 APAs, 2 ROI configurations:

MLT Group Links:
Group: 0
subsystem: Trigger id: 2  <- TP buffer
 subsystem: Trigger id: 4 <- TA buffer
subsystem: Detector_Readout id: 100 <- DLH
subsystem: Detector_Readout id: 101 <- DLH
subsystem: Trigger id: 0 <- TP DLH
Group: 1
subsystem: Trigger id: 3
subsystem: Trigger id: 5
subsystem: Detector_Readout id: 102
subsystem: Detector_Readout id: 103
subsystem: Trigger id: 1
...
Use ROI readout?: 1
ROI CONF
ID: 0
n links: 1
prob: 0.8
time: 1000
mode: kRandom
ID: 1
n links: 2
prob: 0.1
time: 1000
mode: kSequential
...
Sending a decision with triggernumber 4 timestamp 79554162107130183 number of links 7 based on TC of type 5 <- ROI config 1 selected, 1 APA (APA has 5 associated links, 2 are mandatory - TC buffer and HSI)
Sending a decision with triggernumber 5 timestamp 79554162113530183 number of links 12 based on TC of type 5 <- ROI config 2 selected, 2 APAs

[MRiganSUSX]

Report available here:
MLT_random_ROI.pdf