Allow talkgroups to match a range of values

[ArrEssJay]

This change allows talkgroups in talkgroups.csv to be defined as a range <min>:<max> (inclusive) as well as by a single value.

This is useful as a P25 phase 2 system that I receive has 'many' active talkgroups numbered 100-4000, all encrypted, for which I have no further details. Adding each one manually when it is discovered (phase 2 encrypted calls aren't flagged as such in trunking messages) is frustrating, and adding 4000 or so talkgroups, all with a name of 'unknown' and a priority of -1 seems rather inelegant.

This change seeks to remedy this. I had considered adding wildcard/regex matching, but this seems like the simplest solution.

[robotastic]

Let me go dig into this more... it seems like Phase 2 should be flagging encrypted message in the control stream. Maybe I am not processing something correctly. It seems like a huge pain to have to flag all the encrypted talkgroups manually.

[robotastic]

Could you try setting the newCallFromUpdate setting to false in config.json? What is prob happening is that those Calls are being started off UPDATE messages, which do not have the encryption flag set in them.

newCallFromUpdate		true	true / false	Allow for UPDATE trunking messages to start a new Call, in addition to GRANT messages. This may result in more Calls with no transmisions, and use more Recorders. The flipside is that it may catch parts of a Call that would have otherwise been missed. Turn this off if you are running out of Recorders.

[ArrEssJay]

The calls are from grants on patched groups. Phase 2 traffic is not being flagged as encrypted in trunking messages for (what seems like) all of the P2 calls I receive, but I've not exhaustively checked how they're being setup.

Relevant log messages:

023-11-13 11:33:51.833894] (debug)   [ 0]tsbk02	MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT	Channel ID: 53292	Freq: 165.462500 MHz	sg    3423	TDMA 0	sa 320717
[2023-11-13 11:33:51.834031] (info)   [allambee-353]	1C	TG:       3423 (                      -)	Freq: 165.462500 MHz	TG not in Talkgroup File
[2023-11-13 11:33:51.834091] (info)   [allambee-353]	1C	TG:       3423 (                      -)	Freq: 165.462500 MHz	Starting P25 Recorder Num [0]	TDMA: true	Slot: 0	QPSK: true
[2023-11-13 11:33:52.111779] (debug)   [ 0]tsbk30 TDMA SYNCHRONIZATION BROADCAST
[2023-11-13 11:33:52.111942] (debug)   [ 0]tsbk20	Acknowledge Response	ga 65533	sa 3115382	Reserved: 255
[2023-11-13 11:33:52.112048] (debug)   [ 0]tsbk20	Acknowledge Response	ga 65533	sa 3112801	Reserved: 255
11/13/23 11:33:52.112439 MAC_PTT: srcaddr=320717, grpaddr=3423, algid=84, keyid=5d, mi=fd 87 5c 34 1a 16 51 21 00, rs_errs=0
11/13/23 11:33:52.112754 MAC_PTT: srcaddr=320717, grpaddr=3423, algid=84, keyid=5d, mi=fd 87 5c 34 1a 16 51 21 00, rs_errs=0
11/13/23 11:33:52.287761 [0] 4V_BURST(0) ESS: algid=80, keyid=0, mi=00 00 00 00 00 00 00 00 00, rs_errs=0

[ArrEssJay]

I tried adding parsing of voice service options for MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT:

     unsigned long opts  = bitset_shift_mask(tsbk, 72, 0xff);
        bool emergency = (bool)bitset_shift_mask(tsbk, 72, 0x80);
        bool encrypted = (bool)bitset_shift_mask(tsbk, 72, 0x40);
        bool duplex = (bool)bitset_shift_mask(tsbk, 72, 0x20);
        bool mode = (bool)bitset_shift_mask(tsbk, 72, 0x10);
        int priority = bitset_shift_mask(tsbk, 72, 0x07);
        
        unsigned long ch = bitset_shift_mask(tsbk, 56, 0xffff);
        unsigned long sg = bitset_shift_mask(tsbk, 40, 0xffff);
        unsigned long sa = bitset_shift_mask(tsbk, 16, 0xffffff);
        unsigned long f = channel_id_to_frequency(ch, sys_num);

        message.message_type = GRANT;
        message.freq = f;
        message.talkgroup = sg;
        message.source = sa;
        
        message.encrypted = encrypted;
        message.emergency = emergency;
        message.duplex = duplex;
        message.mode = mode;
        message.priority = priority;

      if (get_tdma_slot(ch, sys_num) >= 0) {
        message.phase2_tdma = true;
        message.tdma_slot = get_tdma_slot(ch, sys_num);
      } else {
        message.phase2_tdma = false;
        message.tdma_slot = 0;
      }

      os << "tsbk02\tMOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT\tChannel ID: " << std::setw(5) << ch << "\tFreq: " << format_freq(f) << "\tsg " << std::setw(7) << sg << "\tTDMA " << get_tdma_slot(ch, sys_num) << "\tsa " << sa;
      message.meta = os.str();
      BOOST_LOG_TRIVIAL(debug) << os.str();

These calls are now correctly flagged as encrypted.

[2023-11-13 12:00:42.266377] (debug)   [ 0]tsbk02	MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT	Channel ID: 53316	Freq: 165.612500 MHz	sg    3423	TDMA 0	sa 3112062
[2023-11-13 12:00:42.273226] (info)   [allambee-353]	1C	TG:       3423 (                      -)	Freq: 165.612500 MHz	Not Recording: ENCRYPTED - src: 3112062

Somewhat confusing, I appreciate, given its a manufacturer specific variant of an update message.

Whether the call is phase 1 or 2 doesn't seem to be the issue here. I've seen calls exhibiting the issue using either.

I do also receive MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT_UPDATE messages. I'll double check whether the service options are present here too.

[ArrEssJay]

And as expected MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT_UPDATE messages do not contain service options, in which case yes, calls being started from updates would be a problem. Unfortunately I have issues with co-channel interference and do drop trunking messages from some transmission sites, so this isn't ideal.

Perhaps it would make sense for the recorder to abort with an error when encrypted data is received rather than attempt to decode it and return an empty/gibberish audio file?

[robotastic]

I like that idea of just having a recorder Stop when it receives encrypted - the call would have to stick around and change from being in Record mode to Monitor mode, otherwise it would just start back up again when the next UPDATE message comes through.

One other thought - how about setting the recordUknown key to false in the system section of the config file? That way only the defined talkgroups will be recorded. The only downside is that you would miss any new talkgroups that pop up.

recordUnknown		true	true / false	Record talkgroups if they are not listed in the Talkgroups File.

Ultimately adding support for ranges might be the best option... I am just trying to keep the code base simple as possible and seeing if it is possible to address the root problem.

[robotastic]

Could you add a PR for handling MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT? Nice detective work!

[ArrEssJay]

#869 for MOTOROLA_OSP_PATCH_GROUP_CHANNEL_GRANT

It's straightforward to check whether encryption can/does succeed or not by checking the return state of crypt_algs.prepare(). I'm just taking a look to see if there's a straightforward/reliable way to pass a message back up the stack to the recorder to abort the call if we know it's going to fail.

[robotastic]

Sorry for the delay on this - this seems like a good feature to no matter what TR should be doing with encryption. Since this PR came in, the handling for CSV files changed around. I will try to rebase it with the new approach