Fix TAN request with response code 3955 during initialization of the dialog

[meyerj]

This patch fixed a similar issue as in #155 for me with the "Sparkasse Vorderpfalz": Since a few days a push TAN is required even to request an account balance or get transactions of the last view days. Before, this was working with only the PIN.

I tracked the issue down to two issues by debugging the exchanged messages as described in the documentation, for which I propose a fix here.

• The string "DUMMY" sent as the tan_medium_name was somehow rejected by the bank. Even the call to client.get_tan_media() in minimal_interactive_cli_bootstrap() already triggered the error as described in DKB does not accept connections anymore #121, with the first response being

  [...]
                    fints.segments.dialog.HIRMS2( # Rückmeldungen zu Segmenten
                        header = fints.formals.SegmentHeader('HIRMS', 4, 2, 4), # Segmentkopf
                        responses = [ # Rückmeldung
                                    fints.formals.Response( # Rückmeldung
                                        code = '3920',
                                        reference_element = None,
                                        text = 'Zugelassene Zwei-Schritt-Verfahren für den Benutzer.',
                                        parameters = [
                                                '923',
                                            ],
                                    ),
                                    fints.formals.Response( # Rückmeldung
                                        code = '9955',
                                        reference_element = None,
                                        text = 'Auftrag nicht ausgeführt - Die Gerätebezeichnung ist unbekannt. (MBV07390100255)',
                                    ),
                            ],
                    ),
  [...]
  

  followed by

  [...]
                    fints.segments.dialog.HIRMG2( # Rückmeldungen zur Gesamtnachricht
                        header = fints.formals.SegmentHeader('HIRMG', 3, 2), # Segmentkopf
                        responses = [ # Rückmeldung
                                    fints.formals.Response( # Rückmeldung
                                        code = '9050',
                                        reference_element = None,
                                        text = 'Die Nachricht enthält Fehler.',
                                    ),
                                    fints.formals.Response( # Rückmeldung
                                        code = '9800',
                                        reference_element = None,
                                        text = 'Dialog abgebrochen',
                                    ),
                                    fints.formals.Response( # Rückmeldung
                                        code = '9010',
                                        reference_element = None,
                                        text = 'Die angegebene Bankreferenz/Dialog-ID ist nicht gültig.',
                                    ),
                            ],
                    ),
                    fints.segments.dialog.HIRMS2( # Rückmeldungen zu Segmenten
                        header = fints.formals.SegmentHeader('HIRMS', 4, 2, 3), # Segmentkopf
                        responses = [ # Rückmeldung
                                    fints.formals.Response( # Rückmeldung
                                        code = '9010',
                                        reference_element = None,
                                        text = 'Auftrag wegen genereller Fehler in Auftragsnachricht nicht verarbeitet.',
                                    ),
                            ],
                    ),
  [...]
  

  after the subsequent call to client.get_sepa_accounts() (HKSPA1), likely because the dialog was never actually accepted by the server. Should a 9xxx response code already trigger an exception in FinTSDialog.init()?

  Sending an empty tan_medium_name works and triggers the response in the next bullet point. So I set client.selected_tan_medium = '' before calling minimal_interactive_cli_bootstrap(client) to already skip the call to get_tan_media() (with my patch). I guess this only works because there is only one for my account. I tried different strings, the name of my device and the UUID shown in the Push Tan app in "Verbindungen", nothing worked. And even get_tan_media() requires a TAN when the dialog starts, so it is a kind of chicken-and-egg problem and I have not found a way to not add the HKTAN7 segment to the dialog initialization request triggered by get_tan_media().

• With the empty tan_medium_name the bank server responds with a TAN request and code 3955. That was added in Fix #155 -- Implement HKTAN7 and decoupled TAN process #162 to the client's _send_with_possible_retry() method to react on a TAN request when sending a request after the dialog has been established, but not in FinTSDialog.init() if the bank already sends that response:

  [...]
                    fints.segments.dialog.HIRMG2( # Rückmeldungen zur Gesamtnachricht
                        header = fints.formals.SegmentHeader('HIRMG', 3, 2), # Segmentkopf
                        responses = [ # Rückmeldung
                                    fints.formals.Response( # Rückmeldung
                                        code = '3060',
                                        reference_element = None,
                                        text = 'Bitte beachten Sie die enthaltenen Warnungen/Hinweise.',
                                    ),
                                    fints.formals.Response( # Rückmeldung
                                        code = '3920',
                                        reference_element = None,
                                        text = 'Zugelassene Zwei-Schritt-Verfahren für den Benutzer.',
                                        parameters = [
                                                '923',
                                            ],
                                    ),
                            ],
                    ),
                    fints.segments.dialog.HIRMS2( # Rückmeldungen zu Segmenten
                        header = fints.formals.SegmentHeader('HIRMS', 4, 2, 5), # Segmentkopf
                        responses = [ # Rückmeldung
                                    fints.formals.Response( # Rückmeldung
                                        code = '3955',
                                        reference_element = None,
                                        text = 'Auftrag empfangen - Bitte Auftrag in Ihrer App freigeben.(MBT62870200005)',
                                    ),
                            ],
                    ),
                    fints.segments.auth.HITAN7( # Zwei-Schritt-TAN-Einreichung Rückmeldung, version 7
                        header = fints.formals.SegmentHeader('HITAN', 5, 7, 5), # Segmentkopf
                        tan_process = '4', # TAN-Prozess
                        task_reference = '3115-09-28-19.28.05.705284', # Auftragsreferenz
                        challenge = 'Bitte Auftrag in Ihrer App freigeben.', # Challenge
                    ),
  [...]
  

  So I assume it was an oversight in Fix #155 -- Implement HKTAN7 and decoupled TAN process #162 to not also patch the other method?

So these two changes work for me with an application similar to the one in the troubleshooting guide. I have not found a way to retrieve transactions without having to confirm that with a push TAN via the app interactively. Ideally there would be a way to remember that "device" at least for a couple of days, like it also works with interactive logins via the web page.

This patch may resolve #121 and #165, at least when also setting selected_tan_medium to an empty string (or something else if you already know a valid value) before the call to minimal_interactive_cli_bootstrap(f):

f = FinTS3PinTanClient(*client_args, product_id=product_id)
f.selected_tan_medium = ''
minimal_interactive_cli_bootstrap(f)

[DerMistkaefer]

Thanks a lot for this. This fixed the problem for me with "Erzgebirgsparkasse".
But I still need to set f.selected_tan_medium = ''

The second problem is that I now need to confirm the python script as a new device on my bank account every time.
Can we implement something that identify the device to the bank?

[raphaelm]

This might be for this reason:
https://www.f-i.de/fints

Anyone interested in creating a PR to support persisting a device ID?

[raphaelm]

Okay turns out this is already implemented. What you need to do is, at the end of your script, save the value of client.system_id, e.g. to a file. Then, when the script is called next, pass it as FinTS3PinTanClient(…, system_id="saved value"). Can someone try if that helps? If it does, I'll add it to the docs.

[meyerj]

Okay turns out this is already implemented. What you need to do is, at the end of your script, save the value of client.system_id, e.g. to a file. Then, when the script is called next, pass it as FinTS3PinTanClient(…, system_id="saved value"). Can someone try if that helps? If it does, I'll add it to the docs.

Thanks, I will try that in the coming days and report back. Sounds reasonable.

But I still need to set f.selected_tan_medium = ''

Related to that: Would it be reasonable to send the empty string instead of 'DUMMY' by default if no TAN medium is selected explicitly, so changing the default behavior? Or is DUMMY actually mandatory somewhere in the standards?

[meyerj]

Okay turns out this is already implemented. What you need to do is, at the end of your script, save the value of client.system_id, e.g. to a file. Then, when the script is called next, pass it as FinTS3PinTanClient(…, system_id="saved value"). Can someone try if that helps? If it does, I'll add it to the docs.

I tried and could retrieve and store the system_id.

It cannot be passed to the constructor of FinTS3PinTanClient() at the moment because it has no such keyword argument, but it can be set after the client has been constructed with client.system_id = 'save value' and before starting the first dialog.

However, the following exception is triggered then during the very first response triggered by entering the with client context:

Traceback (most recent call last):
  File "/***/python-fints/fints/dialog.py", line 85, in init
    retval = self.send(*segments, internal_send=True)
  File "/***/python-fints/fints/dialog.py", line 157, in send
    self.client.process_response_message(self, response, internal_send=internal_send)
  File "/***/python-fints/fints/client.py", line 241, in process_response_message
    self._process_response(dialog, segment, response)
  File "/***/python-fints/fints/client.py", line 1332, in _process_response
    self.set_tan_mechanism(parameter.security_function)
  File "/***/python-fints/fints/client.py", line 1392, in set_tan_mechanism
    raise Exception("Cannot change TAN mechanism with a standing dialog")
Exception: Cannot change TAN mechanism with a standing dialog

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    main()
  File "./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/***/lib/python3.6/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/***/lib/python3.6/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/***/lib/python3.6/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/***/lib/python3.6/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/***/apps/buchhaltung/management/commands/get_transactions.py", line 92, in handle
    with client:
  File "/***/python-fints/fints/client.py", line 251, in __enter__
    self._standing_dialog.__enter__()
  File "/***/python-fints/fints/dialog.py", line 37, in __enter__
    self.init()
  File "/***/python-fints/fints/dialog.py", line 104, in init
    raise FinTSDialogInitError("Couldn't establish dialog with bank, Authentication data wrong?") from e
fints.exceptions.FinTSDialogInitError: Couldn't establish dialog with bank, Authentication data wrong?

I guess the relevant part of the response is

	                fints.segments.dialog.HIRMG2( # Rückmeldungen zur Gesamtnachricht
	                    header = fints.formals.SegmentHeader('HIRMG', 3, 2), # Segmentkopf
	                    responses = [ # Rückmeldung
	                                fints.formals.Response( # Rückmeldung
	                                    code = '3060',
	                                    reference_element = None,
	                                    text = 'Bitte beachten Sie die enthaltenen Warnungen/Hinweise.',
	                                ),
	                        ],
	                ),
	                fints.segments.dialog.HIRMS2( # Rückmeldungen zu Segmenten
	                    header = fints.formals.SegmentHeader('HIRMS', 4, 2, 4), # Segmentkopf
	                    responses = [ # Rückmeldung
	                                fints.formals.Response( # Rückmeldung
	                                    code = '3050',
	                                    reference_element = None,
	                                    text = 'BPD nicht mehr aktuell, aktuelle Version enthalten.',
	                                ),
	                                fints.formals.Response( # Rückmeldung
	                                    code = '3920',
	                                    reference_element = None,
	                                    text = 'Zugelassene Zwei-Schritt-Verfahren für den Benutzer.',
	                                    parameters = [
	                                            '923',
	                                        ],
	                                ),
	                                fints.formals.Response( # Rückmeldung
	                                    code = '0020',
	                                    reference_element = None,
	                                    text = 'Der Auftrag wurde ausgeführt.',
	                                ),
	                        ],
	                ),
	                fints.segments.bank.HIBPA3( # Bankparameter allgemein, version 3
	                    header = fints.formals.SegmentHeader('HIBPA', 5, 3, 4), # Segmentkopf
	                    bpd_version = 13, # BPD-Version
	                    bank_identifier = fints.formals.BankIdentifier( # Kreditinstitutskennung
	                            country_identifier = '280',
	                            bank_code = '54550010',
	                        ),
	                    bank_name = 'Sparkasse Vorderpfalz', # Kreditinstitutsbezeichnung
	                    number_tasks = 3, # Anzahl Geschäftsvorfallarten pro Nachricht
	                    supported_languages = fints.formals.SupportedLanguages2( # Unterstützte Sprachen
	                            languages = [
	                                    fints.formals.Language2.DE,
	                                ],
	                        ),
	                    supported_hbci_version = fints.formals.SupportedHBCIVersions2( # Unterstützte HBCI-Versionen
	                            versions = [
	                                    '300',
	                                ],
	                        ),
	                ),
	                fints.segments.bank.HIKOM4( # Kommunikationszugang rückmelden, version 4
	                    header = fints.formals.SegmentHeader('HIKOM', 6, 4, 4), # Segmentkopf
	                    bank_identifier = fints.formals.BankIdentifier( # Kreditinstitutskennung
	                            country_identifier = '280',
	                            bank_code = '54550010',
	                        ),
	                    default_language = fints.formals.Language2.DE, # Standardsprache
	                    communication_parameters = [ # Kommunikationsparameter
	                                fints.formals.CommunicationParameter2( # Kommunikationsparameter
	                                    service_type = fints.formals.ServiceType2.HTTPS, # Kommunikationsdienst
	                                    address = 'banking-rp4.s-fints-pt-rp.de/fints30', # Kommunikationsadresse
	                                ),
	                        ],
	                ),

and many more segments.

[meyerj]

The system_id is part of the data blob that is returned by client.destruct() and that can be passed to the constructor via the from_data argument and which is documented here. I tried that already before and the system_id is also sent in the intial dialog request, but apparently the Sparkasse server is not happy with the communication then either:

	                fints.segments.dialog.HIRMG2( # Rückmeldungen zur Gesamtnachricht
	                    header = fints.formals.SegmentHeader('HIRMG', 3, 2), # Segmentkopf
	                    responses = [ # Rückmeldung
	                                fints.formals.Response( # Rückmeldung
	                                    code = '9050',
	                                    reference_element = None,
	                                    text = 'Die Nachricht enthält Fehler.',
	                                ),
	                                fints.formals.Response( # Rückmeldung
	                                    code = '9075',
	                                    reference_element = None,
	                                    text = 'Banking-Programm nicht PSD2-fähig, bitte aktualisieren.',
	                                ),
	                                fints.formals.Response( # Rückmeldung
	                                    code = '9800',
	                                    reference_element = None,
	                                    text = 'Dialog abgebrochen',
	                                ),
	                                fints.formals.Response( # Rückmeldung
	                                    code = '9340',
	                                    reference_element = None,
	                                    text = 'Ungültige Auftragsnachricht: Ungültige Signatur.',
	                                ),
	                        ],
	                ),

[raphaelm]

Here's a patch for adding the system ID in the constructor: #172