Cannot use UI auth with JWT login

[MichalNemec]

Description

Hello,
im using dart sdk with JWT token from another auth service and when doing bootstrap i get to this point:

flutter: [Matrix] BootstrapState: BootstrapState.loading
flutter: [Matrix] Upload device signing keys.

after that i get flutter: [Matrix] [Bootstrapping] Error setting up cross signing - M_FORBIDDEN: Require additional authentication

based on information i found it seems like its not dart sdk issue.

this issue is probably related to #15779

Steps to reproduce

1. use jwt authentication
2. try to do uploadCrossSigningKeys https://pub.dev/documentation/matrix_api_lite/latest/matrix_api_lite/MatrixApi/uploadCrossSigningKeys.html
3. see error

Homeserver

private synapse

Synapse Version

1.93.0

Installation Method

Docker (matrixdotorg/synapse)

Database

Postgres

Workers

I don't know

Platform

ubuntu

Configuration

jwt_config:
  enabled: true
  secret: |-
    -----BEGIN CERTIFICATE-----
    REDACTED
    -----END CERTIFICATE-----
  algorithm: "RS256"
  subject_claim: "sub"
  issuer: "REDACTED"
  audiences:
    - "REDACTED"

Relevant log output

2023-10-17 03:04:10,463 - synapse.access.http.8008 - 465 - INFO - POST-16 - 178.255.168.37 - 8008 - {@REDACTED:REDACTED.com} Processed request: 0.019sec/0.000sec (0.006sec, 0.001sec) (0.002sec/0.012sec/6) 61B 401 "POST /_matrix/client/v3/keys/device_signing/upload HTTP/1.1" "Dart/3.1 (dart:io)" [0 dbevts]

Anything else that would be useful to know?

No response

[clokep]

This sounds like the incorrect UI authentication information might be being passed by the SDK? (I'm not 100% sure Synapse even supports UI auth when using JWTs, but it must?)

I'd be curious what the actual response JSON is in the response to /_matrix/client/v3/keys/device_signing/upload.

[MichalNemec]

i just tried it as plain POST call
Third party access token didnt work, so i used access token from matrix synapse

POST https://domain.com/_matrix/client/v3/keys/device_signing/upload
Content-Type: application/json
Authorization: Bearer syt_NmVmZDFlMDgtYjMxMi00ZGY1LTkyNTMtNGNiMWFjOTQzZWU1_IrnBgboBIfbqSDmlwUJY_3U1XvT

{

}

and got back:

HTTP/1.1 401 Unauthorized
Server: nginx/1.18.0 (Ubuntu)
Date: Wed, 18 Oct 2023 05:48:34 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: close
Cache-Control: no-cache, no-store, must-revalidate
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Authorization, Date
Access-Control-Expose-Headers: Synapse-Trace-Id, Server

{
  "session": "IjKdUYEUdTgITLupZCZVvxWy",
  "flows": [],
  "params": {}
}

maybe its a hint, but in synapse database i dont see any session rows in sessions table.

When i try password login with matrix id and password, then this works just fine, but table is still empty.

[clokep]

{
  "session": "IjKdUYEUdTgITLupZCZVvxWy",
  "flows": [],
  "params": {}
}

I think this essentially means that UI auth doesn't work with JWT login. 😢 It should have the org.matrix.jwt login type, I would have thought.

[MichalNemec]

@clokep can i do something about it? Because i would really like to have E2EE, but we have users elsewhere (keycloak), we use jwt from keycloak for everything else.

keys are filled, but auth is not sent

it fails exactly here: https://github.com/famedly/matrix-dart-sdk/blob/main/lib/encryption/utils/bootstrap.dart#L474 and https://github.com/famedly/dart_matrix_api_lite/blob/main/lib/src/generated/api.dart#L1715 maybe alteration to add type or something would help?

[clokep]

Because i would really like to have E2EE, but we have users elsewhere (keycloak), we use jwt from keycloak for everything else.

It should work fine w/ single sign-on, from your questions about the Dart SDK I assume you're embedding a client into something else?

---

Realistically, I don't see the team prioritizing a fix for this. JWT is non-standard (and seldom used). We would accept a pull request fixing this, however.

I think the code near this is incorrect:

synapse/synapse/handlers/auth.py

Lines 392 to 419 in 62a1a9b

	async def _get_available_ui_auth_types(self, user: UserID) -> Iterable[str]:
	"""Get a list of the user-interactive authentication types this user can use."""
	ui_auth_types = set()
	# if the HS supports password auth, and the user has a non-null password, we
	# support password auth
	if self._password_localdb_enabled and self._password_enabled_for_reauth:
	lookupres = await self._find_user_id_and_pwd_hash(user.to_string())
	if lookupres:
	_, password_hash = lookupres
	if password_hash:
	ui_auth_types.add(LoginType.PASSWORD)
	# also allow auth from password providers
	for t in self.password_auth_provider.get_supported_login_types().keys():
	if t == LoginType.PASSWORD and not self._password_enabled_for_reauth:
	continue
	ui_auth_types.add(t)
	# if sso is enabled, allow the user to log in via SSO iff they have a mapping
	# from sso to mxid.
	if await self.hs.get_sso_handler().get_identity_providers_for_user(
	user.to_string()
	):
	ui_auth_types.add(LoginType.SSO)
	return ui_auth_types

[MichalNemec]

It should work fine w/ single sign-on, from your questions about the Dart SDK I assume you're embedding a client into something else?

Signing and chat itself works. Im using dart sdk to integrate into existing app. The encryption bootstrap init leads to this issue.

I can try to make the PR, but im not experienced enough in python to make the change production worthy heh

[clokep]

I think you want to add something like:

if self.hs.config.jwt.jwt_enabled:
    ui_auth_types.add(LoginRestServlet.JWT_TYPE)

I'd probably suggest hacking that in and seeing if it works? We can certainly help with making it production worthy. :)

[MichalNemec]

tried creating it as a module and monkey patch the AuthHandler, but to no awail, module cannot find the class name. Tomorrow is another day.

[MichalNemec]

@clokep so, ive added simply ui_auth_types.add(LoginRestServlet.JWT_TYPE) in _get_available_ui_auth_types but it still does not show up at all, no IF statement, im doing dockerfile like this, where i override the file:

FROM docker.io/matrixdotorg/synapse:latest

COPY overrides/auth.py /usr/local/lib/python3.11/synapse/handlers/auth.py

its still empty

EDIT: hardcoded this function and still flows are empty:

def _auth_dict_for_flows(
        self,
        flows: List[List[str]],
        session_id: str,
    ) -> Dict[str, Any]:
        public_flows = []
        for f in flows:
            public_flows.append(f)

        get_params = {
            LoginType.RECAPTCHA: self._get_params_recaptcha,
            LoginType.TERMS: self._get_params_terms,
        }

        params: Dict[str, Any] = {}

        for f in public_flows:
            for stage in f:
                if stage in get_params and stage not in params:
                    params[stage] = get_params[stage]()

        return {
            "session": session_id,
            "flows": [{"stages": ["org.matrix.login.jwt"]}]#[{"stages": f} for f in public_flows],
            "params": params,
        }

and i can see in container that its replaced

EDIT: when i use username and password, client sometimes throws upload keys failed, but when i access the same endpoint i get this:

HTTP/1.1 401 Unauthorized
Server: nginx/1.18.0 (Ubuntu)
Date: Thu, 19 Oct 2023 09:36:20 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: close
Cache-Control: no-cache, no-store, must-revalidate
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Authorization, Date
Access-Control-Expose-Headers: Synapse-Trace-Id, Server

{
  "session": "bVRoPYqmONgZXBPUotRRHHoS",
  "flows": [
    {
      "stages": [
        "m.login.password"
      ]
    }
  ],
  "params": {}
}

[clokep]

EDIT: hardcoded this function and still flows are empty:

This implies to me that the code isn't running, unfortunately. I suspect trying to develop inside the docker container isn't helping.

The response for username/password auth looks correct.

[MichalNemec]

An update: im embarrassed, i copied it to wrong place in dockerfile. I got to a point where i have to complete stage in uia in client. Now i got to the M_UNKNOWN: Unknown login type org.matrix.login.jwt error, where im sending:

{
    "auth": {
        "type": "org.matrix.login.jwt",
        "session": "<sessionid>",
        "token": "<keycloak_token>",
        "identifier": {
            "type": "m.id.user",
            "user": "<name_of_user>" //without @ and :domain.com
        }
    },
    "master_key": ...,
    "self_signing_key": ...,
    "user_signing_key": ...
}

i guess now i have to modify something else in synapse?

[clokep]

i guess now i have to modify something else in synapse?

Probably worth checking the Synapse logs to see if you can figure out where this comes from? I'm not really sure what might be throwing this.