Migration from older Remotely Version to the newest one failed how to migrate DB and get login working again

[badummtss]

I*m running Remotely in Docker with the Files here from Github.
Before that I use the Docker run command which could be found in this repo for the older version.

The Problem is:
if I restore my older Version everything is working fine.
After upgrading to the Docker-Compose Version, and also for testing the Docker run only Version, no login is possible.

I have already 2fA Enabled so it's not really possible to get in without.

In the Logs I found this:

root@ct-docker-01:/var/www/remotely/logs# cat Remotely_Server20240623.log 2024-06-23 12:05:00.284 +00:00 [ERR] Navigation failed when changing the location to https://remotely.zerot3ch.de/Identity/Account/Login { EventId: { Id: 4, Name: "NavigationFailed" }, SourceContext: "Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager", TransportConnectionId: "9yiLlEbFKib4e01jFJD-KQ", RequestId: "0HN4J0R7V8CMM:00000001", RequestPath: "/_blazor", ConnectionId: "0HN4J0R7V8CMM", ThreadId: 42 } System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args) at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args) at Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager.<>c__DisplayClass13_0.<<NavigateToCore>g__PerformNavigationAsync|0>d.MoveNext() 2024-06-23 12:05:00.332 +00:00 [ERR] Unhandled exception in circuit 'AtNOfjOkVbREBG9zUi91_d0cBL3Gxuf-PYhEkzV7_Gs'. { EventId: { Id: 111, Name: "CircuitUnhandledException" }, SourceContext: "Microsoft.AspNetCore.Components.Server.Circuits.CircuitHost", TransportConnectionId: "9yiLlEbFKib4e01jFJD-KQ", RequestId: "0HN4J0R7V8CMM:00000001", RequestPath: "/_blazor", ConnectionId: "0HN4J0R7V8CMM", ThreadId: 42 } System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args) at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args) at Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager.<>c__DisplayClass13_0.<<NavigateToCore>g__PerformNavigationAsync|0>d.MoveNext() 2024-06-23 12:06:03.353 +00:00 [ERR] Navigation failed when changing the location to https://remotely.zerot3ch.de/Identity/Account/Login { EventId: { Id: 4, Name: "NavigationFailed" }, SourceContext: "Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager", TransportConnectionId: "C3OwWd4IfkYO7Rt8BeCjLw", RequestId: "0HN4J0R7V8CO6:00000001", RequestPath: "/_blazor", ConnectionId: "0HN4J0R7V8CO6", ThreadId: 32 } System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args) at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args) at Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager.<>c__DisplayClass13_0.<<NavigateToCore>g__PerformNavigationAsync|0>d.MoveNext() 2024-06-23 12:06:03.355 +00:00 [ERR] Unhandled exception in circuit 'f_dSO1nbMYff19uDqSycqdBDxWhi_tYYz3Ju9j4IY9E'. { EventId: { Id: 111, Name: "CircuitUnhandledException" }, SourceContext: "Microsoft.AspNetCore.Components.Server.Circuits.CircuitHost", TransportConnectionId: "C3OwWd4IfkYO7Rt8BeCjLw", RequestId: "0HN4J0R7V8CO6:00000001", RequestPath: "/_blazor", ConnectionId: "0HN4J0R7V8CO6", ThreadId: 32 } System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args) at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args) at Microsoft.AspNetCore.Components.Server.Circuits.RemoteNavigationManager.<>c__DisplayClass13_0.<<NavigateToCore>g__PerformNavigationAsync|0>d.MoveNext() 2024-06-23 12:17:02.582 +00:00 [INF] Application is shutting down. Stopping data cleanup service. { SourceContext: "Remotely.Server.Services.DataCleanupService", ThreadId: 67 } 2024-06-23 12:20:49.262 +00:00 [WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning { EventId: { Id: 60, Name: "UsingEphemeralFileSystemLocationInContainer" }, SourceContext: "Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository", ThreadId: 1 } 2024-06-23 12:20:49.805 +00:00 [WRN] No XML encryptor configured. Key {c3ae4ced-fc66-49bb-bbf1-361f45c93109} may be persisted to storage in unencrypted form. { EventId: { Id: 35, Name: "NoXMLEncryptorConfiguredKeyMayBePersistedToStorageInUnencryptedForm" }, SourceContext: "Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager", ThreadId: 1 } 2024-06-23 12:20:51.516 +00:00 [WRN] Failed to determine the https port for redirect. { EventId: { Id: 3, Name: "FailedToDeterminePort" }, SourceContext: "Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware", RequestId: "0HN4JFRM0EKUC:00000001", RequestPath: "/hubs/service/negotiate", ConnectionId: "0HN4JFRM0EKUC", ThreadId: 10 } 2024-06-23 12:24:27.006 +00:00 [INF] Application is shutting down. Stopping data cleanup service. { SourceContext: "Remotely.Server.Services.DataCleanupService", ThreadId: 7 } 2024-06-23 12:24:28.984 +00:00 [WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning { EventId: { Id: 60, Name: "UsingEphemeralFileSystemLocationInContainer" }, SourceContext: "Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository", ThreadId: 1 } 2024-06-23 12:24:29.460 +00:00 [WRN] No XML encryptor configured. Key {127beb6e-7eb6-45c3-bfa2-d7d587d8ba2d} may be persisted to storage in unencrypted form. { EventId: { Id: 35, Name: "NoXMLEncryptorConfiguredKeyMayBePersistedToStorageInUnencryptedForm" }, SourceContext: "Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager", ThreadId: 1 } 2024-06-23 12:24:30.318 +00:00 [WRN] Failed to determine the https port for redirect. { EventId: { Id: 3, Name: "FailedToDeterminePort" }, SourceContext: "Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware", RequestId: "0HN4JFTN74N6U:00000001", RequestPath: "/hubs/service/negotiate", ConnectionId: "0HN4JFTN74N6U", ThreadId: 5 }

My Docker Compose is only changed in that way that the line "Version" isn't needed anymore with the newer docker compose.

My Docker Compose File:

`
networks:
remotely:
name: remotely-network
driver: bridge
ipam:
config:
- subnet: 172.22.0.0/16
gateway: 172.22.0.1

services:
remotely:
image: immybot/remotely:latest
container_name: remotely
volumes:
- /var/www/remotely:/app/AppData
restart: unless-stopped
ports:
- "5000:5000"
networks:
- remotely
environment:
- ASPNETCORE_ENVIRONMENT=Production
- ASPNETCORE_HTTP_PORTS=5000
# Other ASP.NET Core configurations can be overridden here, such as Logging.
# See https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-8.0
- Serilog__MinimumLevel__Override__Microsoft.AspNetCore=Warning
- Serilog__MinimumLevel__Override__System=Warning

  # Values for DbProvider are SQLite, SQLServer, and PostgreSQL.
  - Remotely_ApplicationOptions__DbProvider=SQLite
  # The Docker gateway will be used in Forwarded Headers, which is necessary to
  # to correctly determine request hostname and scheme within the app.  If you change
  # the network config, you must update this value as well.
  - Remotely_ApplicationOptions__DockerGateway=172.22.0.1
  # This path shouldn't be changed.  It points to the Docker volume.
  - Remotely_ConnectionStrings__SQLite=Data Source=/app/AppData/Remotely.db
  # If using SQL Server, change the connection string to point to your SQL Server instance.
  #- Remotely_ConnectionStrings__SQLServer=Server=(localdb)\\mssqllocaldb;Database=Remotely-Server-53bc9b9d-9d6a-45d4-8429-2a2761773502;Trusted_Connection=True;MultipleActiveResultSets=true
  # If using PostgreSQL, change the connection string to point to your PostgreSQL instance.
  #- Remotely_ConnectionStrings__PostgreSQL=Server=Host=localhost;Database=Remotely;Username=postgres;`

I hope someone is able to help me getting this running again.

[knumsi]

Just my guess if I was in your situation.

Before doing something: BACKUP YOUR DATA or consider it as deleted / lost.

[WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning { EventId: { Id: 60, Name: "UsingEphemeralFileSystemLocationInContainer" }, SourceContext: "Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository", ThreadId: 1 }

[WRN] No XML encryptor configured. Key {127beb6e-7eb6-45c3-bfa2-d7d587d8ba2d} may be persisted to storage in unencrypted form. { EventId: { Id: 35, Name: "NoXMLEncryptorConfiguredKeyMayBePersistedToStorageInUnencryptedForm" }, SourceContext: "Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager", ThreadId: 1 } 2024-06-23 12:24:30.318 +00:00

Just guessing:
The 2FA Keys are perhaps saved in directory "/root/.aspnet/DataProtection-Keys" and they have been destroyed with the container.

I opened up my SQLite Database with this tool:
https://sqlitebrowser.org/

Then went to Tables -> KeyValueRecords
In there is one Key->Value Pair and in the Value there seems to be the Server configuration. There is written

"Require2FA":false

if you disable 2FA.

Also in the Database Table-> RemotelyUsers you can find the field

"TwoFactorEnabled" -> Set to "0" then 2FA is disabled on the user.

I did not try anything of it on my own, I was just curious because I did not understand the 2FA mechanism persistency. Therefor I would be happy to get some feedback.