Hi everyone,
I’m really puzzled by this one. I am running a self-hosted bitwarden install on my Synology NAS. I’ve been running it since 2023 when I switched from KeePassX.
All in all it has been good and reliable. On my NAS I also have WatchTower which updates all my containers on a regular basis. After one of the updates from a month or so ago (yeah, I didn’t have a lot of time to troubleshoot but today I was looking to login to an account for which I have credentials stored only in Bitwarden), I am not able to login anymore.
Web app, chrome extension, mobile app, all of them return the same “An unexpected error has occured”.
I know for sure the Master password is correct because it was saved in the browser and on my phone as well and I know it very well.
I tried many different things and I’ve noticed some really weird behaviour. Hopefully someone would be able to catch something from what I will be saying and point me in the right direction.
- I tried creating a new account just so I know if things are actually working disregarding my initial account. Well, funny enough, the account creation went fine and I got the welcome email. However, trying to log in, I’m getting the same exact error as for my previous account. An unexpected error…
- I tried to login from a different browser. I normally use Brave but I switched to Edge and tried to login into my old (regular) account with the Master password. By the way, I don’t have 2FA or anything else, just using a master password. And again the same error. HOWEVER, I immediately got an email notifying me that someone logged in to my account from a different browser.
Well, I then went back to my Brave and tried an incognito mode. Same thing, it doesn’t login BUT I do get the notification email every single time. So, something is really messed up and can’t figure it out. - I then opened the Developer console from Brave and tried to login. In the console I get these errors:
POST h**ps://bitwarden.redacted.synology.me/identity/connect/token net::ERR_ABORTED 500 (Internal Server Error) api.service.ts:1599
r {response: null, statusCode: 500} console-log.service.ts:51
Received API exception: r {response: null, statusCode: 500} console-log.service.ts:51
Uncaught (in promise) Error: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received /#/login?email=
Then in the Network tab:
Name: token (shows in red on the left hand side)
General
Request URL:
h**ps://bitwarden.redacted.synology.me/identity/connect/token
Request Method:
POST
Status Code:
500 Internal Server Error
Remote Address:
85.72.232.53:443
Referrer Policy:
same-origin
Response Headers
content-length:
0
date:
Sat, 14 Dec 2024 20:16:28 GMT
server:
nginx
strict-transport-security:
max-age=15768000; includeSubdomains; preload
Request Headers
:authority:
bitwarden.redacted.synology.me
:method:
POST
:path:
/identity/connect/token
:scheme:
https
accept:
application/json
accept-encoding:
gzip, deflate, br, zstd
accept-language:
en,en-US;q=0.9,fr;q=0.8,ro;q=0.7
auth-email:
dm3tbHVjhmlaQHBtOm1l
bitwarden-client-name:
web
bitwarden-client-version:
2024.12.0
content-length:
232
content-type:
application/x-www-form-urlencoded; charset=utf-8
device-type:
9
origin:
hps://bitwarden.redacted.synology.me
priority:
u=1, i
referer:
hps://bitwarden.redacted.synology.me/
sec-ch-ua:
“Brave”;v=“131”, “Chromium”;v=“131”, “Not_A Brand”;v=“24”
sec-ch-ua-mobile:
?0
sec-ch-ua-platform:
“Windows”
sec-fetch-dest:
empty
sec-fetch-mode:
cors
sec-fetch-site:
same-origin
sec-gpc:
1
user-agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
- It looks like everything works except the actual login. Please let me know I can still somehow get my passwords back. I do have an old backup but in the meantime I added some new accounts and possibly changed some passwords.
- I also logged in the actual container for the MariaDB database and was able to log into the database with the master soql password but I have no idea if that helps in any way. Not sure if I could ever retrieve my passwords from that.
- I also tried using the bw CLI and the scary part is when trying to
bw loginI’m gettingUsername or password incorrect. Try againand I know for sure they are both right.
What is wrong? How can I fix it? I’m a developer and tried everything I know about both front-end and backend with databases and all but still cannot figure it out. Am I missing something? Is this something never seen?
Can anyone help? I’m happy to provide whatever other logs you may need.
By the way in the container logs for the MariaDB I get this error:
fail: Microsoft.AspNetCore.Server.Kestrel[13]
=> SpanId:82212d95c072e7d5, TraceId:c84ca67664ab19f464cdfc49578d60cc, ParentId:0000000000000000 => ConnectionId:0HN8SFDKQSL0U => RequestPath:/identity/connect/token RequestId:0HN8SFDKQSL0U:00000001
Connection id “0HN8SFDKQSL0U”, Request id “0HN8SFDKQSL0U:00000001”: An unhandled exception was thrown by the application.
MySqlConnector.MySqlException (0x80004005): Unknown column ‘g.Id’ in ‘field list’
at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in //src/MySqlConnector/Core/ServerSession.cs:line 894
at MySqlConnector.Core.ResultSet.ReadResultSetHeaderAsync(IOBehavior ioBehavior) in //src/MySqlConnector/Core/ResultSet.cs:line 37
at MySqlConnector.MySqlDataReader.ActivateResultSet(CancellationToken cancellationToken) in //src/MySqlConnector/MySqlDataReader.cs:line 130
at MySqlConnector.MySqlDataReader.InitAsync(CommandListPosition commandListPosition, ICommandPayloadCreator payloadCreator, IDictionary2 cachedProcedures, IMySqlCommand command, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlDataReader.cs:line 483 at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(CommandListPosition commandListPosition, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/CommandExecutor.cs:line 56 at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlCommand.cs:line 357 at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlCommand.cs:line 350 at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken)
at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func4 operation, Func4 verifySucceeded, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable1.AsyncEnumerator.MoveNextAsync() at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable1 asyncEnumerable, CancellationToken cancellationToken) at Bit.Infrastructure.EntityFramework.Auth.Repositories.GrantRepository.SaveAsync(IGrant obj) in /source/src/Infrastructure.EntityFramework/Auth/Repositories/GrantRepository.cs:line 80 at Bit.Identity.IdentityServer.PersistedGrantStore.StoreAsync(PersistedGrant pGrant) in /source/src/Identity/IdentityServer/PersistedGrantStore.cs:line 54 at Duende.IdentityServer.Stores.DefaultGrantStore1.StoreItemByHashedKeyAsync(String hashedKey, T item, String clientId, String subjectId, String sessionId, String description, DateTime created, Nullable1 expiration, Nullable1 consumedTime) in //src/IdentityServer/Stores/Default/DefaultGrantStore.cs:line 231
at Duende.IdentityServer.Stores.DefaultGrantStore1.CreateItemAsync(T item, String clientId, String subjectId, String sessionId, String description, DateTime created, Int32 lifetime) in /_/src/IdentityServer/Stores/Default/DefaultGrantStore.cs:line 177 at Duende.IdentityServer.Stores.DefaultRefreshTokenStore.StoreRefreshTokenAsync(RefreshToken refreshToken) in /_/src/IdentityServer/Stores/Default/DefaultRefreshTokenStore.cs:line 43 at Duende.IdentityServer.Services.DefaultRefreshTokenService.CreateRefreshTokenAsync(RefreshTokenCreationRequest request) in /_/src/IdentityServer/Services/Default/DefaultRefreshTokenService.cs:line 222 at Duende.IdentityServer.ResponseHandling.TokenResponseGenerator.CreateAccessTokenAsync(ValidatedTokenRequest request) in /_/src/IdentityServer/ResponseHandling/Default/TokenResponseGenerator.cs:line 453 at Duende.IdentityServer.ResponseHandling.TokenResponseGenerator.ProcessTokenRequestAsync(TokenRequestValidationResult validationResult) in /_/src/IdentityServer/ResponseHandling/Default/TokenResponseGenerator.cs:line 335 at Duende.IdentityServer.ResponseHandling.TokenResponseGenerator.ProcessAsync(TokenRequestValidationResult request) in /_/src/IdentityServer/ResponseHandling/Default/TokenResponseGenerator.cs:line 97 at Duende.IdentityServer.Endpoints.TokenEndpoint.ProcessTokenRequestAsync(HttpContext context) in /_/src/IdentityServer/Endpoints/TokenEndpoint.cs:line 133 at Duende.IdentityServer.Endpoints.TokenEndpoint.ProcessAsync(HttpContext context) in /_/src/IdentityServer/Endpoints/TokenEndpoint.cs:line 81 at Duende.IdentityServer.Hosting.IdentityServerMiddleware.Invoke(HttpContext context, IdentityServerOptions options, IEndpointRouter router, IUserSession userSession, IEventService events, IIssuerNameService issuerNameService, ISessionCoordinationService sessionCoordinationService) in /_/src/IdentityServer/Hosting/IdentityServerMiddleware.cs:line 106 at Duende.IdentityServer.Hosting.IdentityServerMiddleware.Invoke(HttpContext context, IdentityServerOptions options, IEndpointRouter router, IUserSession userSession, IEventService events, IIssuerNameService issuerNameService, ISessionCoordinationService sessionCoordinationService) in /_/src/IdentityServer/Hosting/IdentityServerMiddleware.cs:line 128 at Duende.IdentityServer.Hosting.MutualTlsEndpointMiddleware.Invoke(HttpContext context, IAuthenticationSchemeProvider schemes) in /_/src/IdentityServer/Hosting/MutualTlsEndpointMiddleware.cs:line 95 at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Duende.IdentityServer.Hosting.DynamicProviders.DynamicSchemeAuthenticationMiddleware.Invoke(HttpContext context) in /_/src/IdentityServer/Hosting/DynamicProviders/DynamicSchemes/DynamicSchemeAuthenticationMiddleware.cs:line 51 at Duende.IdentityServer.Hosting.BaseUrlMiddleware.Invoke(HttpContext context) in /_/src/IdentityServer/Hosting/BaseUrlMiddleware.cs:line 27 at Bit.Core.Utilities.CurrentContextMiddleware.Invoke(HttpContext httpContext, ICurrentContext currentContext, GlobalSettings globalSettings) in /source/src/Core/Utilities/CurrentContextMiddleware.cs:line 19 at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Builder.Extensions.UsePathBaseMiddleware.InvokeCore(HttpContext context, PathString matchedPath, PathString remainingPath) at Bit.Identity.Startup.<>c__DisplayClass10_1.<<Configure>b__2>d.MoveNext() in /source/src/Identity/Startup.cs:line 188 --- End of stack trace from previous location --- at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication1 application)
warn: Duende.IdentityServer.Endpoints.TokenEndpoint[0]
=> SpanId:3c06efd3a3992cb9, TraceId:33e1cb138b8e7fd24203e5b15717a332, ParentId:0000000000000000 => ConnectionId:0HN8SFDKQSL0V => RequestPath:/identity/connect/token RequestId:0HN8SFDKQSL0V:00000001 => IpAddress:172.18.0.1 UserAgent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 DeviceType: Origin: ClientVersion:
Invalid HTTP request for token endpoint
