Failed jobs do not become expired to allow you to re-queue them without any
time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete)
attribute to delete them automatically.
The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure.
Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure. ---> Microsoft.Data.SqlClient.SqlException: Cannot open database "TestFridaMalmo" requested by the login. The login failed.
Login failed for user 'TestFridaMalmoApp'.
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObjectstateObj, SqlCommandcommand, BooleancallerHasConnectionLock, BooleanasyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehaviorrunBehavior, SqlCommandcmdHandler, SqlDataReaderdataStream, BulkCopySimpleResultSetbulkCopyHandler, TdsParserStateObjectstateObj, Boolean&dataReady)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(BooleanenlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfoserverInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringconnectionOptions, SqlCredentialcredential, TimeoutTimertimeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimertimeout, SqlConnectionStringconnectionOptions, SqlCredentialcredential, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentityidentity, SqlConnectionStringconnectionOptions, SqlCredentialcredential, ObjectproviderInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringuserConnectionOptions, SessionDatareconnectSessionData, BooleanapplyTransientFaultHandling, StringaccessToken, IDbConnectionPoolpool, Func`3accessTokenCallback)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.CreateObject(DbConnectionowningObject, DbConnectionOptionsuserOptions, DbConnectionInternaloldConnection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.TryGetConnection(DbConnectionowningObject, UInt32waitForMultipleObjectsTimeout, BooleanallowCreate, BooleanonlyOneCheckConnection, DbConnectionOptionsuserOptions, DbConnectionInternal&connection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationTokencancellationToken, BooleanerrorsExpected)
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObjectparameterObject, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumeratorenumerator, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass30_0`2.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TStatestate, Func`4operation, Func`4verifySucceeded, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.AspNetCore.Identity.UserManager`1.FindByNameAsync(StringuserName)
at NP.FRIDA.SystemBLL.ApplicationUserManager.FindByNameAsync(StringuserName) in /mnt/azp/_work/2/s/NP.FRIDA.SystemBLL/Managers/ApplicationUserManager.cs:line 43
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJobAsTenant(FridaTenantInfotenantInfo) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 76
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJob(StringtenantIdentifier) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 45
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.RuntimeMethodInfo.Invoke(Objectobj, BindingFlagsinvokeAttr, Binderbinder, Object[]parameters, CultureInfoculture)
An exception occurred during performance of the job. More details...
06/08/2026 21:02:10
Hangfire.Storage.DistributedLockTimeoutException
Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
Hangfire.Storage.DistributedLockTimeoutException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
at Hangfire.Redis.StackExchange.RedisLock.Acquire(IDatabaseredis, RedisKeykey, TimeSpantimeOut, TimeSpanholdDuration)
at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContextcontext) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 48
at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198
An exception occurred during performance of the job. More details...
06/08/2026 21:01:10
Hangfire.Storage.DistributedLockTimeoutException
Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
Hangfire.Storage.DistributedLockTimeoutException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
at Hangfire.Redis.StackExchange.RedisLock.Acquire(IDatabaseredis, RedisKeykey, TimeSpantimeOut, TimeSpanholdDuration)
at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContextcontext) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 48
at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198
The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure.
Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure. ---> Microsoft.Data.SqlClient.SqlException: Cannot open database "TestFridaMalmo" requested by the login. The login failed.
Login failed for user 'TestFridaMalmoApp'.
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObjectstateObj, SqlCommandcommand, BooleancallerHasConnectionLock, BooleanasyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehaviorrunBehavior, SqlCommandcmdHandler, SqlDataReaderdataStream, BulkCopySimpleResultSetbulkCopyHandler, TdsParserStateObjectstateObj, Boolean&dataReady)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(BooleanenlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfoserverInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringconnectionOptions, SqlCredentialcredential, TimeoutTimertimeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimertimeout, SqlConnectionStringconnectionOptions, SqlCredentialcredential, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentityidentity, SqlConnectionStringconnectionOptions, SqlCredentialcredential, ObjectproviderInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringuserConnectionOptions, SessionDatareconnectSessionData, BooleanapplyTransientFaultHandling, StringaccessToken, IDbConnectionPoolpool, Func`3accessTokenCallback)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.CreateObject(DbConnectionowningObject, DbConnectionOptionsuserOptions, DbConnectionInternaloldConnection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.TryGetConnection(DbConnectionowningObject, UInt32waitForMultipleObjectsTimeout, BooleanallowCreate, BooleanonlyOneCheckConnection, DbConnectionOptionsuserOptions, DbConnectionInternal&connection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationTokencancellationToken, BooleanerrorsExpected)
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObjectparameterObject, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumeratorenumerator, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass30_0`2.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TStatestate, Func`4operation, Func`4verifySucceeded, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.AspNetCore.Identity.UserManager`1.FindByNameAsync(StringuserName)
at NP.FRIDA.SystemBLL.ApplicationUserManager.FindByNameAsync(StringuserName) in /mnt/azp/_work/2/s/NP.FRIDA.SystemBLL/Managers/ApplicationUserManager.cs:line 43
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJobAsTenant(FridaTenantInfotenantInfo) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 76
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJob(StringtenantIdentifier) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 45
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.RuntimeMethodInfo.Invoke(Objectobj, BindingFlagsinvokeAttr, Binderbinder, Object[]parameters, CultureInfoculture)
An exception occurred during performance of the job. More details...
06/08/2026 21:00:10
Hangfire.Storage.DistributedLockTimeoutException
Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
Hangfire.Storage.DistributedLockTimeoutException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'Failed to acquire lock on {bgjobs:hangfire:test-malmo}:SendEmailJobMultiTenant.ExecuteJob within given timeout (00:05:00)' resource.
at Hangfire.Redis.StackExchange.RedisLock.Acquire(IDatabaseredis, RedisKeykey, TimeSpantimeOut, TimeSpanholdDuration)
at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContextcontext) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 48
at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198
The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure.
Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: The maximum number of retries (6) was exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See the inner exception for the most recent failure. ---> Microsoft.Data.SqlClient.SqlException: Cannot open database "TestFridaMalmo" requested by the login. The login failed.
Login failed for user 'TestFridaMalmoApp'.
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObjectstateObj, SqlCommandcommand, BooleancallerHasConnectionLock, BooleanasyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehaviorrunBehavior, SqlCommandcmdHandler, SqlDataReaderdataStream, BulkCopySimpleResultSetbulkCopyHandler, TdsParserStateObjectstateObj, Boolean&dataReady)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(BooleanenlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfoserverInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringconnectionOptions, SqlCredentialcredential, TimeoutTimertimeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimertimeout, SqlConnectionStringconnectionOptions, SqlCredentialcredential, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentityidentity, SqlConnectionStringconnectionOptions, SqlCredentialcredential, ObjectproviderInfo, StringnewPassword, SecureStringnewSecurePassword, BooleanredirectedUserInstance, SqlConnectionStringuserConnectionOptions, SessionDatareconnectSessionData, BooleanapplyTransientFaultHandling, StringaccessToken, IDbConnectionPoolpool, Func`3accessTokenCallback)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.CreateObject(DbConnectionowningObject, DbConnectionOptionsuserOptions, DbConnectionInternaloldConnection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.TryGetConnection(DbConnectionowningObject, UInt32waitForMultipleObjectsTimeout, BooleanallowCreate, BooleanonlyOneCheckConnection, DbConnectionOptionsuserOptions, DbConnectionInternal&connection)
at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(BooleanerrorsExpected, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationTokencancellationToken, BooleanerrorsExpected)
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObjectparameterObject, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumeratorenumerator, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass30_0`2.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4operation, Func`4verifySucceeded, TStatestate, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TStatestate, Func`4operation, Func`4verifySucceeded, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1asyncEnumerable, CancellationTokencancellationToken)
at Microsoft.AspNetCore.Identity.UserManager`1.FindByNameAsync(StringuserName)
at NP.FRIDA.SystemBLL.ApplicationUserManager.FindByNameAsync(StringuserName) in /mnt/azp/_work/2/s/NP.FRIDA.SystemBLL/Managers/ApplicationUserManager.cs:line 43
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJobAsTenant(FridaTenantInfotenantInfo) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 76
at NP.FRIDA.BackgroundJob.MultiTenant.MultiTenantBackgroundJob`1.ExecuteJob(StringtenantIdentifier) in /mnt/azp/_work/2/s/NP.FRIDA.BackgroundJob/MultiTenant/MultiTenantBackgroundJob.cs:line 45
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.RuntimeMethodInfo.Invoke(Objectobj, BindingFlagsinvokeAttr, Binderbinder, Object[]parameters, CultureInfoculture)