Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add suggestions cleanup job #146

Merged
merged 6 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 13 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,38 +280,30 @@ There are two scheduled jobs:

# Scheduled jobs

Scheduled jobs powered by Coravel are included in the package, allowing jobs to run at set intervals.

**Important**
Use this only if the package is not intended for an Optimizely site. Optimizely has built-in scheduled jobs mechanism.

To enable scheduled jobs, you need to configure the following settings:
Scheduled job - process that runs in background
- Suggestions cleanup job - shipped with the package, contains process that cleans up suggestions table.
This job is configured by default to remove records older than 14 days. You can adjust the retention period or timeout as needed.
```
services.AddNotFoundHandler(o =>
{
...
o.ScheduledJobs = true;
o.SuggestionsCleanupOptions.DaysToKeep = 30;
o.SuggestionsCleanupOptions.Timeout = 30 * 60;
});
```

## Suggestions cleanup job
Practice shows that the suggestions table grows quickly in production, so a suggestions cleanup job was added to control its growth.

This job is configured by default to run daily at midnight, removing records older than 14 days.
You can adjust the retention period as needed.

Scheduler - mechanism that triggers scheduled jobs in a recurrent manner
- InternalScheduler - default scheduler, included in the core package, a scheduler that uses [Coravel](https://docs.coravel.net/).
To enable the scheduler, you need to enable UseInternalScheduler flag. Additionally, you can adjust the scheduler run interval:
```
services.AddNotFoundHandler(o =>
{
o.ScheduledJobs = true;
o.SuggestionsCleanupOptions.CronInterval = "0 0 * * 0" // weekly on Sunday midnight
o.SuggestionsCleanupOptions.DaysToKeep = 30;
...
o.UseInternalScheduler = true;
o.InternalSchedulerCronInterval = "0 0 * * *" // by default it's configured to run daily at midnight
});
```
**Note**
For Optimizely was added job that is powered by built-in scheduled jobs mechanism.

[Geta NotFoundHandler] Suggestions cleanup job
- OptimizelyScheduler - uses Optimizely to schedule job runs.
An Optimizely scheduled job was added - <code>[Geta NotFoundHandler] Suggestions cleanup job</code>.

# Troubleshooting

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ public SuggestionsCleanupJob(ISuggestionsCleanupService suggestionsCleanupServic

public override string Execute()
{
return _suggestionsCleanupService.Cleanup() ? "": "Unable to cleanup suggestions; please refer to the logs.";
_suggestionsCleanupService.Cleanup();

return string.Empty;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
using Geta.NotFoundHandler.Infrastructure.Configuration;
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace Geta.NotFoundHandler.Core.ScheduledJobs;
Expand All @@ -17,14 +18,19 @@ public static IApplicationBuilder UseScheduler(this IApplicationBuilder app)
var services = app.ApplicationServices;

var options = services.GetRequiredService<IOptions<NotFoundHandlerOptions>>().Value;

var logger = services.GetRequiredService<ILogger>();

services.UseScheduler(scheduler =>
jevgenijsp marked this conversation as resolved.
Show resolved Hide resolved
{
scheduler
.Schedule<SuggestionsCleanupJob>()
.Cron(options.SuggestionsCleanupOptions.CronInterval)
.PreventOverlapping(nameof(SuggestionsCleanupJob));
});
{
scheduler
.Schedule<SuggestionsCleanupJob>()
.Cron(options.InternalSchedulerCronInterval)
.PreventOverlapping(nameof(SuggestionsCleanupJob));
})
.OnError(x =>
{
logger.LogError(x, "Something went wrong, scheduled job fails with exception");
jevgenijsp marked this conversation as resolved.
Show resolved Hide resolved
});

return app;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ public static IServiceCollection EnableScheduler(this IServiceCollection service
using var serviceProvider = services.BuildServiceProvider();
var options = serviceProvider.GetRequiredService<IOptions<NotFoundHandlerOptions>>().Value;

if (options.UseScheduler)
if (options.UseInternalScheduler)
{
services.AddScheduler();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ namespace Geta.NotFoundHandler.Core.Suggestions;

public interface ISuggestionsCleanupService
{
bool Cleanup();
void Cleanup();
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,4 @@ public class SuggestionsCleanupOptions
{
public int DaysToKeep { get; set; } = 14;
public int Timeout { get; set; } = 30 * 60;
public string CronInterval { get; set; } = "0 0 * * *";
}
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ IF OBJECT_ID('[NotFoundHandler.Suggestions]', 'U') IS NOT NULL
";


public bool Cleanup()
public void Cleanup()
{
try
{
Expand All @@ -49,14 +49,12 @@ public bool Cleanup()
command.CommandTimeout = _options.Value.SuggestionsCleanupOptions.Timeout;
command.Connection.Open();
command.ExecuteNonQuery();

return true;
}
catch (Exception ex)
{
_logger.LogError(ex, "There was a problem while performing cleanup on connection");

return false;
throw;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ public class NotFoundHandlerOptions
public int BufferSize { get; set; } = 30;
public int ThreshHold { get; set; } = 5;
public SuggestionsCleanupOptions SuggestionsCleanupOptions { get; set; } = new();
public bool UseScheduler { get; set; }
public bool UseInternalScheduler { get; set; }
public string InternalSchedulerCronInterval { get; set; } = "0 0 * * *";
public FileNotFoundMode HandlerMode { get; set; } = FileNotFoundMode.On;
public TimeSpan RegexTimeout { get; set; } = TimeSpan.FromMilliseconds(100);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ public static IApplicationBuilder UseNotFoundHandler(this IApplicationBuilder ap

var options = services.GetRequiredService<IOptions<NotFoundHandlerOptions>>().Value;

if (options.UseScheduler)
if (options.UseInternalScheduler)
{
app.UseScheduler();
}
Expand Down
Loading