feat(feed): allow per-feed refresh interval override#4293
Open
ChrisJr404 wants to merge 1 commit into
Open
Conversation
Closes miniflux#412. Adds an optional refresh_interval_minutes column on feeds. When set, it overrides the global polling frequency for that feed only. NULL (the default) keeps the existing behaviour, so existing installs are unaffected by the migration. The override is honoured by Feed.ScheduleNextCheck. The HTTP refresh delay (RSS TTL, Retry-After, Cache-Control, Expires) is still applied as a floor so a small override cannot be used to ignore a server's explicit back-off. Exposed in three places: - The Edit Feed page gets a "Refresh interval (minutes)" input. Leaving it at zero clears the override and falls back to the global scheduler. - The REST API Feed, FeedCreationRequest and FeedModificationRequest payloads gain a refresh_interval_minutes field. Sending zero on a modification clears the override; omitting the field leaves it unchanged. - The Go client (client/model.go) exposes the same field so existing third-party integrations can read and write it. Validation enforces 5 <= value <= 10080 minutes (one week, matching the round-robin cap). Below the floor would let users hammer publishers harder than the configured global minimum. Tested: - New unit tests in internal/model and internal/validator cover the override path, the refresh-delay floor, the global-cap behaviour, the nil/zero fallback, and the Patch semantics for set/clear/leave. - Added an API integration test (TestUpdateFeedRefreshInterval) that exercises the full create -> read -> set -> clear -> reject path through the HTTP API. - go build ./... and go test ./... pass; go vet clean.
Collaborator
|
A dumb issue with allowing users to set their own refresh rates is that this allows someone to set a super-short one, getting the IP address of the miniflux instance blacklisted/429 for the other users :/ |
Author
|
Fair concern. The PR has two protections that should cover the abuse case:
So a hostile user setting the shortest legal value (5 min) on a feed whose origin returns Happy to also add a per-instance setting if you would prefer the floor to be operator-configurable rather than hardcoded to 5. Let me know. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #412.
Description
Adds an optional
refresh_interval_minutescolumn on feeds. When set, it overrides the global polling frequency for that feed only. NULL (the default) keeps the existing behaviour, so the migration does not disturb existing installations.The override is honoured by
Feed.ScheduleNextCheck. The HTTP refresh delay (RSS TTL,Retry-After,Cache-Control,Expires) is still applied as a floor, so a small override cannot be used to ignore a server's explicit back-off.Motivation
Issue #412 (44+ thumbs-up, multi-year ask) reports that some users want to refresh certain feeds more often than the global
POLLING_FREQUENCY(e.g. fast news, status pages) while keeping a longer interval for others (e.g. arXiv, blogs that 429 if you fetch them too aggressively). Today everything moves at one global rate, which leads to either missed updates or rate-limit bans depending on which way the user tunes the global value.The earlier attempt in #949 was closed in 2021, partly because at that point per-user polling was tangled with multi-user semantics. Each
feedsrow already belongs to a singleuser_id, so a per-row override is well-defined and does not require touching the multi-user feed-fetching design at all.Changes
refresh_interval_minutesinteger column onfeeds, NULL by default, with aCHECKconstraint preventing values < 1. Added as a fresh migration step at the end ofinternal/database/migrations.go.Feed.ScheduleNextCheckhonours the override when set. When NULL or zero, the existing global scheduler logic runs unchanged.model.Feed,FeedCreationRequestandFeedModificationRequestgain arefresh_interval_minutesfield. On modification, sending0clears the override; omitting the field leaves it unchanged. Mirrored inclient/model.goso existing Go consumers work too.Refresh interval (minutes)number input with help text.0(or empty) means "use the global default".MinFeedRefreshIntervalMinutes(5) andMaxFeedRefreshIntervalMinutes(10080, one week). The floor prevents a user from accidentally pointing a tighter interval at a publisher than the configured global minimum.error.feed_invalid_refresh_interval,form.feed.label.refresh_interval_minutesandform.feed.help.refresh_interval_minuteskeys added to all 22 translation files (English text used as a placeholder for the non-English files; happy to drop those if maintainers prefer to add them in a follow-up).Testing
go build ./...,go vet ./...andgo test ./...all clean.internal/model/feed_test.go:TestFeedScheduleNextCheckRefreshIntervalOverrideexercises the basic override.TestFeedScheduleNextCheckRefreshIntervalOverrideRespectsRefreshDelayconfirms that aRetry-Afterlonger than the override still wins.TestFeedScheduleNextCheckRefreshIntervalOverrideIgnoresGlobalCapconfirms the override is not clamped by the global round-robin maximum.TestFeedScheduleNextCheckRefreshIntervalNilFallsBackToGlobaland...ZeroFallsBackToGlobalconfirm the existing scheduler runs unchanged when the override is unset.TestFeedModificationRequestPatch*cases cover set / clear (zero) / leave-alone (nil) semantics.internal/validator/feed_test.gocovers the bounds check.internal/api/api_integration_test.goaddsTestUpdateFeedRefreshInterval, which drives create -> read (must benil) -> set 120 -> read back -> clear with0-> attempt to set1(must be rejected) through the real HTTP API.Breaking changes
None. The new column is NULL by default; the API field is
*intso existing clients that don't send it continue to work unchanged.Related issues