http/response: precompute compressed variants for immutable static assets#4378
Open
jvoisin wants to merge 1 commit into
Open
http/response: precompute compressed variants for immutable static assets#4378jvoisin wants to merge 1 commit into
jvoisin wants to merge 1 commit into
Conversation
As this is quite an invasive change, I'm taking the time to write an elaborate commit message, with sections, wow! Background ---------- Every Miniflux page load fetches CSS, JavaScript and SVG assets. These are content-hashed and served with `Cache-Control: public, immutable` for 48-72 hours, and their bytes never change for the lifetime of a running process. Despite that, the shared response builder re-ran Brotli/Gzip compression from scratch on *every* request, at the default compression level. This was repeated CPU work on the hot request path for content that is fundamentally static, and it forced a lower compression level to keep per-request latency acceptable. This commit moves that work to startup time so each immutable resource is compressed exactly once, at maximum level. Asset generation (internal/ui/static) -------------------------------------- The `asset` struct gains a `Brotli` and a `Gzip` field holding the precomputed variants. They are nil for assets not worth compressing. A new `precompressAsset()` helper compresses a bundle with Brotli and Gzip at their *best* compression level, producing smaller payloads for free since the cost is paid once at startup rather than per request. JavaScript bundles ------------------ Two things previously prevented the JS bundles from having stable, precompressible bytes; both are now solved: 1. The LibreJS license markers (`//@license ... //@license-end`) were appended around the bundle on every request. They are now wrapped once during generation. 2. The service worker had its `OFFLINE_URL` constant injected on every request. Since the URL is just `BasePath() + "/offline"`, a config value fixed at startup, `GenerateJavascriptBundles` takes the offline URL and prepends the constant to the service-worker source before minifying. As a result the service worker is precompressed like every other bundle and no longer needs any special-casing in the handler. Response builder (internal/http/response) ----------------------------------------- The new `Builder.WithCompressedVariants(brotli, gzip)` hands the builder precomputed encodings. When the client's `Accept-Encoding` matches and a variant is present, the builder writes those bytes directly; otherwise it falls back to compressing on the fly. A nil variant simply means "none available for this encoding". Note that on-the-fly compression is deliberately kept, as it still serves all the dynamic responses (rendered HTML pages, JSON API output, OPML, etc.) as well as the `deflate` encoding, which is not precomputed. The reasons for not precomputing `deflate` are that nobody uses it in 2026, `gzip` is `deflate` with a header/footer and a CRC, and the fallback is super-cheap anyway. The compression size threshold, previously duplicated as an unexported constant in both packages, is now a single exported `response.CompressionThreshold` referenced by both the builder's serving gate and the asset precompression helper, as I'm not a fan of having the same information duplicated in the codebase. Handlers -------- The stylesheet, app-icon and JavaScript handlers pass the precomputed variants through to the builder. The JavaScript handler collapses to a single uniform path now that the service worker no longer needs runtime injection. All in all, the per-request compression is exchanged for an immutable static one, improving a tad the responsiveness of miniflux as fixed assets don't have to be compressed on the fly, and are also slightly smaller since we're now using the maximum compression level.
gudvinr
reviewed
Jun 7, 2026
| // precomputed bytes are served directly. A nil slice simply means no variant | ||
| // is available for that encoding, in which case the builder handles it as | ||
| // usual (compressing on the fly, or sending it uncompressed for small bodies). | ||
| func (b *Builder) WithCompressedVariants(brotliBody, gzipBody []byte) *Builder { |
Contributor
There was a problem hiding this comment.
One request would need only one variant.
It'd be much easier to resolve accept-encoding outside. Then you just need to disable compression and set raw body value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As this is quite an invasive change, I'm taking the time to write an elaborate commit message, with sections, wow!
Background
Every Miniflux page load fetches CSS, JavaScript and SVG assets. These are content-hashed and served with
Cache-Control: public, immutablefor 48-72 hours, and their bytes never change for the lifetime of a running process.Despite that, the shared response builder re-ran Brotli/Gzip compression from scratch on every request, at the default compression level. This was repeated CPU work on the hot request path for content that is fundamentally static, and it forced a lower compression level to keep per-request latency acceptable.
This commit moves that work to startup time so each immutable resource is compressed exactly once, at maximum level.
Asset generation (internal/ui/static)
The
assetstruct gains aBrotliand aGzipfield holding the precomputed variants. They are nil for assets not worth compressing. A newprecompressAsset()helper compresses a bundle with Brotli and Gzip at their best compression level, producing smaller payloads for free since the cost is paid once at startup rather than per request.JavaScript bundles
Two things previously prevented the JS bundles from having stable, precompressible bytes; both are now solved:
//@license ... //@license-end) were appended around the bundle on every request. They are now wrapped once during generation.OFFLINE_URLconstant injected on every request. Since the URL is justBasePath() + "/offline", a config value fixed at startup,GenerateJavascriptBundlestakes the offline URL and prepends the constant to the service-worker source before minifying. As a result the service worker is precompressed like every other bundle and no longer needs any special-casing in the handler.Response builder (internal/http/response)
The new
Builder.WithCompressedVariants(brotli, gzip)hands the builder precomputed encodings. When the client'sAccept-Encodingmatches and a variant is present, the builder writes those bytes directly; otherwise it falls back to compressing on the fly. A nil variant simply means "none available for this encoding".Note that on-the-fly compression is deliberately kept, as it still serves all the dynamic responses (rendered HTML pages, JSON API output, OPML, etc.) as well as the
deflateencoding, which is not precomputed. The reasons for not precomputingdeflateare that nobody uses it in 2026,gzipisdeflatewith a header/footer and a CRC, and the fallback is super-cheap anyway.The compression size threshold, previously duplicated as an unexported constant in both packages, is now a single exported
response.CompressionThresholdreferenced by both the builder's serving gate and the asset precompression helper, as I'm not a fan of having the same information duplicated in the codebase.Handlers
The stylesheet, app-icon and JavaScript handlers pass the precomputed variants through to the builder. The JavaScript handler collapses to a single uniform path now that the service worker no longer needs runtime injection.
All in all, the per-request compression is exchanged for an immutable static one, improving a tad the responsiveness of miniflux as fixed assets don't have to be compressed on the fly, and are also slightly smaller since we're now using the maximum compression level.
Have you followed these guidelines?