Rate limit: 100 at once, 10 a second; the page's own files uncounted #13
@@ -4,6 +4,10 @@ One line per change, grouped by the release that shipped it. Newest
|
||||
first. A release is a Cargo.toml bump plus a `v<version>` tag; the
|
||||
content repos pin the tag they run (see the end of this file).
|
||||
|
||||
## Unreleased
|
||||
|
||||
- The rate limit is a hundred at once, refilled at ten a second (it was 30, one every two seconds, which a page with a picture strip used up in one load: a few reloads and the stylesheet was refused). The files a page is made of, from portal's own disk, are not counted at all. `PORTAL_RATE_PER_SECOND` replaces `PORTAL_RATE_SECONDS`; no site set it.
|
||||
|
||||
## 0.6.3 (2026-09-30)
|
||||
|
||||
- Pages move the way their addresses do. Deeper in the address comes in from the right, back up comes in from the left, and a page at the same depth comes from below or above, with the index on top. The cards on screen leave as layers of their own, one after another, and the next page's cards arrive the same way. The old page stays on screen until the new one has loaded, where it used to go blank in between. Only transforms and opacity are animated, through the browser's View Transitions (`public/page-motion.js`, before the router). Browsers without them, and anyone who asks for reduced motion, navigate as before.
|
||||
|
||||
+16
-7
@@ -82,22 +82,31 @@ values.
|
||||
| `GARAGE_*` | uploads |
|
||||
| `MAPBOX_TOKEN` | map tiles, proxied; unset, maps are off |
|
||||
| `GARAGE_*` (4) | attachments; see below. Unset, upload fields fail closed |
|
||||
| `PORTAL_RATE_BURST`, `PORTAL_RATE_SECONDS` | the rate limit; see below |
|
||||
| `PORTAL_RATE_BURST`, `PORTAL_RATE_PER_SECOND` | the rate limit; see below |
|
||||
|
||||
Mail is delivered by [gdo](https://project.uhhm.no/uhhm/gdo), a
|
||||
separate unit on the same host reading the same NATS.
|
||||
|
||||
## Rate limiting
|
||||
|
||||
Every request passes a per-caller-IP limit before a session is loaded
|
||||
or a handler is entered: a burst of 30, refilled one every two
|
||||
seconds. A visitor filling in a form never reaches it (a page load
|
||||
fetches several resources at once); a script does.
|
||||
Every request that makes portal do something passes a per-caller-IP
|
||||
limit before a session is loaded or a handler is entered: a burst of
|
||||
100, refilled at ten a second. More than a hundred requests in ten
|
||||
seconds is a script; someone reloading a page again and again stays
|
||||
well inside it.
|
||||
|
||||
The files a page is made of, served from portal's own disk, are not
|
||||
counted: the stylesheet and scripts under `/pkg`, the fonts, the
|
||||
favicons, the vendored libraries. Refusing one of those is what turns
|
||||
a reloaded page into an unstyled mess, and serving one costs next to
|
||||
nothing. Anything that makes portal ask someone else is counted: a
|
||||
content picture under `/site/` is fetched from Gitea, a map tile from
|
||||
the map provider.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `PORTAL_RATE_BURST` | how many requests at once, default 30 |
|
||||
| `PORTAL_RATE_SECONDS` | seconds per refilled request, default 2 |
|
||||
| `PORTAL_RATE_BURST` | how many requests at once, default 100 |
|
||||
| `PORTAL_RATE_PER_SECOND` | requests refilled per second, default 10 |
|
||||
|
||||
Either set to `0` turns it off, for an instance behind something that
|
||||
already does this. The caller is read from `X-Forwarded-For` or
|
||||
|
||||
+54
-44
@@ -166,30 +166,32 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.with_name("portal_session");
|
||||
|
||||
// What a person does costs a little; what a script does costs a
|
||||
// lot. A burst of 30 with one refilled every two seconds is well
|
||||
// above anything a visitor filling in a form will reach - a page
|
||||
// load fetches several resources at once - and well below what it
|
||||
// takes to be a nuisance. Per caller IP, in memory, no store: this
|
||||
// is a fence, not an accounting system, and the fence that matters
|
||||
// for uploads is the policy check on the route itself.
|
||||
// lot. More than a hundred requests in ten seconds is a script: a
|
||||
// burst of 100, refilled at ten a second. Someone reloading a page
|
||||
// again and again stays well inside it, the more so since the
|
||||
// files a page is made of (stylesheet, scripts, fonts, pictures)
|
||||
// are not counted at all - see `assets` below. Per caller IP, in
|
||||
// memory, no store: this is a fence, not an accounting system, and
|
||||
// the fence that matters for uploads is the policy check on the
|
||||
// route itself.
|
||||
//
|
||||
// PORTAL_RATE_PER_SECOND / PORTAL_RATE_BURST retune it per site;
|
||||
// 0 for either turns it off, for an instance behind something that
|
||||
// already does this.
|
||||
let rate_seconds: u64 = std::env::var("PORTAL_RATE_SECONDS")
|
||||
let rate_per_second: u64 = std::env::var("PORTAL_RATE_PER_SECOND")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(2);
|
||||
.unwrap_or(10);
|
||||
let rate_burst: u32 = std::env::var("PORTAL_RATE_BURST")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(30);
|
||||
let governor = (rate_seconds > 0 && rate_burst > 0).then(|| {
|
||||
.unwrap_or(100);
|
||||
let governor = (rate_per_second > 0 && rate_burst > 0).then(|| {
|
||||
// SmartIpKeyExtractor reads X-Forwarded-For/X-Real-IP, which is
|
||||
// what every one of these instances is behind; it falls back to
|
||||
// the socket address when neither is set.
|
||||
let conf = tower_governor::governor::GovernorConfigBuilder::default()
|
||||
.per_second(rate_seconds)
|
||||
.per_millisecond((1000 / rate_per_second).max(1))
|
||||
.burst_size(rate_burst)
|
||||
.key_extractor(tower_governor::key_extractor::SmartIpKeyExtractor)
|
||||
.finish()
|
||||
@@ -197,8 +199,8 @@ async fn serve() -> anyhow::Result<()> {
|
||||
std::sync::Arc::new(conf)
|
||||
});
|
||||
match &governor {
|
||||
Some(_) => tracing::info!(rate_seconds, rate_burst, "rate limiting per caller IP"),
|
||||
None => tracing::warn!("rate limiting is off (PORTAL_RATE_SECONDS or PORTAL_RATE_BURST is 0)"),
|
||||
Some(_) => tracing::info!(rate_per_second, rate_burst, "rate limiting per caller IP"),
|
||||
None => tracing::warn!("rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"),
|
||||
}
|
||||
|
||||
async fn server_fn_handler(
|
||||
@@ -233,34 +235,13 @@ async fn serve() -> anyhow::Result<()> {
|
||||
let fonts_dir = format!("{}/fonts", leptos_options.site_root);
|
||||
let page_motion = format!("{}/page-motion.js", leptos_options.site_root);
|
||||
|
||||
let app = Router::new()
|
||||
.route("/auth/login", get(oidc::login))
|
||||
.route("/auth/callback", get(oidc::callback))
|
||||
.route("/auth/logout", get(oidc::logout))
|
||||
.route("/api/{*fn_name}", any(server_fn_handler))
|
||||
// axum caps request bodies at 2 MB by default, which had made the
|
||||
// 20 MB upload limit unreachable; raise the fence on this route only,
|
||||
// with room for the multipart framing.
|
||||
.route(
|
||||
"/upload",
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)),
|
||||
)
|
||||
.route(
|
||||
"/attachment/{bucket}/{record}/{field}",
|
||||
get(upload::attachment),
|
||||
)
|
||||
.route("/gitea-repo", get(content::gitea_repo_handler))
|
||||
// What the storylines did, as counts (report.rs).
|
||||
.route("/report", get(portal::report::handler))
|
||||
// Content-shipped assets (hero module etc.), same-origin.
|
||||
.route("/site/{*path}", get(content::site_asset_handler))
|
||||
.route("/automation/kv/{bucket}", get(content::automation_kv_handler))
|
||||
// Maps, proxied: the browser asks this origin for every tile
|
||||
// and every still, so nothing third-party loads in a page and
|
||||
// the token stays here (see maps.rs).
|
||||
.route("/map/style.json", get(portal::maps::style))
|
||||
.route("/map/tiles/{z}/{x}/{y}", get(portal::maps::tile))
|
||||
.route("/map/still/{place}/{size}", get(portal::maps::still))
|
||||
// The files a page is made of, from this disk: the stylesheet, the
|
||||
// scripts, the fonts. Not counted by the rate limit - they cost
|
||||
// next to nothing, and refusing one is what turns a reloaded page
|
||||
// into an unstyled mess - and no session is loaded for them.
|
||||
// Anything that makes portal ask someone else (content pictures
|
||||
// from Gitea, map tiles) stays below, counted.
|
||||
let assets: Router<AppState> = Router::new()
|
||||
// no-cache = "revalidate before reuse", not "don't cache":
|
||||
// pkg files keep the same names across releases (portal.js,
|
||||
// portal.wasm), and without this browsers heuristically cache
|
||||
@@ -296,7 +277,36 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.service(ServeFile::new(page_motion)),
|
||||
)
|
||||
.route_service("/maplibre-gl.js", ServeFile::new(maplibre_js))
|
||||
.route_service("/maplibre-gl.css", ServeFile::new(maplibre_css))
|
||||
.route_service("/maplibre-gl.css", ServeFile::new(maplibre_css));
|
||||
|
||||
let app = Router::new()
|
||||
.route("/auth/login", get(oidc::login))
|
||||
.route("/auth/callback", get(oidc::callback))
|
||||
.route("/auth/logout", get(oidc::logout))
|
||||
.route("/api/{*fn_name}", any(server_fn_handler))
|
||||
// axum caps request bodies at 2 MB by default, which had made the
|
||||
// 20 MB upload limit unreachable; raise the fence on this route only,
|
||||
// with room for the multipart framing.
|
||||
.route(
|
||||
"/upload",
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)),
|
||||
)
|
||||
.route(
|
||||
"/attachment/{bucket}/{record}/{field}",
|
||||
get(upload::attachment),
|
||||
)
|
||||
.route("/gitea-repo", get(content::gitea_repo_handler))
|
||||
// What the storylines did, as counts (report.rs).
|
||||
.route("/report", get(portal::report::handler))
|
||||
// Content-shipped assets (hero module etc.), same-origin.
|
||||
.route("/site/{*path}", get(content::site_asset_handler))
|
||||
.route("/automation/kv/{bucket}", get(content::automation_kv_handler))
|
||||
// Maps, proxied: the browser asks this origin for every tile
|
||||
// and every still, so nothing third-party loads in a page and
|
||||
// the token stays here (see maps.rs).
|
||||
.route("/map/style.json", get(portal::maps::style))
|
||||
.route("/map/tiles/{z}/{x}/{y}", get(portal::maps::tile))
|
||||
.route("/map/still/{place}/{size}", get(portal::maps::still))
|
||||
.leptos_routes_with_context(
|
||||
&state,
|
||||
routes,
|
||||
@@ -318,8 +328,8 @@ async fn serve() -> anyhow::Result<()> {
|
||||
let app = match governor {
|
||||
Some(conf) => app.layer(tower_governor::GovernorLayer::new(conf)),
|
||||
None => app,
|
||||
}
|
||||
.with_state(state);
|
||||
};
|
||||
let app = assets.merge(app).with_state(state);
|
||||
|
||||
tracing::info!("listening on http://{addr}");
|
||||
let listener = tokio::net::TcpListener::bind(&addr).await?;
|
||||
|
||||
Reference in New Issue
Block a user