Configuration: every administrator's setting settable by an environment variable, with one registry, naming rules and generated documentation #1186

New issue

Open

opened 2026-10-31 21:29:01 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-31 21:29:01 +00:00

Owner

The point is that an instance can be configured whole from its compose.yml or its orchestrator: reproducible, reviewable, and the same on every restart, with no step done by hand in the Server pages.

What exists today (checked against `main` at `8eedc02de`)

Configuration lives in three places, and they agree only in part.

Environment variables.
- The source reads 121 POSTULO_* names, mostly through django-environ in config/settings/base.py, plus DJANGO_SETTINGS_MODULE and POSTGRES_PASSWORD.
- The documentation has drifted:
  - 17 names the code reads are missing from the wiki's Configuration page, among them POSTULO_ALLOW_WILDCARD_HOSTS, POSTULO_CACHE_MAX_ENTRIES, POSTULO_EMAIL_CODE_ATTEMPTS, …_RESENDS and …_TIMEOUT, POSTULO_MEDIA_SWEEP, …_GRACE_HOURS, POSTULO_NUMBER_RATE, POSTULO_PROXY_ADDR and POSTULO_RATELIMIT_DIR.
  - 31 are missing from .env.example.
  - The wiki documents two the code no longer reads: POSTULO_SKIP_GEONAMES and POSTULO_SKIP_PLUGIN_SYNC.
SiteSettings (core/models.py:1260). The administrator sets 44 fields in the Server pages.
- Some have an environment form that overrides them one by one: mail (POSTULO_EMAIL_*), map tiles (POSTULO_MAP_*), the geocoder (POSTULO_GEOCODER*), capture (POSTULO_CAPTURE_*) and POSTULO_REGISTRATION_OPEN.
- Many have none: instance_name, tagline, motto, sso_is_second_factor, email_sign_in, offered_languages, default_time_zone, default_paper, geocoder_consent, and the mail transport and text-message transport choices with their configuration.
PluginPolicy (plugins/models.py:107). The administrator's decision on each plugin (on, off, or on by default) has no environment form at all. The same goes for a plugin's instance-wide settings.

The names follow no written rule.

Units are spelled several ways: _MAX_MB (POSTULO_BACKUP_UPLOAD_MAX_MB, POSTULO_UPLOAD_CEILING_MB) beside _MAX_BYTES (POSTULO_CAPTURE_SOURCE_MAX_BYTES, POSTULO_LOG_MAX_BYTES), and _SECONDS, _HOURS, _DAYS and a bare _TIMEOUT (POSTULO_EMAIL_TIMEOUT) all for durations.
A flag is sometimes a noun (POSTULO_MEDIA_SWEEP) and sometimes a phrase (POSTULO_CAPTURE_IGNORE_ROBOTS).
The 22 _RATE settings take a value like 30/h. They are checked at start (core/checks.py:55), but the syntax is written down nowhere a reader of the wiki finds it.

Proposal

One registry of settings, the single source. A module (postulo.core.configuration or similar) declares each setting once:
- its environment name;
- its type: boolean, integer with unit, size, duration, rate, URL, list, choice, secret or text;
- its default, its allowed range or choices, and the database field it backs (if any);
- whether it needs a restart;
- a one-line description, marked for translation where the Server pages show it.
config/settings/*.py reads through it; it does not call env(...) itself.
Everything an administrator can set has an environment name. That covers every SiteSettings field, every PluginPolicy decision (POSTULO_PLUGIN_<NAME>=on|off|default-on|default-off), and each plugin's instance-wide setting (POSTULO_PLUGIN_<NAME>_<SETTING>, declared by the plugin through postulo.plugins.api).
- A person's own preferences stay theirs. An instance-wide default for a preference counts as an administrator's setting.
One precedence, shown where it applies: environment, then the database, then the default.
- A field the environment sets is shown read-only in the Server pages, with "Set by POSTULO_…" beside it, as mail's fields should already be.
- The startup check names each variable it does not know (a typo) and each value it refuses, with the expected syntax.
Naming rules, written in CONTRIBUTING.md and enforced by a test:
- POSTULO_ + the area + the thing, in upper snake case. The area is the Server page or the plugin it belongs to.
- A unit is the last word, and only these are allowed: _BYTES (a size), _SECONDS (a duration), _RATE (a rate). Sizes and durations also accept a suffixed value (25MB, 90s, 2h, 7d), so no name needs to carry its unit twice.
- Booleans read as a statement: POSTULO_REGISTRATION_OPEN, POSTULO_CAPTURE_KEEP_SOURCE. They take true/false/1/0/yes/no/on/off, case-insensitive.
- Lists are comma-separated. Rates are <count>/<s|m|h|d>. Choices take their documented spellings only.
- A secret may also be given as <NAME>_FILE, a path to a file holding it (Docker and Kubernetes secrets).
- Renaming follows a deprecation path: the old name keeps working for one minor and logs a warning naming the new one.
Documentation generated from the registry, never written by hand:
- the wiki's Configuration page, one table per area: name, type, default, syntax and description;
- .env.example;
- a manage.py settings command (or --list) that prints the effective value of each setting and where it came from (environment, database or default), with secrets hidden.
A test fails when the registry, the generated files and the code disagree, as the README language tables are already checked.

Decisions (recommended option first)

Scope of "every setting":
- (A, recommended) every setting an administrator can make: the environment, SiteSettings, plugin policies and plugins' instance-wide settings.
- (B) a person's own preferences as well. This is not recommended: a preference is the person's, and the environment is the instance's. Instance-wide defaults for preferences are under A.
Precedence:
- (A, recommended) the environment wins and locks the field in the Server pages.
- (B) the environment is only a first-run seed that the database then overrides. This is simpler, but the running instance no longer matches its compose.yml.
Renaming the existing names to the rules:
- (A, recommended) rename them now, keeping every old name as a deprecated alias for one minor (0.6 → removed in 0.7), with a warning at start.
- (B) keep the old names for ever and apply the rules to new names only.
Units in values or in names:
- (A, recommended) the name says the kind (_BYTES, _SECONDS), and the value may carry a suffix (25MB, 2h).
- (B) the unit fixed in the name and bare numbers only.
Where the documentation lives:
- (A, recommended) generated into the wiki's Configuration page and .env.example, by a script CI checks.
- (B) a separate docs/CONFIGURATION.md in the repository.

Done when

Every administrator's setting has an environment name that follows the rules, and the old names are deprecated aliases.
The registry is the only place a setting is declared. The wiki page, .env.example and manage.py settings come from it, and a test keeps them in step.
The Server pages show which fields the environment holds.
The startup check refuses an unknown POSTULO_* name or a malformed value, and names the syntax it expected.

The maintainer's request (2026-10-31): every feature and every setting can be set, or overridden, by an environment variable. Each one is documented properly, and the names, the defaults and the syntax of their values follow clear rules. The point is that an instance can be configured whole from its `compose.yml` or its orchestrator: reproducible, reviewable, and the same on every restart, with no step done by hand in the Server pages. ## What exists today (checked against `main` at `8eedc02de`) Configuration lives in three places, and they agree only in part. - **Environment variables.** - The source reads 121 `POSTULO_*` names, mostly through `django-environ` in `config/settings/base.py`, plus `DJANGO_SETTINGS_MODULE` and `POSTGRES_PASSWORD`. - **The documentation has drifted:** - 17 names the code reads are missing from the wiki's *Configuration* page, among them `POSTULO_ALLOW_WILDCARD_HOSTS`, `POSTULO_CACHE_MAX_ENTRIES`, `POSTULO_EMAIL_CODE_ATTEMPTS`, `…_RESENDS` and `…_TIMEOUT`, `POSTULO_MEDIA_SWEEP`, `…_GRACE_HOURS`, `POSTULO_NUMBER_RATE`, `POSTULO_PROXY_ADDR` and `POSTULO_RATELIMIT_DIR`. - 31 are missing from `.env.example`. - The wiki documents two the code no longer reads: `POSTULO_SKIP_GEONAMES` and `POSTULO_SKIP_PLUGIN_SYNC`. - **`SiteSettings`** (`core/models.py:1260`). The administrator sets 44 fields in the Server pages. - Some have an environment form that overrides them one by one: mail (`POSTULO_EMAIL_*`), map tiles (`POSTULO_MAP_*`), the geocoder (`POSTULO_GEOCODER*`), capture (`POSTULO_CAPTURE_*`) and `POSTULO_REGISTRATION_OPEN`. - Many have none: `instance_name`, `tagline`, `motto`, `sso_is_second_factor`, `email_sign_in`, `offered_languages`, `default_time_zone`, `default_paper`, `geocoder_consent`, and the mail transport and text-message transport choices with their configuration. - **`PluginPolicy`** (`plugins/models.py:107`). The administrator's decision on each plugin (on, off, or on by default) has no environment form at all. The same goes for a plugin's instance-wide settings. **The names follow no written rule.** - Units are spelled several ways: `_MAX_MB` (`POSTULO_BACKUP_UPLOAD_MAX_MB`, `POSTULO_UPLOAD_CEILING_MB`) beside `_MAX_BYTES` (`POSTULO_CAPTURE_SOURCE_MAX_BYTES`, `POSTULO_LOG_MAX_BYTES`), and `_SECONDS`, `_HOURS`, `_DAYS` and a bare `_TIMEOUT` (`POSTULO_EMAIL_TIMEOUT`) all for durations. - A flag is sometimes a noun (`POSTULO_MEDIA_SWEEP`) and sometimes a phrase (`POSTULO_CAPTURE_IGNORE_ROBOTS`). - The 22 `_RATE` settings take a value like `30/h`. They are checked at start (`core/checks.py:55`), but the syntax is written down nowhere a reader of the wiki finds it. ## Proposal 1. **One registry of settings, the single source.** A module (`postulo.core.configuration` or similar) declares each setting once: - its environment name; - its type: boolean, integer with unit, size, duration, rate, URL, list, choice, secret or text; - its default, its allowed range or choices, and the database field it backs (if any); - whether it needs a restart; - a one-line description, marked for translation where the Server pages show it. `config/settings/*.py` reads through it; it does not call `env(...)` itself. 2. **Everything an administrator can set has an environment name.** That covers every `SiteSettings` field, every `PluginPolicy` decision (`POSTULO_PLUGIN_<NAME>=on|off|default-on|default-off`), and each plugin's instance-wide setting (`POSTULO_PLUGIN_<NAME>_<SETTING>`, declared by the plugin through `postulo.plugins.api`). - A person's own preferences stay theirs. An instance-wide default for a preference counts as an administrator's setting. 3. **One precedence, shown where it applies:** environment, then the database, then the default. - A field the environment sets is shown read-only in the Server pages, with "Set by `POSTULO_…`" beside it, as mail's fields should already be. - The startup check names each variable it does not know (a typo) and each value it refuses, with the expected syntax. 4. **Naming rules, written in `CONTRIBUTING.md` and enforced by a test:** - `POSTULO_` + the area + the thing, in upper snake case. The area is the Server page or the plugin it belongs to. - A unit is the last word, and only these are allowed: `_BYTES` (a size), `_SECONDS` (a duration), `_RATE` (a rate). Sizes and durations also accept a suffixed value (`25MB`, `90s`, `2h`, `7d`), so no name needs to carry its unit twice. - Booleans read as a statement: `POSTULO_REGISTRATION_OPEN`, `POSTULO_CAPTURE_KEEP_SOURCE`. They take `true/false/1/0/yes/no/on/off`, case-insensitive. - Lists are comma-separated. Rates are `<count>/<s|m|h|d>`. Choices take their documented spellings only. - A secret may also be given as `<NAME>_FILE`, a path to a file holding it (Docker and Kubernetes secrets). - Renaming follows a deprecation path: the old name keeps working for one minor and logs a warning naming the new one. 5. **Documentation generated from the registry, never written by hand:** - the wiki's *Configuration* page, one table per area: name, type, default, syntax and description; - `.env.example`; - a `manage.py settings` command (or `--list`) that prints the effective value of each setting and where it came from (environment, database or default), with secrets hidden. A test fails when the registry, the generated files and the code disagree, as the README language tables are already checked. ## Decisions (recommended option first) - **Scope of "every setting":** - **(A, recommended)** every setting an *administrator* can make: the environment, `SiteSettings`, plugin policies and plugins' instance-wide settings. - (B) a person's own preferences as well. This is not recommended: a preference is the person's, and the environment is the instance's. Instance-wide *defaults* for preferences are under A. - **Precedence:** - **(A, recommended)** the environment wins and locks the field in the Server pages. - (B) the environment is only a first-run seed that the database then overrides. This is simpler, but the running instance no longer matches its `compose.yml`. - **Renaming the existing names to the rules:** - **(A, recommended)** rename them now, keeping every old name as a deprecated alias for one minor (0.6 → removed in 0.7), with a warning at start. - (B) keep the old names for ever and apply the rules to new names only. - **Units in values or in names:** - **(A, recommended)** the name says the kind (`_BYTES`, `_SECONDS`), and the value may carry a suffix (`25MB`, `2h`). - (B) the unit fixed in the name and bare numbers only. - **Where the documentation lives:** - **(A, recommended)** generated into the wiki's *Configuration* page and `.env.example`, by a script CI checks. - (B) a separate `docs/CONFIGURATION.md` in the repository. ## Done when - Every administrator's setting has an environment name that follows the rules, and the old names are deprecated aliases. - The registry is the only place a setting is declared. The wiki page, `.env.example` and `manage.py settings` come from it, and a test keeps them in step. - The Server pages show which fields the environment holds. - The startup check refuses an unknown `POSTULO_*` name or a malformed value, and names the syntax it expected.

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 21:29:11 +00:00

tiagoagueda added the

documentation

enhancement

labels

2026-10-31 21:29:11 +00:00

tiagoagueda referenced this issue

2026-10-31 21:33:40 +00:00

Plugins: tell the administrator an update is available, and update one or all from the interface with a single button #1187

tiagoagueda referenced this issue

2026-11-01 14:17:38 +00:00

Pipeline redesign: a per-person "Try the new look" switch during the transition #1196

tiagoagueda commented

2026-11-12 08:22:25 +00:00

Author

Owner

Decided by the maintainer (2026-11-12)

The three open points of stage 1 are left to the implementer, choosing what is best rather than what is easiest:

The generated .env.example keeps an active line for every variable a real deployment must set or that the old file set with an example value (the sender address POSTULO_DEFAULT_FROM_EMAIL among them), so a fresh cp .env.example .env behaves as before.
A plugin's own variables are declared by the plugin through the plugin surface (stage 2's declaration, brought forward), so the unknown-variable check knows them; no prefix guess.
A value that cannot be read stops the start with a message that names the variable and the expected syntax (fail early rather than run misconfigured), as stage 1 does now.

🤖 Generated with Claude Code

## Decided by the maintainer (2026-11-12) The three open points of stage 1 are left to the implementer, choosing what is best rather than what is easiest: - **The generated `.env.example`** keeps an active line for every variable a real deployment must set or that the old file set with an example value (the sender address `POSTULO_DEFAULT_FROM_EMAIL` among them), so a fresh `cp .env.example .env` behaves as before. - **A plugin's own variables** are declared by the plugin through the plugin surface (stage 2's declaration, brought forward), so the unknown-variable check knows them; no prefix guess. - **A value that cannot be read** stops the start with a message that names the variable and the expected syntax (fail early rather than run misconfigured), as stage 1 does now. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

tiagoagueda referenced this issue

2026-11-12 10:51:20 +00:00

Environment registry, the kept test database on CI, and the vector mark #1241

tiagoagueda referenced this issue from a commit

2026-11-12 10:51:20 +00:00

Declare every environment variable once, and read and document them from there

tiagoagueda referenced this issue from a commit

2026-11-12 10:51:20 +00:00

Let plugins declare their variables, keep .env.example's settings, say what to fix

tiagoagueda referenced this issue from a commit

2026-11-12 10:51:20 +00:00

Record #1186, #1033 and #1238 in the changelog and extract their strings