Dashboard figures: the cache key leaves out reminders, interviews, the clock and the zone, so they contradict Gone quiet #396

New issue

Closed

opened 2026-10-01 10:46:23 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:23 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading what build() depends on against what fingerprint() hashes, and comparing cached and fresh figures in the in-memory test database.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at applications/views.py:864-876 (Snooze), :705 (ReminderCreateView), :739 (ReminderCompleteView), :752 (ReminderUpdateView), :788 (ReminderLaterView), :776 (ReminderDeleteView) and :977-987 (InterviewUpdateView.form_valid); applications/analytics.py and the other locations have not moved. Since #316 the same reminder actions are also offered on the calendar, through the same views.

Related to #231 (closed), whose item 10 introduced this cache. This issue adds what that key does not cover.

insights_for() keeps the result of build() for up to a day under fingerprint(), whose docstring says that none of its inputs moving means the figures "cannot have" moved. They can. The key covers applications, timeline entries, listings, companies, contacts, the language and the quiet threshold. build() also reads reminders, interviews, the current time and the active time zone, and none of those is in the key. So snoozing an application, correcting an interview's kind, an interview ending or a day passing leaves the cached figures wrong, while the Gone quiet and Coming up widgets beside them are computed per request and disagree.

Where:
- applications/analytics.py:303-359, fingerprint: counts and newest stamps of Application, ApplicationEvent, JobPosting, Company and Contact, plus SHAPE, the language, the user and quiet_after_days. No Reminder, no Interview, no time, no zone.
- applications/analytics.py:362-381, insights_for, with INSIGHTS_TTL of 24 hours (:300).
- applications/analytics.py:384-553, build, reads outside the key:
  - :463 interviews_ahead = diary.upcoming().count(), which compares ends_at with the time (applications/models.py:678-680);
  - :464-471 interview_kinds, from Interview.kind;
  - :476-484 and :496-497 quiet_now, quiet_by_company and every SourceRow.quiet, from quiet_applications(), whose condition (applications/models.py:83-100, annotations at :248-264) reads the time, has_future_reminder and has_unsettled_interview;
  - :541-551 by_month, where TruncMonth("applied_at") buckets in the zone core/middleware.py:127-129 activated for the request;
  - :441-454 still_waiting compares applied_at with the time, which matters only for an application dated in the future.
- Writes the key does not see: Snooze (applications/views.py:824-836) and ReminderCreateView (:671) create a Reminder; ReminderCompleteView (:693), ReminderUpdateView (:706), ReminderLaterView (:748) and ReminderDeleteView (:735) change or remove one; none touches the application or its timeline. InterviewUpdateView.form_valid (:937-947) saves a changed kind, and reschedule_interview (applications/services.py:441-442) returns before writing a timeline entry when the times are unchanged.
- core/widgets.py:368-380: the figures widgets read insights_for; Sources.quiet (the Gone quiet widget) is computed per request.
- Shown by templates/widgets/sources.html:30-50, templates/widgets/_source_table.html:33, templates/widgets/industries.html:29 and templates/widgets/interviews_summary.html:9-13.
What happens (1 to 4 were checked on the snapshot):
1. Reminders. An application has been quiet for 40 days. The person presses Snooze on it, or adds a reminder for tomorrow. Gone quiet drops it at once and a fresh build() says 0, but the cached figures still say quiet_now == 1, the Quiet column of the sources and industries tables still counts it, and the line naming quiet companies still names it. Completing, moving or deleting a reminder goes stale the other way.
2. Interviews. An interview's kind is corrected from Phone screen to Video call on its edit form. The Interviews widget keeps the old kind.
3. The clock, for the diary. An interview starting in an hour is counted In the diary; five hours later the cached figure still says 1, while Coming up on the same dashboard shows none.
4. The clock, for silence. An application sent 20 days ago crosses the 21-day threshold with nothing recorded meanwhile: Gone quiet lists it, the cached tables do not. A running scheduler mostly hides this, because announce_quiet_applications stamps updated_at on the application it announces (applications/quiet.py:98), which moves the key within one pass (300 seconds in docker/compose.yml:63). It is not hidden when no scheduler runs (it is an opt-in Compose profile, docker/compose.yml:46), or when the application was already announced for this silence, for instance quiet again once a reminder falls due.
5. The zone. After a person changes their time zone, or an administrator the instance default, the applications-per-month chart keeps the old buckets for applications sent near a month's edge.
Each lasts until an application, timeline entry, listing, company or contact changes, or 24 hours pass.
Why it matters: two widgets on one dashboard give different answers to "what has gone quiet", and the one the person just acted on appears not to have listened. quiet_condition promises that the badge and the block "can never come to disagree" and applications/quiet.py:11-12 that the figures agree with them; with the cache they do not. The fingerprint's docstring states a guarantee that a later change will rely on.
Fix: either
- take the figures that depend on the time, reminders or the diary (quiet_now, quiet_by_company, each SourceRow.quiet, interviews_ahead, interview_kinds, still_waiting) out of the cached object and compute them per request from Sources.quiet and one count query each, as Gone quiet already is; or
- add to the key the count and Max("updated_at") of the person's Reminder and Interview rows, and a time bucket that ends at the next moment a figure can change (the local date, the earliest ends_at of upcoming() and the earliest due time of an open reminder).
In both cases add the active zone's name to the key (timezone.get_current_timezone_name()), since by_month stays cached, and correct the docstring of fingerprint.

Done when

tests/test_insights.py has a test that warms insights_for, puts a future reminder on a quiet application (through Snooze and through the reminder form), and expects insights_for(user).quiet_now and the source row's quiet to equal quiet_applications(user).count(), which is 0. It fails today.
A test there changes a held interview's kind through applications:interview_update and expects interview_kinds to follow.
A test there moves the clock past the quiet threshold, and another past an interview's end, and expects the cached figures to match build().
A test there warms insights_for in one zone, activates another in which an application's applied_at falls in a different month, and expects by_month to match build().

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading what `build()` depends on against what `fingerprint()` hashes, and comparing cached and fresh figures in the in-memory test database. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `applications/views.py:864-876` (*Snooze*), `:705` (`ReminderCreateView`), `:739` (`ReminderCompleteView`), `:752` (`ReminderUpdateView`), `:788` (`ReminderLaterView`), `:776` (`ReminderDeleteView`) and `:977-987` (`InterviewUpdateView.form_valid`); `applications/analytics.py` and the other locations have not moved. Since #316 the same reminder actions are also offered on the calendar, through the same views. Related to #231 (closed), whose item 10 introduced this cache. This issue adds what that key does not cover. `insights_for()` keeps the result of `build()` for up to a day under `fingerprint()`, whose docstring says that none of its inputs moving means the figures "cannot have" moved. They can. The key covers applications, timeline entries, listings, companies, contacts, the language and the quiet threshold. `build()` also reads reminders, interviews, the current time and the active time zone, and none of those is in the key. So snoozing an application, correcting an interview's kind, an interview ending or a day passing leaves the cached figures wrong, while the *Gone quiet* and *Coming up* widgets beside them are computed per request and disagree. - **Where:** - `applications/analytics.py:303-359`, `fingerprint`: counts and newest stamps of `Application`, `ApplicationEvent`, `JobPosting`, `Company` and `Contact`, plus `SHAPE`, the language, the user and `quiet_after_days`. No `Reminder`, no `Interview`, no time, no zone. - `applications/analytics.py:362-381`, `insights_for`, with `INSIGHTS_TTL` of 24 hours (`:300`). - `applications/analytics.py:384-553`, `build`, reads outside the key: - `:463` `interviews_ahead = diary.upcoming().count()`, which compares `ends_at` with the time (`applications/models.py:678-680`); - `:464-471` `interview_kinds`, from `Interview.kind`; - `:476-484` and `:496-497` `quiet_now`, `quiet_by_company` and every `SourceRow.quiet`, from `quiet_applications()`, whose condition (`applications/models.py:83-100`, annotations at `:248-264`) reads the time, `has_future_reminder` and `has_unsettled_interview`; - `:541-551` `by_month`, where `TruncMonth("applied_at")` buckets in the zone `core/middleware.py:127-129` activated for the request; - `:441-454` `still_waiting` compares `applied_at` with the time, which matters only for an application dated in the future. - Writes the key does not see: *Snooze* (`applications/views.py:824-836`) and `ReminderCreateView` (`:671`) create a `Reminder`; `ReminderCompleteView` (`:693`), `ReminderUpdateView` (`:706`), `ReminderLaterView` (`:748`) and `ReminderDeleteView` (`:735`) change or remove one; none touches the application or its timeline. `InterviewUpdateView.form_valid` (`:937-947`) saves a changed kind, and `reschedule_interview` (`applications/services.py:441-442`) returns before writing a timeline entry when the times are unchanged. - `core/widgets.py:368-380`: the figures widgets read `insights_for`; `Sources.quiet` (the *Gone quiet* widget) is computed per request. - Shown by `templates/widgets/sources.html:30-50`, `templates/widgets/_source_table.html:33`, `templates/widgets/industries.html:29` and `templates/widgets/interviews_summary.html:9-13`. - **What happens** (1 to 4 were checked on the snapshot): 1. Reminders. An application has been quiet for 40 days. The person presses *Snooze* on it, or adds a reminder for tomorrow. *Gone quiet* drops it at once and a fresh `build()` says 0, but the cached figures still say `quiet_now == 1`, the *Quiet* column of the sources and industries tables still counts it, and the line naming quiet companies still names it. Completing, moving or deleting a reminder goes stale the other way. 2. Interviews. An interview's kind is corrected from *Phone screen* to *Video call* on its edit form. The Interviews widget keeps the old kind. 3. The clock, for the diary. An interview starting in an hour is counted *In the diary*; five hours later the cached figure still says 1, while *Coming up* on the same dashboard shows none. 4. The clock, for silence. An application sent 20 days ago crosses the 21-day threshold with nothing recorded meanwhile: *Gone quiet* lists it, the cached tables do not. A running scheduler mostly hides this, because `announce_quiet_applications` stamps `updated_at` on the application it announces (`applications/quiet.py:98`), which moves the key within one pass (300 seconds in `docker/compose.yml:63`). It is not hidden when no scheduler runs (it is an opt-in Compose profile, `docker/compose.yml:46`), or when the application was already announced for this silence, for instance quiet again once a reminder falls due. 5. The zone. After a person changes their time zone, or an administrator the instance default, the applications-per-month chart keeps the old buckets for applications sent near a month's edge. Each lasts until an application, timeline entry, listing, company or contact changes, or 24 hours pass. - **Why it matters:** two widgets on one dashboard give different answers to "what has gone quiet", and the one the person just acted on appears not to have listened. `quiet_condition` promises that the badge and the block "can never come to disagree" and `applications/quiet.py:11-12` that the figures agree with them; with the cache they do not. The fingerprint's docstring states a guarantee that a later change will rely on. - **Fix:** either - take the figures that depend on the time, reminders or the diary (`quiet_now`, `quiet_by_company`, each `SourceRow.quiet`, `interviews_ahead`, `interview_kinds`, `still_waiting`) out of the cached object and compute them per request from `Sources.quiet` and one count query each, as *Gone quiet* already is; or - add to the key the count and `Max("updated_at")` of the person's `Reminder` and `Interview` rows, and a time bucket that ends at the next moment a figure can change (the local date, the earliest `ends_at` of `upcoming()` and the earliest due time of an open reminder). In both cases add the active zone's name to the key (`timezone.get_current_timezone_name()`), since `by_month` stays cached, and correct the docstring of `fingerprint`. ## Done when - `tests/test_insights.py` has a test that warms `insights_for`, puts a future reminder on a quiet application (through *Snooze* and through the reminder form), and expects `insights_for(user).quiet_now` and the source row's `quiet` to equal `quiet_applications(user).count()`, which is 0. It fails today. - A test there changes a held interview's kind through `applications:interview_update` and expects `interview_kinds` to follow. - A test there moves the clock past the quiet threshold, and another past an interview's end, and expects the cached figures to match `build()`. - A test there warms `insights_for` in one zone, activates another in which an application's `applied_at` falls in a different month, and expects `by_month` to match `build()`.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:46:23 +00:00

tiagoagueda added the

tier

bug

labels

2026-10-01 10:46:23 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:05:01 +00:00

Key the dashboard's kept figures on what the clock and reminders move

tiagoagueda closed this issue

2026-10-04 10:05:01 +00:00