Pushing to Forgejo works from one machine and one protocol only #62
Labels
No labels
blocked-physical
cleanup
hardware
infra
kernel
P1-critical
P2-high
P3-normal
P4-later
reliability
security
upstream
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
tiagoagueda/a80#62
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Every clone of this project authenticates to Forgejo a different way, and one of them
cannot push at all. The result is that work accumulates locally without anyone noticing.
Measured 2026-08-29
a80on the workstationhttps://source.tiagoagueda.com/tiagoagueda/a80.gita80-u-booton ouranosgithub.com/u-boot/u-boota80-linuxon ouranosgithub.com/torvalds/linuxHTTPS with the API token fails
forgejo_token.txtauthenticates against the REST API perfectly well —GET /api/v1/userreturns the account with
is_admin: true, andGET /api/v1/repos/tiagoagueda/a80-u-bootreturns 200. But git-over-HTTPS with the same token as the password is refused:
That linked issue is about scoped tokens, so the likely cause is that this token carries
read scopes and
write:issue(which is all it has ever been used for) but notwrite:repository. Not confirmed — the API does not expose an existing token's scopes.SSH works, but only from the Forgejo host itself
The build host's key is registered (
zips8497@ouranos-a80 (a80 build host)) andauthenticates:
But only over
localhost:3322. The public hostname refuses that port, because Forgejo runsin a container publishing SSH on a non-standard port and nothing in front of it forwards
that port. So the working remote URL is
which is only valid on the machine Forgejo runs on. Any other clone that copies it gets a
connection refused.
A transient failure on top
A
git push origin mainthat had succeeded twice minutes earlier failed once with the sameCredentials are incorrect or have expired, then succeeded verbatim on retry with no change.That looks like a short lockout after the repeated failed token attempts above, which makes
the real problem harder to recognise: the failure mode for "wrong scope" and for "try again in
a minute" is the same message.
Why this matters more than it looks
Until 2026-08-29 the branch
vendor-firmware-and-gpu-planexisted on exactly one laptop. Itcarried 653 lines — notes 53, 54 and 55 plus
tools/awimg.py— and six open issues(#55, #56, #57, #58, #59, #60) cited those notes as their plan of record, as did the eMMC
provenance finding. Nothing warned that it was unpushed. Issue #33 was closed on the strength
of the branches existing on the server, but nothing keeps them there.
Done when
write:repository, or a deploy key, orSSH reachable on a port that resolves from off-host
a80*clone carries the same remote URL for the same repositoryNot in scope
Whether the upstream
github.comremotes should stay alongside the Forgejo ones is a separatedecision; they were removed on 2026-08-29 in favour of Forgejo-only remotes, which also means
git fetchno longer reaches upstream Linux or U-Boot from those clones.Root cause found, and my original diagnosis of the workstation half was wrong
I attributed the intermittent workstation failure to "a short lockout after repeated failed
token attempts". That is not what it is. It is OAuth access-token expiry, it is
deterministic rather than transient, and it is reproducible on demand.
The mechanism
Git Credential Manager is authenticating to Forgejo with OAuth, not with a personal access
token. Two things show it. The Windows Credential Manager holds entries under GCM's OAuth
pseudo-host:
and the credential GCM hands to git is a 517-character JWT, not the 40-character hex string
a Forgejo PAT would be. Its claims:
So the sequence is:
Credentials are incorrect or have expiredand aborts.Reproduced
With the cached token nine minutes past expiry, two identical commands back to back, nothing
changed in between:
Checking the credential afterwards shows a token minted during the failed attempt:
iat 09:12:34, exp 10:12:34. So this fails once per hour of idleness, every time - it onlylooks random because nobody pushes on a fixed schedule.
This is a different fault from the build-host one
The two failures share an error message and nothing else:
forgejo_token.txthas nowrite:repositoryscopeSharing one message is a large part of why this was confusing.
Fix
One artifact solves both: a Forgejo personal access token with
write:repository.expire, so the hourly first-attempt failure disappears.
ssh://git@localhost:3322/...remote is no longer needed.Creating it is a UI action and deliberately not automated here.
Two configuration problems found on the way
Neither causes this bug, both are worth cleaning, and both affect every repository on the
workstation rather than just this one:
Four
credential.helperentries, one of them empty.An empty value discards everything configured before it, so only the last entry is live. It
works, but the behaviour depends on file ordering.
manager-coreis also a deprecated aliasfor
manager. This should be a single entry.credential.usehttppath = trueis set globally. The system config already sets it fordev.azure.comspecifically, which is the case that needs it. Setting it globally keys everycredential by full repository path, which is why the store holds roughly twenty separate
entries for this one Forgejo host - and why a newly cloned repo always needs fresh
authentication even though the host is already trusted.
Fixing either would invalidate the existing per-path credentials and force one re-auth across
the other repositories, so it should be a deliberate change rather than a side effect.
Correction: the build-host half of this issue does not exist
My previous comment said the build host failed because the token in
forgejo_token.txtlackswrite:repository. That is wrong, and so was the original issue text. There is only onefault here, on the workstation.
The token was always fine
Listing the account's tokens shows the one in use is id 16, and its scope is
all- whichincludes
write:repository:Probing the git HTTP push endpoint directly confirms the server accepts it:
And a real push from the build host over HTTPS now succeeds:
What actually happened
The original failure was my own scripting error, not the server. The token was piped into
ssh host 'bash -s'while the script was also supplied by a heredoc:The heredoc replaces stdin, so
readtook a line of the script itself and the credentialwritten was
https://tiagoagueda:<a line of shell>@source.tiagoagueda.com. The servercorrectly rejected it, with the same "Credentials are incorrect or have expired" message that
the workstation's expired OAuth token produces - which is what made the two look like one
fault.
Consequences
ssh://git@localhost:3322/...remote wasnever necessary, and the current HTTPS-only remotes are correct as they stand.
previous comment and unaffected by this correction: a PAT stored for the host replaces the
60-minute JWT and removes the once-per-hour first-attempt failure.
credential.helperentries includingan empty one, and
credential.usehttppathset globally - still stand as cleanup.Revised done-when
the first attempt
one documented way to push that works from any clone- HTTPS with a PAT works fromboth hosts today
the token question settled- the existing token already carriesallevery- done, all HTTPS to Forgejoa80*clone carries the same remote URLResolved 2026-08-30
Fixed with a Forgejo personal access token from the vault (
Forgejo > Tiago Agueda, customfield
pat-vaultwarden-powerfull) - a 40-character PAT, verified against the push endpointbefore anything was changed:
Being a static PAT rather than a 60-minute OAuth JWT, it cannot expire, which removes the
once-per-hour first-attempt failure at its source.
What changed on the workstation
~/.gitconfigbacked up to~/.gitconfig.bak-20260830first.credential.helperconsolidated. The three global entries -manager-core, an emptyvalue, and an absolute path to the GCM executable - were removed, leaving the single
managerentry from the system config. The empty value had been silently resetting thelist, so which helper ran depended on file ordering.
credential.usehttppathunset globally. The system config still sets it fordev.azure.com, which is the case that needs it. Globally it was keying credentials by fullrepository path, which is why the store held roughly twenty separate entries for this one
host.
https://source.tiagoagueda.comwith usernametiagoagueda.What changed on the build host
credential.helper storewith~/.git-credentialsat mode 600. It had no credential helperat all before, which is why pushing from there had never worked.
Verified
Three consecutive
git push --dry-runin this repository, all exit 0 - previously the firstwould fail whenever the cached token was over an hour old.
All twelve Forgejo clones on the workstation authenticate on the first attempt from the single
host-level credential:
and both build-host clones:
Left deliberately undone
The credential store still holds the superseded per-path entries for this host, including
OAuth refresh tokens under
refresh_token.source.tiagoagueda.com. They are inert now thatusehttppathis off, and deleting the local copies would not revoke anything - the OAuthgrant itself lives in Forgejo's application settings. Worth revoking there if the OAuth path
is not wanted any more.
Closing: the fault is fixed and verified on both hosts.
Follow-up: local OAuth credentials purged
The item left undone above is done. The Windows Credential Manager held 29 entries for
this host; 28 were superseded and are now removed, leaving only the host-level PAT:
refresh_token.source.tiagoagueda.com/...(OAuth refresh tokens)source.tiagoagueda.com/tiagoagueda/<repo>.git(per-path access tokens)git:https://source.tiagoagueda.com- the PATTotal credential targets on the machine went from 119 to 91, and no
refresh_tokenentryremains anywhere, for any host.
Verified afterwards: the store returns the 40-character hex PAT rather than a JWT, and all
twelve local Forgejo clones still push on the first attempt.
Still live server-side
Purging the local copies does not revoke the authorisation. The account owns no OAuth2
applications of its own (
GET /api/v1/user/applications/oauth2returns an empty list), soGit Credential Manager was using one of Forgejo's built-in OAuth clients - which is why the
grant is not visible or revocable through the API.
To revoke it properly: Settings > Applications > Authorized OAuth2 Applications in the
Forgejo web UI. Not required for correctness now that nothing uses that path, but it is the
only way to invalidate any refresh token that was issued before today.