thepeoples-io — main
Clone: git clone https://src.thepeoples.io/thepeoples-io/.git
README.md
# thepeoples.io AWS Infrastructure
This repository manages the AWS infrastructure for `thepeoples.io` with
OpenTofu. Run OpenTofu through `scripts/tofu`; the wrapper sources the ignored
repo-root `.env` handoff and obtains short-lived AWS credentials.
## Bootstrap Adoption
The first stack imports and manages the resources created by runbook 0001:
- `openbao-aws-broker`
- `ToFuBootstrap`
- `thepeoples-tofu-state-823027858253`
- `thepeoples-tofu-locks`
- Route53 hosted zone `Z0681058336FMIOA8450L`
- AWS budget `Monthly Cost Budget`
## Commands
```bash
scripts/tofu init
scripts/tofu fmt -recursive
scripts/tofu validate
scripts/tofu plan
scripts/build-apex-catalog.sh
scripts/validate-apex-catalog.sh
scripts/validate-sites-registry.sh
scripts/verify-transit-vendor.sh
scripts/e2e-transit.sh
scripts/e2e-apex-site.sh
scripts/e2e-release-publish.sh
scripts/e2e-source-snapshot.sh
scripts/e2e-flatpak-publish.sh
scripts/e2e-oci-publish.sh
scripts/e2e-source-spa.sh
scripts/e2e-source-app-integration.sh
```
Published-Flatpak install/test/cleanup lives in `desktop/flatpak-catalog`
(`scripts/e2e-published-remote.sh` on runner label `desktop-flatpak`).
To add an app in the published shape (artwork, screenshots, helper, scan),
follow the peer playbook `desktop/flatpak-catalog` `docs/submit-app.md`.
Budget notification subscribers are intentionally not committed. Existing
notification blocks are preserved by lifecycle policy; set
`TF_VAR_budget_alert_emails` only when deliberately creating or replacing budget
notifications.
## Budget Guardrails
The default implementation target is less than `$20/month` all-in. CloudFront
uses pay-as-you-go by default. Flat-rate plans, WAF, high-volume logging,
customer-managed KMS keys, NAT gateways, load balancers, and large AWS backup
migrations are not part of the first stack.
## Publishing Surfaces
| Surface | Domain | App kind | SPA source | Design |
|---|---|---|---|---|
| Apex catalog (Zola) | `thepeoples.io` | existing apex module | n/a | 0003 |
| Transit utility | `transit.thepeoples.io` | `spa-with-uploads` | `sites/transit-src/` | 0006 |
| Source app + release bundles | `src.thepeoples.io` | `static-source-browser` | `sites/source-src/` | 0007 |
| Flatpak ostree remote | `flatpak.thepeoples.io` | `ostree-flatpak-remote` | n/a (machine API) | 0008 |
| Public OCI registry | `oci.thepeoples.io` | `oci-distribution-remote` | n/a (machine API) | 0009 |
The apex catalog is the public front door for personal projects. The
transit utility is an anonymous-session file and text sharing app with
URL-fragment AES-GCM encryption; cross-machine retrieval is allowed via
app-wide S3 reads on `uploads/*` per the transit-specific IAM contract.
The source app hosts the source browser, per-ref publishing (`main` default;
configurable additional branches; release branches like `release/1.x` are
supported), and Shape A release bundles (a git tag is the canonical release
handle; the bundle binds source, SBOM, and binaries to the tag's commit with
six independent verifications before publish).
## Design Doc Set
| Doc | Title | Status |
|---|---|---|
| 0001 | AWS root pre-bootstrap + DNS migration (runbook) | Executed |
| 0002 | Static hosting + artifact publishing + offsite encrypted backups | Accepted |
| 0002-amendment | CloudWatch alarm budget (≤10 standard alarms) | Accepted |
| 0003 | Static site generation + open-source portfolio | Accepted |
| 0003-amendment | Surface split (apex vs source app) | Accepted (pending review) |
| 0003-amendment | Apex as a personal project catalog | Accepted |
| 0004 | Auth and identity (Cognito Identity Pools per app) | Accepted (pending review) |
| 0005 | App-specific S3 bucket provisioning (`modules/app-site`) | Accepted (pending review) |
| 0006 | Transit utility | Accepted (pending review) |
| 0007 | Source app (per-ref publishing + Shape A bundles) | Accepted (pending review) |
| 0008 | Gitea-built Flatpaks on a custom ostree remote | Accepted |
| 0009 | Public OCI registry on thepeoples.io | Accepted (Implemented spike) |
## Implementation Status
- **Apex catalog** (0002, 0003): implemented and live.
- **Auth + identity foundation** (0004): `modules/cognito-identity-pool` exists
with per-app pool + unauthenticated (and placeholder authenticated) IAM
roles. Wired into the root config.
- **App-site module** (0005): `modules/app-site` exists with bucket,
distribution, cert, IAM publisher, 5xx (+ bucket size when unbounded). Per-app-kind cache
behaviors for `static-spa`, `spa-with-uploads`, `static-source-browser`.
Wired into the root config; reads from `sites/registry/apps.json`.
- **Apps registry** (0005): `sites/registry/apps.json` + JSON Schema
+ `scripts/validate-sites-registry.sh`. Declares `transit`, `source`,
`flatpak`, and `oci`. The Flatpak hostname is live
(`https://flatpak.thepeoples.io/summary` 200; `GET /` stays 403).
The OCI hostname is provisioned as kind `oci-distribution-remote`
(`oci.thepeoples.io` CloudFront `/v2*` → Lambda Function URL, blobs via
S3 OAC). Tracker: `docs/design/0009-public-oci-registry.md`
and `docs/design/0009-spike-notes.md`.
- **Generic SPA publisher** (0005): `scripts/publish-app.sh` +
`.gitea/workflows/reusable-app-publish.yml`. Validates caller repo +
OpenBao-vended STS role, syncs `BUILD_DIR` to
`s3://<bucket>/app/<app-id>/`, generates per-file `.sha256` sidecars,
invalidates CloudFront.
- **Transit utility** (0006): SPA source at `sites/transit-src/` complete
(HTML, CSS, `crypto.mjs` WebCrypto helpers, `webauth.mjs` Cognito client
+ hand-rolled SigV4 for S3, view modules for upload/paste/retrieve/snippet).
Vendor manifest + verifier in place. Smoke test
(`scripts/e2e-transit.sh`) passes structure, crypto round-trip, and
export checks without AWS credentials.
- **Source app** (0007): per-ref publishing and Shape A process integrity
are implemented (`scripts/publish-source-snapshot.sh` + e2e; six
process integrity verifications in `scripts/publish-project-release.sh`).
The SPA shell at `sites/source-src/` is live (`/` and `/site.css` match
the committed tree). Static fallback HTML is generated for every
tree/blob URL so direct links work without JavaScript. CloudFront
invalidation runs after every publish. Per-project `.git/` directories
are generated so `git clone https://src.thepeoples.io/<slug>/.git`
works via dumb-HTTP (works with `git --no-smart-http` or older git).
Queued in 0007: live history/blame sidecars on the next ingest
(SPA slices already landed). Smart-HTTP `git-upload-pack` is a v1.1
follow-up. Daily publish is `publish-source-snapshots.yml` at 07:17 UTC:
allowlisted refs are refreshed from Gitea, in-repo symlinks are stored
as the target file (same mode; a link that leaves the commit or points
outside the include list aborts), then static objects go to S3 and
CloudFront. The platform job does not read `desktop/quarry`.
- **Custom sign-in UI** (0004 future work): deferred; gated on a future
Cognito User Pool design.
- **Flatpak ostree remote** (0008): signed remote is live
(`GET /summary` 200; `GET /` stays 403). Dedicated broker key is in
OpenBao KV `oidc-kv/thepeoples.io/aws/openbao-flatpak-broker`. Tracker:
`docs/design/0008-gitea-flatpak-custom-remote.md` § Remaining work and
§ Next. Apex `/flatpak/` is the browsable catalog (ten admitted apps,
including Plex Desktop; ChatGPT Desktop is `/flatpak/elsewhere/`).
Close-out is live
2026-08-20. CLI tools: use Mise for pins; do not build a second
Flatpak-shaped remote. Catalog schedule is below and in 0008
§ Catalog schedule (UTC). Peer detail:
`desktop/flatpak-catalog` `docs/vendor-updates.md`.
- **Public OCI registry** (0009): kind `oci-distribution-remote` is in
`modules/app-site` and `sites/registry/apps.json`. Dedicated broker
user `openbao-oci-broker` trusts only `thepeoples-oci-publisher`
(no DeleteObject). Read-only Distribution front lives in
`tools/oci-distribution-front/` and runs as a Lambda Function URL
origin for CloudFront `/v2*`. Sole public pull URL:
`https://oci.thepeoples.io`. Public promote is fail-closed without
cosign; public key is `/keys/oci.thepeoples.io.pub`. Notes:
`docs/design/0009-spike-notes.md`.
### Flatpak catalog schedule (UTC)
Gitea cron is UTC. Catalog jobs use runner `desktop-flatpak`. Ostree
publish and apex use `thepeoples-io`. A pin bump dispatches ostree then
publish then apex; the 08:17 apex tick is the catch-up if a dispatch
is skipped. Scan stays before the morning pin poll so it does not share
`desktop-flatpak` with a bump rebuild.
| UTC | Repo | Workflow | What |
|---|---|---|---|
| `47 6 * * *` (06:47) | `desktop/flatpak-catalog` | `scan-vendor-inputs.yml` | Trivy refresh of current pins; invert-dispatch apex. No ostree sign. |
| `17 7 * * *` and `17 19 * * *` (07:17, 19:17) | `desktop/flatpak-catalog` | `check-vendor-updates.yml` | Official-source poll. After 24h cooldown, commit the pin and dispatch ostree. |
| On pin commit, or `17 9 * * 1` (Mon 09:17) | `desktop/flatpak-catalog` | `build-flatpak-remote.yml` | Verify, extract, scan, GPG-sign ostree, invert-dispatch this repo’s publish. |
| After that dispatch | this repo | `publish-flatpak-remote.yml` | PUT the signed tree to `flatpak.thepeoples.io` (no S3 delete). Then dispatch apex. |
| After scan or ostree publish, or `17 8 * * *` (08:17) | this repo | `build-apex-catalog.yml` | Clone catalog peer; rebuild `/flatpak/` (pins, `//version`, scans). |
### Remaining work (ordered)
1. **Apex / src UX** — live 2026-08-17 (Source nav, `/source/` front
door, noscript links, extensionless `/<slug>` and `/<slug>/tree/`).
Transit stays queued.
2. **Flatpak close-out** — live 2026-08-20. Close-out shipped nine
admitted apps. The catalog now lists ten, including Plex Desktop.
Trivy reports, submit path, and 24h pin cooldown are live.
ChatGPT Desktop is `/flatpak/elsewhere/`. Standard Notes usage is
not finalized and is not in progress.
3. **Src explorer v2** (0007) (**now**): SPA slices landed; catalog
ingest writes `history.json`. Live `_log`/`_blame` sidecars land
when snapshot publish runs. No Lambda or search cluster.
4. **Transit** (0006): SPA is written; hostname is empty 403. Hidden
from apex nav until `scripts/publish-app.sh transit` substitutes
Cognito placeholders. Do not start this until (3) is done.
5. **Alarms** (0002 amendment): live count is **6 / 10**. Headroom
reserved; do not add 4xx/request/bytes.
## Repository Layout
```
docs/
design/ Accepted design documents (0001-0008)
runbooks/ Operational runbooks (bootstrap, handoffs, 0004 Flatpak host)
modules/
app-site/ Per-app S3 + CloudFront + ACM + IAM publisher + alarms
cognito-identity-pool/ Per-app Cognito Identity Pool with unauth/auth roles
static-site/ Apex catalog (existing)
binary-distribution/ Release artifacts (existing; reused by 0007)
sites/
apex-src/ Apex Zola source (existing)
apex/ Generated apex output (existing)
catalog/
schemas/ JSON schemas for all registries
source-snapshots/ Approved source snapshots per project (0003)
registry/
apps.json Per-app config (transit, source, flatpak)
projects.json Project registry (existing)
transit-src/ Transit SPA source (0006; new in this session)
scripts/
tofu OpenTofu wrapper (sources OpenBao handoff)
build-apex-catalog.sh Build the apex Zola site
validate-apex-catalog.sh Validate apex manifests and content
validate-sites-registry.sh Validate registries against JSON schemas
verify-transit-vendor.sh Verify AWS SDK vendor manifest SHA-256s
e2e-apex-site.sh End-to-end apex site smoke test
e2e-transit.sh End-to-end transit SPA smoke test
e2e-release-publish.sh End-to-end Shape A release publisher
e2e-source-snapshot.sh End-to-end per-ref source snapshot publisher
e2e-source-spa.sh End-to-end source SPA shell + tree/projects JSON
e2e-source-app-integration.sh End-to-end HTTP integration of published source content
e2e-cross-component.sh Cross-component static analysis (modules, IAM, registry)
e2e-flatpak-publish.sh Dry-run ostree publisher (0008; no AWS)
e2e-source-content-validation.sh End-to-end HTML escaping, binary detection, content-types
e2e-source-idempotency.sh End-to-end idempotency + concurrent publish safety
(catalog) e2e-published-remote.sh Install/test/cleanup live Flatpaks on desktop-agent-01
publish-app.sh Generic per-app SPA publishing
publish-flatpak-remote.sh Ostree/Flatpak remote publisher (no delete)
fetch-flatpak-publish-payload.sh Fetch catalog Actions artifact
publish-source-snapshot.sh Per-ref source snapshot publisher (also writes tree + projects JSON)
publish-fresh-snapshots.sh Ingest from Gitea, require freshness, then publish
ingest-source-snapshot.sh Allowlisted committed-blob ingest (policy, denylist, in-repo symlinks)
verify-snapshot-freshness.sh Fail unless the snapshot commit is the Gitea ref
e2e-source-freshness.sh Denylist, ref lock, and freshness fixture checks
publish-project-release.sh Shape A release publisher
source-openbao-aws-handoff.sh Source OpenBao-vended STS creds
.gitea/workflows/
build-apex-catalog.yml Apex Zola build workflow (existing)
reusable-app-publish.yml Generic per-app publish workflow
reusable-project-release-publish.yml Shape A project release publish
publish-flatpak-remote.yml Ostree remote publish (workflow_dispatch)
publish-source-snapshots.yml Gitea-fresh static source publish (S3 + CloudFront)
```