Unity CI: Content-Addressed Artifacts and Selective Verification

This guide documents how Unity builds, verification, and releases work on the durable S3 artifact registry introduced for issue #684. It describes the contracts implemented on this branch. It does not report post-merge measurements; use workflow summaries and S3 inspection to validate behavior after deploy.

What changed

Before this work, every pull request and deployment paid the full Unity build and verification cost even when game inputs were unchanged. GitHub Actions caches stored multi-gigabyte Library and editor layers under keys that were hard to reuse across workflows.

The new model separates three concerns:

1. Verification selects only affected games, keeps skipped checks visible, and aggregates policy in a single unity-verification job.
2. Build artifacts are content-addressed, immutable objects in a private S3 bucket. A manifest written last signals readiness.
3. Rolling caches (Library and Bee) speed up compiles. They never change correctness: a cache miss or corrupt snapshot only costs time.

Verification path

Path classification

pnpm script resolve-unity-verify compares changed files on a pull request against configured prefixes in shared/ts/games-config/unity-ci-impact.util.ts. Each enabled Unity game receives a shouldVerify flag and a reason.

Fail-safe reasons that select every game without path matching:

When no inputs match, shouldVerify is false and reason is no_matching_inputs.

Stable skipped lint and tests

verify.yml always materializes a matrix entry for every enabled Unity game. The reusable verify-unity-game.yml workflow gates work with if: ${{ inputs.should-verify }} on both lint and tests. Unaffected games still appear in the run graph with skipped lint and test jobs rather than disappearing from the matrix.

Skipped games do not pull the Unity editor image, restore S3 caches, or bill Unity-sized runners.

unity-verification aggregate

A final unity-verification job runs if: always() after prepare-game-matrix and verify-games. It:

1. Writes a step summary table of every game, its shouldVerify value, and reason.
2. Lists resolved changed paths (or notes when none were resolved).
3. Fails only when preparation failed, or when a selected game's verification ended in failure or cancelled.

Skipped games do not fail the aggregate. A no-game pull request can pass unity-verification while every per-game lint and test job is skipped.

Content fingerprint

The artifact fingerprint is a lowercase SHA-256 over a canonical payload. It identifies what was built, independent of commit SHA.

Payload fields

The hasher concatenates, in order:

1. Schema version: trashcat-unity-artifact-v1
2. gameId
3. Build profile: development or production
4. Unity editor version (from ProjectSettings/ProjectVersion.txt)
5. Exact GameCI editor image (for example unityci/editor:ubuntu-6000.3.8f1-webgl-3)
6. Every tracked file: path\0sha256\0 sorted lexicographically by path

Tracked file inputs

For each enabled Unity game, git ls-files collects all paths under these prefixes and exact files:

Symlinks are hashed by target path. Duplicate tracked paths are rejected.

Changing any tracked byte, the build profile, Unity version, or editor image tag produces a new fingerprint and therefore a new artifact prefix.

Lanes and build profiles

Lanes express trust boundaries. Profiles express compile settings passed to Unity (-buildProfile development|production).

Preview and integration both use the development profile, so an integration-built artifact can satisfy a preview deploy when the fingerprint matches. Production never reads preview or integration artifacts.

Buckets:

Replace <app> with the deployed application name from CDK (getResourceName).

Immutable artifacts

Prefix layout

Ready artifacts live under:

artifacts/v1/<lane>/<profile>/<gameId>/<fingerprint>/

Example:

artifacts/v1/integration/development/snake-wars/a1b2c3…f0/

Manifest-last readiness

An artifact is ready only when artifact-manifest.json exists at the prefix root. UnityArtifactService.findReadyArtifact uses HeadObject on that key; object bodies may already be present, but consumers treat the prefix as a miss until the manifest lands.

publish-unity-artifact uploads all build files first, then writes artifact-manifest.json last. The manifest records schemaVersion, gameId, fingerprint, lane, profile, unityVersion, editorImage, sourceCommit, workflowRunId, and a files array (path, size, sha256). Required WebGL outputs include index.html, loader, framework, data, and wasm entries.

Publishing is idempotent: an equivalent manifest is a no-op. A conflicting manifest at the same prefix fails with a checksum conflict error.

Build concurrency

Concurrent builds for the same (lane, profile, gameId, fingerprint) serialize via a GitHub Actions concurrency group. The loser can observe artifact_status=hit on re-check and skip compilation (concurrent_hit).

Release staging and promotion

Staging path

After a registry hit or fresh build, stage-unity-artifact copies manifest-listed objects from the artifact prefix into the public games prefix:

games/<gameId>/<gameVersion>/

gameVersion is commit-derived (for example pr0123-a1b2c3d on pull-request deploys, or {branch}-a1b2c3d on branch pushes). This shape is unchanged from prior releases.

Promotion gate

promote-release runs only when all of the following succeeded:

• source-ready (generated-source gate, see below)
• unity-release-summary (every enabled Unity game staged)
• frontend-upload
• build-and-upload-games
• infrastructure

It writes app-configs/<environment>/version.json with { "gameVersion": "<gameVersion>" }, invalidates CloudFront, and updates the last-deployed commit SSM parameter. Post-deploy workflows (deploy-ephemeral, deploy-integration, deploy-production) gate notifications and follow-up work on release-promoted=true.

Artifact and cache buckets use RemovalPolicy.RETAIN. Rollback does not delete them.

Rolling cache contracts

Caches affect speed only. A missing or corrupt cache never changes which artifact prefix is selected.

Library cache key

library/v1/<lane>/<profile>/<gameId>/WebGL/<unityVersion>/<editorImageVersion>/<compatibilityHash>/latest.tar.zst

editorImageVersion is the Docker tag after the final colon in editorImage (for example ubuntu-6000.3.8f1-webgl-3).

compatibilityHash is a SHA-256 over ProjectSettings/ProjectVersion.txt, Packages/manifest.json, and Packages/packages-lock.json.

Bee cache key

bee/v1/<lane>/<profile>/WebGL/<unityVersion>/<editorImageVersion>/latest.tar.zst

Bee archives are shared across games on the same lane, profile, Unity version, and editor image. On GitHub runners, the scripts snapshot $RUNNER_TEMP/_github_home/.cache/unity3d/bee: GameCI mounts _github_home at /root, so this is Unity's default Linux Bee cache (/root/.cache/unity3d/bee) before the Editor starts. Local runs fall back to .unity-bee-cache/bee under the repository.

Lifecycle and limits

CDK applies a 30-day expiration lifecycle rule on library/v1/ and bee/v1/ prefixes. Saves warn when total cache bytes exceed 30 GiB or the bucket contains more than 9 Library / 3 Bee rolling objects.

Unity workflows no longer use actions/cache@ for Library or editor layers.

Runtime environment vs build profile

Preview and integration deploy development-profile artifacts, but Sentry, analytics, and API routing must reflect the runtime environment (pr1234, integration, production).

At frontend bootstrap, configureGameRuntimeEnvironment sets window.__FLUENCY_GAME_ENVIRONMENT__ from VITE_ENVIRONMENT (with optional ?setEnv= URL override). Unity WebGL reads it through ReactBridge.jslib → RuntimeEnvironmentProvider.GetEnvironment(fallback), using the serialized build-time profile only when the interop call fails or returns empty.

This lets preview and integration share development artifacts while runtime identity stays correct per environment.

Generated-source gate

Non-production deploys run OpenAPI regeneration before Unity release jobs. When regeneration commits new API client files, source-ready sets ready=false and skips frontend-build, prepare-game-matrix, build-and-upload-games, and promotion for that run. The deployment is superseded by the follow-up commit on the same branch.

Production skips regeneration and always sets ready=true.

Never edit generated C# manually

Bridge and shared models are generated from TypeScript Zod schemas. Generated files include an auto-generated header:

//     DO NOT EDIT THIS FILE MANUALLY

Regenerate with:

pnpm script generate-bridge-models

Manual edits are overwritten on the next generation, drift from CI fingerprints, and can fail lint or produce runtime bridge mismatches. Change the TypeScript source under frontend/src/<game>/bridge/ or shared/ts/, then regenerate.

Adding a new game or shared generator

New Unity game

1. Add a games.config.json entry with projectPath, bridgePath, and generatedCsPath.
2. Ensure bridge TS lives under the game's bridge/ folder.
3. Run pnpm script generate-bridge-models so generated C# exists before CI fingerprints the tree.
4. No workflow edit is required: resolve-unity-verify and prepare-unity-deploy discover enabled Unity games from config.

New shared generator or tracked path

If a new generator or directory affects Unity binaries for all games, add it to the global lists in shared/ts/games-config/unity-ci-impact.util.ts:

• Directory prefixes → GLOBAL_BINARY_DIRECTORY_PREFIXES or GLOBAL_VERIFICATION_DIRECTORY_PREFIXES
• Single files → GLOBAL_BINARY_FILE_PATHS or GLOBAL_VERIFICATION_FILE_PATHS

Update unity-ci-impact.util.test.ts and any contract tests. Omitting a path means changes there will not invalidate fingerprints or trigger verification.

Reading workflow summary fields

Verify (resolve-unity-verify / unity-verification)

Deploy (prepare-unity-deploy / unity-release-summary)

Script step summaries

Operator procedures

Use placeholders below. Confirm bucket names and prefixes before any destructive command.

Force a cold artifact build (one game)

Deleting only the manifest forces a miss on the next deploy while leaving prior object bodies in place (orphaned bytes may need separate lifecycle review).

# 1. Confirm the target prefix
aws s3 ls s3://<bucket>/artifacts/v1/<lane>/<profile>/<gameId>/<fingerprint>/

# 2. Review the manifest
aws s3 cp s3://<bucket>/artifacts/v1/<lane>/<profile>/<gameId>/<fingerprint>/artifact-manifest.json -

# 3. Delete ONLY the manifest (type the full key exactly)
aws s3 rm s3://<bucket>/artifacts/v1/<lane>/<profile>/<gameId>/<fingerprint>/artifact-manifest.json

# 4. Re-run deploy or Verify with a change that selects the game

To force rebuild for all games at a fingerprint, repeat for each gameId. Do not delete the entire artifacts/v1/ tree unless you intend a full cold start.

Clear corrupt Library or Bee snapshots

When restores fail or compiles behave inconsistently, delete the rolling cache object so the next job uploads a fresh archive:

# Library (per game)
aws s3 rm s3://<bucket>/library/v1/<lane>/<profile>/<gameId>/WebGL/<unityVersion>/<editorImageVersion>/<compatibilityHash>/latest.tar.zst

# Bee (shared per lane/profile/editor)
aws s3 rm s3://<bucket>/bee/v1/<lane>/<profile>/WebGL/<unityVersion>/<editorImageVersion>/latest.tar.zst

Also remove the local .unity-bee-cache directory during local reproduction if a job is stuck mid-run. GitHub runners are ephemeral.

Inspect S3 prefix sizes

# Artifact prefix byte total (manifest + WebGL outputs)
aws s3 ls s3://<bucket>/artifacts/v1/<lane>/<profile>/<gameId>/<fingerprint>/ --recursive --summarize

# Rolling cache object
aws s3 ls s3://<bucket>/library/v1/<lane>/<profile>/<gameId>/WebGL/<unityVersion>/<editorImageVersion>/<compatibilityHash>/ --recursive --summarize

Clean legacy GitHub Actions caches

New Unity workflows do not write actions/cache entries. Legacy keys from the previous pipeline may still exist.

List caches (adjust ref as needed):

gh api "/repos/<owner>/<repo>/actions/caches?per_page=100" --jq '.actions_caches[] | select(.key | startswith("unity-library-") or startswith("unity-docker-")) | {id, key, ref}'

Delete only matching legacy keys:

# Confirm ID and key before deleting
gh api --method DELETE "/repos/<owner>/<repo>/actions/caches/<cache_id>"

destroy-ephemeral.yml deletes caches for the PR merge ref only. It does not scan for unity-library- or unity-docker- prefixes globally.

Rollback

1. Release pointer: Re-run promote-release with a known-good gameVersion, or upload a prior games/<gameId>/<gameVersion>/ tree and point version.json at that version.
2. Workflow: Revert the offending commit or re-run deploy from a known-good SHA.
3. Runtime: Roll back frontend bootstrap and RuntimeEnvironmentProvider usage together if environment detection changed. Artifact buckets are retained; rollback does not purge artifacts/v1/, library/v1/, or bee/v1/.

Baseline and acceptance targets

Issue #684 recorded these pre-change measurements on a representative no-game pull request:

Acceptance targets for the new system (to be validated on this branch after merge, not assumed here):

Measure after merge using workflow summaries (unity-verification, unity-release-summary), S3 object listings, and GitHub Actions usage reports. Compare against the baseline row, not against hypothetical savings claimed before evidence exists.