Hydra tutorial series — Part 5: Starting a Hydra run on a real app
The full recipe from issue to green PR, including the HYDRA_LABEL_PREFIX setting that lets multiple devs run Hydra at the same time without overwriting each other's labels. Fifth of six short modules.
The first four parts were about concept, pipelines, gates and skills. Time to actually run. In this part you start a full Hydra run on a target app, paying attention to the practical pitfalls: tokens, images, and — if you work with multiple devs against the same repos — the HYDRA_LABEL_PREFIX trick.
Step 1: clone and look around
git clone git@github.com:ConductionNL/hydra.git
cd hydra
cat CLAUDE.md
CLAUDE.md is the canonical architecture doc. Worth scrolling through the first time. Then on to secrets/ — that folder is not in git and you have to fill it yourself.
Step 2: credentials.json
Hydra reads one file for all Claude OAuth tokens and all GitHub PATs:
{
"claude_tokens": [
{ "name": "account-1", "token": "sk-ant-oat-…" },
{ "name": "account-2", "token": "sk-ant-oat-…" }
],
"git_tokens": {
"builder": "gho_…",
"reviewer": "gho_…",
"security": "gho_…"
}
}
Three separate GitHub PATs. No shared org-admin tokens. You saw the reason in part 1: each persona has its own scope.
Got multiple Claude Max accounts? Put them all in claude_tokens in order of preference. run_container_with_fallback() rotates them automatically on rate-limit.
Step 3: secrets/.env
The non-credential overrides. Critical — different for production/CI than for your laptop.
# Which images and tag to use. Locally you often build with -t localhost/hydra-*:test.
HYDRA_IMAGE_PREFIX=localhost/hydra
HYDRA_IMAGE_TAG=test
# Optional: label namespace so colleagues don't step on your toes.
HYDRA_LABEL_PREFIX=wilco
# Default review scope (ADR-020): PR diff only. Set to 'full' only when onboarding a repo.
HYDRA_REVIEW_SCOPE=diff
Tip: the defaults are ghcr.io/conductionnl/hydra as prefix and latest as tag. In production/CI you leave HYDRA_IMAGE_* off and Hydra picks up those defaults.
scripts/hydra-supervisor.sh and scripts/orchestrate.sh source this file themselves at startup. Previously you had to export it in your shell — a silly bug that went unnoticed for months and was fixed as part of the decidesk-44-45 retrospective (April 2026). Every new long-running entrypoint MUST source this file at startup; that's a repo-wide contract.
Step 4: HYDRA_LABEL_PREFIX — why and how
By default Hydra works with bare labels: build:queued, code-review:running, security-review:pass, and so on. One supervisor running, one pipeline at a time per issue. No problem.
But: as soon as multiple devs run Hydra locally against the same target repos things start crackling. Your supervisor writes build:queued. A colleague's supervisor sees that same label, picks it up, runs their build. Result: race conditions, duplicate PRs, labels ping-ponging over each other.
HYDRA_LABEL_PREFIX is the fix. Set it in secrets/.env:
HYDRA_LABEL_PREFIX=wilco
From that moment on, Hydra prefixes EVERY label it manages with wilco-:
| Default (no prefix) | With HYDRA_LABEL_PREFIX=wilco |
|---|---|
ready-to-build | wilco-ready-to-build |
build:queued | wilco-build:queued |
build:running | wilco-build:running |
code-review:queued | wilco-code-review:queued |
security-review:pass | wilco-security-review:pass |
applier:fail | wilco-applier:fail |
needs-input | wilco-needs-input |
retry:queued, rebuild:queued | wilco-retry:queued, wilco-rebuild:queued |
yolo, openspec (metadata) | not prefixed — shared metadata |
Your Hydra reads and writes only labels in its own namespace. A colleague's supervisor with HYDRA_LABEL_PREFIX=jan operates in a different namespace, so you don't see each other's in-flight work and you don't block each other.
Constraints on the value
- Regex:
^[a-z0-9]([a-z0-9-]{0,14}[a-z0-9])?$— 1 to 16 characters, lowercase a-z 0-9 and hyphen, not starting or ending with a hyphen. - Empty / unset = default behaviour (no prefix). That's also what production/CI always runs.
- Changing during a running run = DO NOT. Drain the pipeline first.
Seeding labels on a target repo
Before your first run with a new prefix you have to create the labels on the target repos once:
./scripts/create-labels.sh --repo ConductionNL/<app> --prefix wilco
Idempotent — repeated calls only overwrite colours. The script also seeds the right colour codes (red/orange/yellow/green/blue) so the board stays legible at a glance.
When to use it, when not
| Situation | Use prefix? |
|---|---|
| Production / GitHub Actions / cron-managed deployment | No. Leave it empty. Production = the canonical labels. |
| Local dev workstation, single Hydra instance | Optional. Works without too. |
| Local, multiple devs, shared target repos | Yes. Everyone their own prefix. |
| Demo / experiment on a target repo where production also runs | Yes, definitely. Otherwise production-Hydra picks up your demo issue. |
Step 5: build the container images
In CI all images are pushed to GHCR by GitHub Actions. Locally — if you have HYDRA_IMAGE_PREFIX=localhost/hydra set — you build them yourself:
# Base image: Nextcloud + PostgreSQL + OpenRegister (builder dependency)
docker build -t ghcr.io/conductionnl/hydra-nextcloud-test:stable32 \
-f images/nextcloud-test/Dockerfile .
# The three pipeline images
docker build -t localhost/hydra-builder:test -f images/builder/Dockerfile .
docker build -t localhost/hydra-reviewer:test -f images/reviewer/Dockerfile .
docker build -t localhost/hydra-security:test -f images/security/Dockerfile .
Then verify with docker images | grep hydra that the three tags are there. Reviewer and security are standalone (no NC dependency); the builder leans on the hydra-nextcloud-test base.
Step 6: start the supervisor
# Foreground for your first run so you see live what's happening
./scripts/hydra-supervisor.sh
What you expect to see:
[supervisor] loaded HYDRA_LABEL_PREFIX=wilco — restricting to own namespace
[supervisor] image prefix=localhost/hydra tag=test
[supervisor] pool size=5, slots free=5
[supervisor] polling …
In a second terminal you can follow the logs:
tail -f logs/supervisor.log
ls /tmp/hydra-slots/ # see which slots are in use
Step 7: trigger an issue
In the target repo you open an issue. On that issue you set:
- The trigger label. With prefix:
wilco-ready-to-build. Without prefix:ready-to-build. - Optionally
yolo— not prefixed, since it's metadata. - Optionally
openspecif you're working via OpenSpec.
Example via gh:
# With prefix (local dev run):
gh issue edit <N> --repo ConductionNL/<app> --add-label "wilco-ready-to-build"
# Or via the Hydra helper (syncs the board straight away):
./scripts/hydra-label.sh ConductionNL/<app> <N> add ready-to-build
scripts/hydra-label.sh is the preferred path: it respects HYDRA_LABEL_PREFIX automatically, uses the label helpers from scripts/lib/labels.sh, and syncs the associated GitHub Project board in real-time. Direct gh issue edit --add-label works too, but then the board stays "in the old column" until reconcile (every 10 minutes) picks it up.
Step 8: follow the pipeline
As soon as the supervisor sees the issue, this happens (prefixed or not, depending on your config):
ready-to-build → build:queued → build:running → build:pass
↳ code-review:queued → :running → :pass / :fail
↳ security-review:queued → :running → :pass / :fail
↳ applier:queued → :running → :pass / :fail → done / needs-input
You follow it in three places:
logs/supervisor.log— what the supervisor is doing.- GitHub issue — labels change automatically.
- GitHub PR — pipeline status comment is updated every phase by
scripts/lib/pipeline-comment.sh.
Set yolo and is everything green? Then you'll see done appear on the issue, an approval on the PR, and an auto-merge. No yolo? Then the PR is ready for a human to merge.
Troubleshooting
First-run problems
Supervisor picks default ghcr.io images even though I built locallyCheck that secrets/.env actually contains HYDRA_IMAGE_PREFIX=localhost/hydra and HYDRA_IMAGE_TAG=test — and that the file lives in secrets/ and not in your home dir. The supervisor and orchestrate.sh source secrets/.env via a fixed path relative to the repo root.
Issue stays on wilco-ready-to-build, supervisor doesn't pick it upHave you already run ./scripts/create-labels.sh --repo ... --prefix wilco? Without the prefixed labels on the target repo the supervisor polls fine, but GitHub returns no matches. Verify with gh label list --repo ConductionNL/<app> | grep wilco.
Build starts, no image found errordocker images | grep hydra-builder. No hit? Build the base image first (hydra-nextcloud-test) and then the builder. The builder Dockerfile has the base as FROM.
Issue jumps straight to wilco-needs-input without me seeing anythingStale labels from a previous run are still hanging around. Strip the old stage labels with ./scripts/hydra-label.sh ConductionNL/<app> <N> transition ready-to-build — that puts the issue back into a defined starting state.
HYDRA_LABEL_PREFIX validation fails on startupRegex is ^a-z0-9?$. No uppercase, no punctuation, no underscore, no leading/trailing hyphen. Max 16 characters.
Test yourself
Four short questions to check whether you've grasped this part. Stuck? Click Hint. Curious about the answer? Click Answer.
1. What problem does HYDRA_LABEL_PREFIX solve, and when should you explicitly NOT set it?
Hint
What happens when two Hydra instances look at the same target repo at the same time? And why is that exactly not a problem in production?
Answer
Problem it solves: as soon as multiple devs run Hydra locally at the same time against the same target repos, their supervisors see each other's labels and pick up each other's work. Race conditions, duplicate PRs, labels ping-ponging over each other. With your own prefix (wilco-build:queued) every dev gets their own namespace.
When NOT to set it:
- Production / GitHub Actions / cron-managed deployment — there you specifically want the canonical (bare) labels: a single source of truth, no stray dev prefix.
- Local solo run without colleagues on the same repos — works without too; allowed but not required.
2. Which labels ARE and which ARE NOT prefixed by HYDRA_LABEL_PREFIX?
Hint
What Hydra itself drives in its state machine gets prefixed. What is about the work itself (a property of the issue, not pipeline state) is shared.
Answer
- Prefixed: stage labels (
ready-to-build,build:queued/running/pass/fail,code-review:*,security-review:*,applier:*) and recovery labels (retry:queued,rebuild:queued,needs-input). All labels Hydra manages itself in the state machine. - NOT prefixed: metadata that describes the nature of the work, not a Hydra instance:
yolo(may auto-merge) andopenspec(follows the OpenSpec working model). Two devs on one issue agree on whether it's a yolo issue — no reason to split that per dev.
3. Why call ./scripts/hydra-label.sh instead of gh issue edit --add-label when triggering a run?
Hint
Two things the script handles automatically that the direct gh call does not.
Answer
Two reasons:
- Respects
HYDRA_LABEL_PREFIXautomatically — you typeadd ready-to-build, the script turns it intowilco-ready-to-build. Withgh issue edit --add-labelyou'd have to type the prefix by hand and update it on every change — recipe for typos and namespace drift. - Syncs the GitHub Project board immediately.
gh issue editworks too, but the board then stays "in the old column" untilreconcile.sh(every 10 minutes) picks it up. Withhydra-label.shyou see the column transition immediately.
4. What happens if you change HYDRA_LABEL_PREFIX while a Hydra run is still in progress? And what's the right way to change it?
Hint
The running run looks at labels in the old namespace, your new supervisor at another. What's left?
Answer
What happens: the running run looks at labels in the old namespace; your supervisor with the new prefix looks at a different set. Work in progress is picked up by no one (orphaned), or worse — labels from both namespaces end up on a single issue.
Right way:
- Stop putting new work into the old prefix.
- Drain the pipeline: wait until all issues with the old prefix are at
doneorneeds-input. - Only then change
HYDRA_LABEL_PREFIXinsecrets/.envand restart the supervisor. ./scripts/create-labels.sh --repo ... --prefix <new>to seed the new namespace on the target repos.
Next step
Pipeline running? Great. But what do you do when it goes wrong? Part 6 covers recovery and escalation patterns.
