Run a FLOP miner on a GPU host

Run a FLOP miner on a GPU host

Availability (29 September 2026): devnet development only. A current miner is a CPU miner-worker paired with a GPU miner-vllm-toploc backend. The backend runs the model; the worker signs chain operations, handles compute-channel sessions, publishes TOPLOC evidence, and renews host-wide calibration. SOFT is the default tier and needs no confidential-computing hardware. Its full settlement/dispute path remains open (#650); HARD is specified but its bridge is not deployable. This guide does not promise mainnet participation or earnings.

The runnable path is both services on one Docker-capable NVIDIA VM or owned Linux host. The RunPod pod path is blocked for registered model identity (#1995) and lacks backend authentication (#1398); see GPU provider setup. The repository's docs/guides/miner-docker.md old chain-node image examples are not the inference miner.

1. Prepare identity, funds, model, and network

1. Obtain access to the private devnet, its tailnet, the gateway operator, the ghost-task issuer, a DA worker, and SHA-pinned private GHCR images. The current operator instructions are in infra/devnet/devnet-ts/miners/README.md. Public access details are pending: https://ops.flop.invalid and https://images.flop.invalid are placeholders, not live services.
2. Create a dedicated sr25519 miner account, separate from every validator account. Until #1552 and #1554 are fixed, a validator that submits verified work can become the entire finality committee. Keep its one-line seed in a root-owned mode-0600 file on the host and back it up privately. The worker refuses a signer file owned by someone else.
3. Fund the account for the 10,000 FLOP base self-stake, capacity stake, and fees; register the miner. Capacity stake scales with the measured cap. The A40 devnet drill budgeted roughly 1 million FLOP; query the live runtime requirement before funding. The faucet does not currently fund miner-tier amounts. Use the private operator for genesis funding.
4. Use the devnet operator's pre-registered Qwen/Qwen2.5-0.5B-Instruct model and obtain its MODEL_HASH (modelpack verity root) and exact checkpoint commit. It is the only model with a validated TOPLOC capture in the present devnet instructions. A new SOFT model can use flop model onboard --soft with no TEE claim, but must pass measured backend capture validation on its actual GPU before admission. Do not claim NvidiaCc for this non-attested backend.
5. Put the host on the private tailnet as tag:miner. The worker needs outbound WebSocket RPC and DA worker access; the gateway needs inbound TCP 8645 on the tailnet IP. Do not expose the backend's unauthenticated port or the worker's session port publicly.

The current repo file infra/devnet/devnet-ts/miners/miner.env.example is the complete field-by-field Compose template. The commands below fill in its model and policy values.

Chain, host, and account preflight

Use the approved flop CLI release and the same repository version as the pinned images. Replace the private RPC placeholder with the operator-supplied address, then check the genesis identity before any signed transaction:

flop --network devnet --rpc ws://rpc.flop.invalid network status --json
nvidia-smi
docker info

The RPC URL is a placeholder. Require a reachable RPC, the expected devnet genesis hash (matches_expected: true when the release profile pins one), and the expected runtime version. Stop on mismatch. On the GPU host also confirm Docker Compose and NVIDIA GPU container access; use an approved CUDA diagnostic image with docker run --rm --gpus all <approved-cuda-image> nvidia-smi before renting for a long period.

Generate a dedicated sr25519 mnemonic offline with the approved node CLI (zkv-relay key generate --output-type json) or an equivalent trusted key tool. Never use //Alice or a validator key. Write only the mnemonic/secret URI as one line to a private file, back it up offline, then install it on the host with sudo install -o root -g root -m 600 seed.txt /opt/flop-miner/signer. The Compose worker runs as root and rejects a signer file with the wrong owner or group/other permissions. For signed CLI operations, import the same secret into an encrypted Polkadot JSON keystore using flop keys import --type sr25519 --secret-env MINER_SECRET --out miner.json --password-env FLOP_KEY_PASSWORD on a trusted administration machine. Supply both environment variables securely, then clear them. Never put the mnemonic on a command line or in a committed file.

After the private operator funds the account, register with the encrypted keystore:

flop --network devnet --rpc ws://rpc.flop.invalid --keystore miner.json \
  --password-env FLOP_KEY_PASSWORD miner register --commission 10
flop --network devnet --rpc ws://rpc.flop.invalid --keystore miner.json \
  --password-env FLOP_KEY_PASSWORD miner status --json

The CLI miner register locks runtime-defined self-stake and rejects duplicate registration. Confirm registered: true, the expected address and stake. Capacity stake is a later measured-cap top-up by the worker, not a fixed amount to guess here.

Pin the checkpoint and decode policy

First record the approved 40-character Hugging Face commit from the model operator. Use a fresh dedicated model directory so other files do not change the root. Hugging Face's hf download reference supports --revision and --local-dir; uv's tool runner can invoke it without a system-wide Python install:

MODEL_REV='REPLACE_WITH_APPROVED_40_HEX_COMMIT'
MODEL_CHECKPOINT_DIR=/opt/flop-miner/models/qwen2.5-0.5b-instruct
uvx --from huggingface_hub hf download Qwen/Qwen2.5-0.5B-Instruct \
  --revision "$MODEL_REV" --local-dir "$MODEL_CHECKPOINT_DIR"
rm -rf "$MODEL_CHECKPOINT_DIR/.cache"
(cd tools/modelpack && uv run --project . python -m modelpack pack \
  "$MODEL_CHECKPOINT_DIR" --strict-members --json)

Only remove the download metadata from this newly downloaded model directory. Record verity_root and compare 0x<verity_root> byte-for-byte with the chain's measured root:

flop --network devnet --rpc ws://rpc.flop.invalid model get \
  Qwen/Qwen2.5-0.5B-Instruct --precision BF16 --json

For the pre-registered model, use its on-chain hash for MODEL_HASH; the backend remeasures the mount at startup. If the chain has no registered BF16 model or the roots differ, stop and ask the model operator to resolve the mismatch. For a new SOFT checkpoint, use 0x<verity_root> as the candidate MODEL_HASH, then configure and start its measured backend below before running model onboard.

From the repository root, derive the policy JSON and hash used by both services:

POLICY_FILE=infra/vultr/decode-policy-devnet-drill.json
jq -c . "$POLICY_FILE"
DECODE_POLICY_HASH="0x$(jq -c . "$POLICY_FILE" | uv run --no-project python infra/miner/vllm-toploc/decode_policy.py)"
printf '%s\n' "$DECODE_POLICY_HASH"

Place the first output, single-quoted, in DECODE_POLICY; place the printed hash in DECODE_POLICY_HASH. The backend checks that the JSON preimage hashes to this value and refuses a mismatch.

2. Configure the two services

From a checkout matching the approved images, make a private miner-01.env copy of infra/devnet/devnet-ts/miners/miner.env.example and set mode 0600. Keep it outside git. Use one copy per miner.

This is a non-secret shape example, not a ready-to-run file; the .invalid hostnames and REPLACE values must be filled from the approved release and operator:

COMPOSE_PROJECT_NAME=flop-miner-01
WORKER_NAME=devnet-ts-miner-01
FLOP_WORKER_IMAGE=ghcr.io/flop-labs/flop-core/miner-worker:sha-REPLACE
FLOP_BACKEND_IMAGE=ghcr.io/flop-labs/flop-core/miner-vllm-toploc:sha-REPLACE
FLOP_RPC=ws://rpc.flop.invalid,ws://rpc-backup.flop.invalid:9944
MINER_SIGNER_FILE=/opt/flop-miner/signer
MODEL_NAME=Qwen/Qwen2.5-0.5B-Instruct
MODEL_HASH=0xREPLACE
MODEL_CHECKPOINT_DIR=/opt/flop-miner/models/qwen2.5-0.5b-instruct
MODEL_PRECISION=BF16
TOPLOC_PRECISION=bf16
DECODE_POLICY='REPLACE_WITH_COMPACT_JSON'
DECODE_POLICY_HASH=0xREPLACE
OPENAI_BASE_URL=http://backend:8000/v1
SESSION_BIND_IP=REPLACE_WITH_TAILNET_IP
SESSION_HOST_PORT=8645
SESSION_AUTH_TOKEN=REPLACE_WITH_PRIVATE_RANDOM_VALUE
DA_URL=http://da.flop.invalid:7001

The worker Compose file sets MINER_BACKEND=openai, turns on TOPLOC, and mounts the signer read-only. The backend Compose file mounts the measured model read-only, reserves NVIDIA GPUs, and publishes no port. Keep exactly one inference backend configured. The backend requires both the model root and decode-policy hash to match their measured contents.

3. Start and inspect

Log into GHCR with a read-only read:packages token, then run from infra/devnet/devnet-ts/miners/:

docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml config --quiet
docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml pull
docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml up -d backend
docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml ps
docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml logs --tail=100 backend

The backend's private /health response reports actual model identity, precision, and policy hash. Query it from inside the backend container, where Python is already installed:

docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml \
  exec -T backend python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:8000/health").read().decode())'

Confirm model_identity: "measured", the expected model hash, precision and decode-policy hash. For a new SOFT checkpoint, arrange a private capture endpoint reachable by the publisher CLI, set BASE to that endpoint, and run onboarding now. The Compose backend has no published port; do not expose its unauthenticated API publicly. If no private endpoint is available, stop and ask the devnet operator. Use the same model name as MODEL_NAME in the env file and the policy hash derived above:

MODEL_NAME='REPLACE_WITH_NEW_MODEL_NAME'
BASE='REPLACE_WITH_PRIVATE_CAPTURE_ENDPOINT'
flop --network devnet --rpc ws://rpc.flop.invalid --keystore publisher.json \
  --password-env FLOP_KEY_PASSWORD model onboard \
  --source "$MODEL_CHECKPOINT_DIR" --precision BF16 \
  --capture-endpoint "$BASE" --decode-policy-hash "$DECODE_POLICY_HASH" \
  --registry-model "$MODEL_NAME" --backend-model-name "$MODEL_NAME" --soft

The publisher keystore must have devnet model authority. This records an empty TEE platform list. Model discovery can return it after model approval, but SOFT miner availability still uses off-chain routing because modelRegistry.registerMiner requires a TEE enum. Skip this command for the pre-registered model.

Start the worker only after the backend health check and, for a new model, successful onboarding:

docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml up -d worker
docker compose --env-file miner-01.env -f docker-compose.miner.yml -f docker-compose.backend.yml logs --tail=100 worker backend

Every worker session route, including GET /session/info, requires the bearer token. From an authorized tailnet peer, read the token silently into a temporary shell variable, then send the header through curl's standard-input config so it is absent from shell history and curl's argv:

FLOP_MINER_HOST='REPLACE_WITH_TAILNET_IP'
read -r -s -p 'Session token: ' FLOP_SESSION_PROBE_TOKEN
printf '\n'
printf 'header = "Authorization: Bearer %s"\n' "$FLOP_SESSION_PROBE_TOKEN" |
  curl --fail --silent --show-error --config - \
    "http://${FLOP_MINER_HOST}:8645/session/info" |
  jq '{minerAddress, modelHash, calibration, is_ready}'
unset FLOP_SESSION_PROBE_TOKEN

The response reports calibration and is_ready. After calibration, repeat flop --network devnet --rpc ws://rpc.flop.invalid --keystore miner.json --password-env FLOP_KEY_PASSWORD miner status --json from the administration machine and confirm its on-chain calibration record agrees with the worker's active cap and miner address. This CLI reads hwCalibration.benchmarkBurstCaps as well as miner registration. An uncalibrated host is not ready: /session/open returns 503.

The ghost-task issuer must enroll the miner account and public enclave key before jobs arrive. The worker reads live calibration constants, measures host-wide throughput, tops up capacity stake, and signs acceptance. Source runtime requirements are 64 verified jobs over at least 600 finalized blocks; a deployed devnet may still require 512 jobs over 21,600 blocks. The cap is renewed after eight fresh jobs plus required work and expires after seven days. Changed GPU inventory or fingerprint requires the initial flow again. Ask the gateway operator to allowlist the worker's tailnet URL, token, miner account, enclave key, and model identity. The worker generates a new enclave key on every start, so refresh the allowlist after restarts.

4. Operate and troubleshoot

Keep the env, signer, checkpoint, and Compose worker-data volume persistent. Record image tags and checkpoint revision before upgrades. Change both approved image pins deliberately, pull, recreate, then check backend /health, worker /session/info, finalized chain progress, and gateway access. Stop with the same Compose file pair and down; avoid --volumes unless retiring the miner and deliberately deleting calibration state. If the GPU is rented, confirm deletion in the provider console to stop billing.

For protocol stake, cap, renewal, and slashing rules, see the repository source docs/onboarding/miner.md (public source access pending). For the GPU VM and RunPod topology, continue to GPU provider setup.