---
name: container-operations
description: Inspect and diagnose an existing CitusIO node, verify readiness and message delivery, and restart it while preserving state and ownership boundaries.
---

# Operate an existing node

Begin with its saved manifest, exact images, configurations, mounts, process
identities and frontiers. Keep working images and evidence. Do not reprovision
a store or repeat a completed migration to obtain status.

For browser operation, read [panel-operations](../panel-operations/SKILL.md).
The panel is bundled in every tier; the backend determines each tier's views,
capabilities and command permissions.

For headless API operation, use the bundled
[admin operation guide](../references/agent-administration.md) and
[task catalog](../contracts/agent-tasks.json). Inspect identity and a fresh live
dashboard; require matching node/configuration/fences, available workflow and
verified role before validating one exact intent. Validation sends GETs only.
Submit an authorized intent once, preserve its idempotency key and command ID,
then observe that ID until audit and reconciliation evidence establish completion.
An accepted response is pending. If the response loses its ID, recover through
bounded events/status reads using the original exact inputs; never use a new
key to obtain status. Recovery stays read-only after write authority is withdrawn.

The CLI reads the daemon's admin snapshot. On the owning node:

```sh
ngz-iot --admin-file /run/nginz-iot/iotd/admin.json status
ngz-iot --admin-file /run/nginz-iot/iotd/admin.json readiness
ngz-iot --admin-file /run/nginz-iot/iotd/admin.json metrics
```

Check installed command support before adding optional flags/commands. Liveness,
route readiness and actual MQTT delivery are separate checks. Require fresh
daemon-owned `route_to_node=true` before routing. Verify an isolated, authorized
topic with deployed credentials/protocol/QoS and exact payload/application
receipts. Verify selected sink commitments separately from broker ACKs.

## Diagnose the failing boundary

Use your chosen Docker tooling to inspect selected image/user, health and
namespace-join facts against the extracted kit's `requirements.json`. Do not
export full environment, logs or secret hashes. Follow the README's direct
Compose/native checks before a start; these do not prove route readiness or
delivery. No supplied diagnostic client or host language runtime is required.

- **No readiness:** inspect guardian state/logs and snapshot freshness; confirm
  provider reachability and valid writer ownership.
- **Connect succeeds but MQTT does not:** compare IPC, PID and network namespace
  membership, ring settings, actual worker UID and policy-file access.
- **Permission failure:** compare UID/GID 10001 and mount ownership/modes. Keep
  credentials private and preserve the intended unprivileged identity.
- **Provider/sink failure:** inspect configured addresses, trust, credentials,
  bounded retries and quarantine state. Queued work is not successful delivery;
  do not automatically replay completed or ambiguous effects.
- **Listener/TLS failure:** inspect bindings, namespace-owner port publication,
  certificate/key access, expiry, trust and client identity.
- **Capacity pressure:** inspect backlog metrics and bounds; retain rejected
  work/timing evidence rather than weakening a guard.

Collect redacted status, image/config hashes, timestamps, command outcomes and
the first failed boundary. Keep logs, raw configs, payloads and credentials
private. Use the support command after checking its installed contract and
reviewing the resulting archive.

## Restart and recovery

Quiesce publishers and side-effecting bridges, withdraw traffic, and observe
the bounded drain condition. Stop dependent gateway/bridge containers before
replacing their namespace owner; recreate their joins when it is recreated.
Preserve durable volumes/databases. Fresh rings/readiness must come from the
recovered authority, not a stale copied snapshot.

Accepted-memory state has no durable restart guarantee. For durable state,
read `durable-recovery` in the TOC and recover without provisioning. Cross-version
upgrades need a compatible record, ring and store contract; image replacement
alone does not establish one. Verify readiness and delivery before traffic.