Operate

High availability

Sirius can run as a pool of instances with one primary at a time. Membership, leadership, failover, and config sync are in the product. Sirius does not ship etcd, Consul, ZooKeeper, or Pacemaker. It does not cluster Postgres or the message broker for you.

Day-to-day work lives on the Cluster page: join a pool, mint an enrollment token, publish a config bundle, drain, promote, and register an external peer. The same actions are REST endpoints and agent tools.

What the product does vs what you bring

flowchart TB
  subgraph inproduct [In the product]
    Pool[Pool and primary election]
    Fence[Lease and epoch fence]
    Bundle[Config bundle]
    UI[Cluster page, REST, agent]
  end
  subgraph you [You provide]
    PG[(Postgres)]
    BR[(NATS or RabbitMQ)]
    LB[Load balancer — optional]
    WIT[Witness URL — optional]
    CERT[TLS certificates — optional]
  end
  Pool --> PG
  Fence --> PG
  Bundle --> PG
  UI --> Pool
  inproduct --> BR
  LB -.-> UI
  WIT -.-> Pool
  CERT -.-> LB

Compose and Helm can start Postgres and NATS or RabbitMQ next to Sirius. That is convenience for a lab or a small box. Production HA of those stores is still an infrastructure choice.

Shared topology (default)

Every instance attaches to one datastore and one broker. There is nothing to replicate. The primary is the elected leader for singleton work (retention, attestation, port-bound listeners). Message workers run on every member.

Use this when you already run managed or clustered Postgres and want more Sirius processes.

flowchart LR
  LB[Load balancer — optional]
  subgraph pool [Shared pool]
    S0[Sirius 0 — primary]
    S1[Sirius 1]
    S2[Sirius 2]
  end
  PG[(One HA Postgres)]
  BR[(One HA broker)]
  LB --> S0
  LB --> S1
  LB --> S2
  S0 --> PG
  S1 --> PG
  S2 --> PG
  S0 --> BR
  S1 --> BR
  S2 --> BR

Setup

1. Provide one reachable Postgres and one reachable broker. Production HA of those stores is yours.
2. Point every Sirius process at that pair.
3. Form a pool on the Cluster page (or SIRIUS_CLUSTER_POOL). The first instance becomes primary. Others join with a one-time enrollment token.
4. Optionally put a load balancer in front of the HTTP ports. Terminate TLS on that ingress.
5. Set a peer-routable advertise URL (SIRIUS_CLUSTER_ADVERTISE_URL) — the public HTTPS URL, or Kubernetes headless-service DNS. A bare :8080 is not routable to peers.

Helm: cluster.enabled=true in the production values (shared, three replicas, headless service, instance id from the pod name). That wires membership. It does not make Postgres or the broker highly available.

Port-bound listeners (MLLP, inbound HTTP, and similar) bind on the primary only. A leadership change rebinds them on the new primary. Broker consumers still run everywhere.

Replicated topology (on-prem)

Each instance owns its own Postgres and broker. The primary publishes a versioned, checksummed config bundle — integrations, connectors, schemas, streams, EDI guides, and IAM role definitions. Secret values never travel. Replicas pull, verify the checksum, and apply.

Use this when sites cannot share a datastore.

flowchart LR
  subgraph siteA [Site A — config authority]
    P[Primary Sirius]
    PGA[(Local Postgres)]
    BRA[(Local broker)]
    P --> PGA
    P --> BRA
  end
  subgraph siteB [Site B]
    R[Replica Sirius]
    PGB[(Local Postgres)]
    BRB[(Local broker)]
    R --> PGB
    R --> BRB
  end
  P -->|checksummed config bundle| R

Setup

1. Each site runs its own Postgres and broker.
2. Designate one site the config authority. Edit integrations there and Publish config.
3. Join other sites with an enrollment token. Create the same secret references on each site.
4. Failover is Promote unless you turn on automatic failover. Auto-failover needs peer-routable advertise URLs and, for even-sized pools, an optional witness URL you already operate.

Users, credentials, and group memberships stay local. Only role shapes sync.

Two-site active/active

A special case of replicated: two full environments behind your load balancer, kept in sync on integration definitions only. Runtime message state is per-site and is not replicated. Make downstream delivery idempotent if the load balancer can fan the same event to both sites.

flowchart LR
  Clients[Clients] --> LB[Your load balancer]
  subgraph siteA [Site A]
    A[Sirius — config authority]
  end
  subgraph siteB [Site B]
    B[Sirius — follower]
  end
  LB --> A
  LB --> B
  A -->|config bundle| B

Setup

1. Follow the replicated steps.
2. Put your load balancer in front of both sites.
3. Register each site as an external peer of the other so reachability and TLS are validated.
4. Treat one site as the integration source of truth.

How leadership works

Leadership is not a side process. Three mechanisms, in order:

1. A lease decides who leads. A compare-and-set row in Postgres (pool-primary:<pool>), renewed every 10 seconds, valid for 30 seconds. Two instances cannot hold it at once. After a crash the lease expires and a peer takes it.
2. A local deadline decides how long a claim is believed. If an instance cannot renew (Postgres unreachable), it stands itself down before the lease TTL. That prevents a cut-off node from acting as primary for the whole outage.
3. An epoch decides which writes land. Leadership-scoped writes carry a monotonic primary_epoch checked in the same SQL statement. A superseded primary is refused and stands down.

flowchart TD
  A[Who leads?] --> B["Lease in Postgres — renew 10s, TTL 30s"]
  B -->|cannot renew| C[Local deadline — stand down]
  B -->|still holds lease| D[Act as primary]
  D --> E[Leadership-scoped write]
  E --> F[Same SQL checks epoch]
  F -->|matches| G[Write lands]
  F -->|moved| H[Refused — stand down]

The message path does not use the epoch. Inbox and outbox rows have their own claim predicates, so two writers cannot win the same row. Singleton sweeps (retention, attestation) are idempotent.

Failover

Controlled handover: Drain the current primary, optionally Promote a specific instance, then do maintenance. The old primary rejoins as a member.

What you get

Next: Deploy, Operate, Sessions.