Testing strategy
Backend layers
| Layer | Command | What it protects |
|---|---|---|
| Unit, architecture, in-process end-to-end | make test | Domain behavior, dependency boundaries, HTTP identity/authorization/admission flows |
| Race detector | make test-race | Shared-state and lifecycle concurrency under tests |
| Live infrastructure integration | make test-integration | Broker topology, persistence, projections, query behavior, recovery and registry/PKI effects |
| Load | make test-load | Gateway behavior against a real broker under declared scenarios |
| Benchmarks | make bench | Isolated allocation and execution costs |
| Reachable vulnerabilities | make vulncheck | Known reachable Go dependency vulnerabilities |
make verify aggregates lint, module checks, tests, and race checks; it does not include every expensive live-infrastructure or load scenario. make test-integration starts its own test services and uses explicit addresses from the Makefile.
Important correctness evidence
Recovery integration tests exercise engine loss during stateful windows, rebalance, scale changes, and sole-reader violation. Writer tests distinguish transient persistence failure from permanent record quarantine. API tests exercise certificate/tenant handling and denied operations. Shipped-rule tests reject untested or unheld cases.
Inventory tests matter at both item and scan-watermark boundaries. Advisory tests preserve provenance, withdrawal semantics, and feed freshness after failed synchronization. Do not generalize one happy-path integration test into a guarantee for every crash point.
Agent and contract layers
Agent tests cover local state durability/permissions, runtime/module lifecycle, strict configuration, private key integrity, architecture restrictions, and recorded backend wire exchanges. Contracts use schema linting, regeneration consistency, and Buf breaking checks. See agent verification.
Gaps
The complete endpoint-to-backend chaos suite, collector performance/soak evidence, native package tests, signed update rollback tests, and full production readiness gate remain planned. There is no comprehensive fleet benchmark or turnkey production HA validation.
Source evidence
Reviewed against the source baseline. Seagull-backend-v2/Makefile · Seagull-backend-v2/tests/integration/recovery_test.go · Seagull-agent-v2/Makefile · Seagull-contracts/Makefile.