Skip to main content

Testing strategy

Backend layers​

LayerCommandWhat it protects
Unit, architecture, in-process end-to-endmake testDomain behavior, dependency boundaries, HTTP identity/authorization/admission flows
Race detectormake test-raceShared-state and lifecycle concurrency under tests
Live infrastructure integrationmake test-integrationBroker topology, persistence, projections, query behavior, recovery and registry/PKI effects
Loadmake test-loadGateway behavior against a real broker under declared scenarios
Benchmarksmake benchIsolated allocation and execution costs
Reachable vulnerabilitiesmake vulncheckKnown reachable Go dependency vulnerabilities

make verify aggregates lint, module checks, tests, and race checks; it does not include every expensive live-infrastructure or load scenario. make test-integration starts its own test services and uses explicit addresses from the Makefile.

Correctness evidence​

Recovery integration tests exercise engine loss during stateful windows, rebalance, scale changes, and sole-reader violation. Writer tests distinguish transient persistence failure from permanent record quarantine. API tests exercise certificate/tenant handling and denied operations. Shipped-rule tests reject untested or unheld cases.

Inventory tests matter at both item and scan-watermark boundaries. Advisory tests preserve provenance, withdrawal semantics, and feed freshness after failed synchronization. Do not generalize one happy-path integration test into a guarantee for every crash point.

Agent and contract layers​

Agent tests cover local state durability/permissions, runtime/module lifecycle, strict configuration, private key integrity, architecture restrictions, and recorded backend wire exchanges. Contracts use schema linting, regeneration consistency, and Buf breaking checks. See agent verification.

Gaps​

The complete endpoint-to-backend chaos suite, collector performance/soak evidence, native package tests, signed update rollback tests, and full production readiness gate remain planned. There is no comprehensive fleet benchmark or turnkey production HA validation.

Source evidence​

Reviewed against the source baseline. Seagull-backend-v2/Makefile · Seagull-backend-v2/tests/integration/recovery_test.go · Seagull-agent-v2/Makefile · Seagull-contracts/Makefile.