Chapter 9 · Test

Test — Turn Uncertainty into Evidence

Test like you mean it. Start with why, and let evidence — not optimism — carry you into release.

Test phase — disciplined test culture, hypothesis-first approach building confidence through evidence.

Step Inside

Testing turns theory into evidence — replace "we ran the tests" with "we know we're ready."

Testing turns theory into evidence. Start with why: write a hypothesis, define pass/fail up front, and choose the right kind of test — developmental or acceptance; verification or validation. Two valid cultures frame the work: NASA's caution-first (simulate exhaustively, build in redundancy, verify before flight) and SpaceX's iterate-fast (prototype, fly often, learn from every failure). The Right-Sized move is to blend them: simulate for safety, iterate for speed.

Run a crisp TRR to lock versions, stabilize the environment, set roles and abort criteria, and confirm data capture works before the first run — because environmental instability loves to masquerade as system failure. On test day, over-communicate, lean on a skilled technician, keep safety nonnegotiable, and let no one freelance (the "rogue agent" quadcopter is a cautionary tale).

Afterward, document or lose it: hotwash while it's fresh, verify data integrity, log anomalies, run root-cause analysis, and update the risk register and baseline. Testing spans seven realms — from risk-reduction ("does this idea even work?") to beta ("does it solve the customer's problem?") — and the goal throughout is the same: replace "we ran the tests" with "we know we're ready."

"The more we sweat in testing, the less we bleed in production."

Action Plan

Leader Action Items for Better Test Outcomes

Four steps to get started

  1. Use the TRR Template.
    Before your next test, download the TRR checklist from the RSF Framers Community. Structure a focused, high-quality review to surface issues before hitting the range.
  2. Start with Why.
    Every test should begin with a clear purpose. Write down why you're testing and what success looks like. Then make sure every team member sees it.
  3. Hire a Tech.
    A skilled technician can catch manufacturing issues, execute safety procedures, and save the day when plans break down. Technicians are often the institutional memory of your system. Ignore them at your peril.
  4. Design for Test and Record Everything.
    Build in test points. Use debug ports, instrumentation, and cameras. A single video can validate performance, capture a surprise, or become your next PR milestone.
Test action plan — TRR checklist, hypothesis-first setup, and disciplined documentation for better test outcomes.
Test tools — PCB test points, fixtures, and instrumentation for disciplined design-for-test execution.

Directed Tools and Templates

Test smarter, not harder.

The Test Hub holds everything you need to run a right-sized test phase — TRR checklists, test plans, hotwash templates, and root-cause worksheets. Review Hub covers the gate criteria that turn test results into a confident release decision.

Self-Assessment

Test-Phase Health Check

Testing is where confidence is built or quietly lost — the difference between "we ran the tests" and "we know we're ready." This scores the six disciplines of a right-sized test phase separately, so you can see which is building confidence and which is leaving you on hope. Answer honestly; it's a mirror, not a report card.

1 = Never true · 2 = Rarely · 3 = Sometimes · 4 = Often · 5 = Consistently — answer honestly. This is a mirror, not a report card.

Purpose & Hypothesis (Start with Why) Total possible: 20

1. Every test starts with a written hypothesis and pass/fail criteria defined up front.

2. Before each test, we can state why we're running it and what success looks like — and every team member sees it.

3. We run tests to answer a specific question, not just to "see what happens."

4. We prioritize tests by risk, so the most critical failure modes get tested first.

Right Test for the Right Reason (V&V) Total possible: 20

5. We deliberately distinguish verification ("did we build it right?") from validation ("did we build the right thing?").

6. We choose the right test type for the maturity stage — developmental vs. acceptance.

7. We're clear which testing realm we're in (risk-reduction, component, subsystem, system, regression, beta) and the question it answers.

8. We test against documented verification criteria and specifications, not just "does it seem to work."

TRR & Configuration Control Total possible: 20

9. We run a focused TRR before tests — objectives, environment, roles, abort criteria, and data capture.

10. The exact hardware, firmware, and software versions under test are locked down and traceable.

11. The test environment is configured, validated, and stable before we begin.

12. We do dry runs and confirm instrumentation/data capture works before the real event.

Test-Day Execution & Safety Total possible: 20

13. A skilled technician is on-site to catch build/tolerance issues and protect equipment.

14. It's clear who has authority to pause or stop the test, and roles are respected.

15. We over-communicate — channels, terminology, and escalation set in advance.

16. Safety is nonnegotiable; protocols are followed and no one freelances on test day.

Capture the Value (Document or Lose It) Total possible: 20

17. We hold a hotwash immediately after every test, while memories are fresh.

18. We verify data integrity before the team disperses, and log and classify every anomaly.

19. We do root-cause analysis (e.g., 5 Whys, fault trees) on significant issues — not just surface fixes.

20. We update the risk register, requirements/test matrix, and system baseline after testing.

Design for Test & Continuous Testing Total possible: 20

21. We design for test early — test points, debug hooks, simulators, and cameras built in.

22. Regression tests are automated to the maximum extent, protecting hard-won capability.

23. We plan for rework (e.g., at least one PCB re-spin) rather than assuming first-time success.

24. We keep deployable, fully tested code (or a working build) at the end of every cycle/sprint.

References & Further Study

Go Deeper