Skip to content

testing_developer_guide.md

Latest commit

 

History

History
557 lines (453 loc) · 52.4 KB

testing_developer_guide.md

File metadata and controls

557 lines (453 loc) · 52.4 KB
Raw

PinPoint Studio — Testing Developer Guide

Audience: Developers adding or running unit tests in PinPoint Studio Location: tests/ (umbrella + shared CMake module + presets), src/<Sub>/tests/ (the suites) Language: CMake + C++17/20 (GoogleTest or a hand-rolled main()) Status: 9 suites, runnable as one umbrella build or individually

Contents

  1. Philosophy — why tests are decoupled from the app
  2. The big picture
  3. How the umbrella orchestration works
  4. The shared module — tests/cmake/PinPointTests.cmake
  5. pp_add_test reference
  6. Running the tests
  7. Adding a test to an existing suite
  8. Adding a brand-new suite
  9. Conventions and gotchas
  10. File map

1. Philosophy — why tests are decoupled from the app

The app target is heavy: configuring it pulls in whisper.cpp/ggml, FFmpeg, espeak-ng (built from source), OpenCV and ONNX Runtime — on the order of 20+ seconds just to configure, before a single object compiles. A unit test for, say, pure shaft-kinematics math has no business waiting on any of that.

⚠ The QML module is no longer part of that argument, and the difference matters. The Gui suite declares the PinPointStudio QML module itself, from the same lists the app builds it from (cmake/PinPointQmlModule.cmake), so its offscreen UI tests can press real components. It still depends on no app target — the module's C++ needs only Qt, and its link closure stays inside sources other suites already compile. The invariant is decoupled from the app; never builds QML was only a proxy for it, and holding the proxy is what kept the only UI coverage in the repo out of every release gate on every platform.

So the test suites are not part of the app build. The root CMakeLists.txt forces BUILD_TESTING OFF; building the app never compiles a test. Instead each suite is a small, self-contained CMake project that recompiles only the handful of .cpp it needs and stubs out anything that would drag in the heavy dependencies (most notably PpLogStream, whose real implementation in pp_debug.cpp pulls in whisper/ggml).

Two consequences worth internalising:

  • There is no pinpoint_analysis / pinpoint_core / pinpoint_gui library. Tests list source files directly. The only real library target is pinpoint_buffer (src/Buffer), which the EventBuffer and IMU suites link.
  • Tests must be buildable offline-ish and fast. Anything a test needs that the app would normally provide (a log sink, EventBuffer::nowMicros, …) is supplied by a local stub file in the suite.

This decoupling is the property to preserve. Do not "simplify" by folding the suites into the app build under BUILD_TESTING ON — that reintroduces the heavy configure on every test iteration.

2. The big picture

tests/
├── CMakeLists.txt            # the umbrella: builds all suites at once
├── CMakePresets.json         # tests / tests-asan / tests-tsan presets
├── cmake/
│   └── PinPointTests.cmake   # shared infra: Qt prefix, Eigen, sanitizers, pp_add_test
└── MIGRATION.md              # how the suites were consolidated (historical record)

src/<Sub>/tests/
└── CMakeLists.txt            # one suite; uses pp_add_test; works standalone too

Every suite runs two ways, from the same src/<Sub>/tests/CMakeLists.txt:

  • Standalone — cmake -S src/<Sub>/tests -B <build> for fast single-suite iteration. A small bootstrap block at the top of each suite pulls in the shared module.
  • Under the umbrella — cmake -S tests -B build/tests configures all suites in one shot; the bootstrap block is skipped (the shared module is already loaded) and the suite is add_subdirectory'd.

This is the same file in both cases — there is no duplication and no "umbrella copy" of a suite to keep in sync.

3. How the umbrella orchestration works

tests/CMakeLists.txt is a thin driver:

cmake_minimum_required(VERSION 3.16)
project(pinpoint_tests LANGUAGES CXX)

list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
include(PinPointTests)        # Qt prefix, Eigen locate, sanitizers, gtest, pp_add_test
enable_testing()

# Buffer is special — see below.
add_subdirectory("${PP_SRC}/Buffer"       "${CMAKE_BINARY_DIR}/Buffer-lib")
add_subdirectory("${PP_SRC}/Buffer/tests" "${CMAKE_BINARY_DIR}/Buffer")

foreach(suite Analysis Audio Core Gui IMU LaunchMonitor Pose Update)
    add_subdirectory("${PP_SRC}/${suite}/tests" "${CMAKE_BINARY_DIR}/${suite}")
endforeach()

Three details make this work:

  1. The shared module is included once, at the top. It defines pp_add_test, pp_find_eigen, the sanitizer knob and the repo-layout variables (PP_SRC, PP_BUFFER, PP_CORE, PP_THIRD). Because it is loaded before any add_subdirectory, every suite sees pp_add_test already defined — which is exactly the signal each suite's bootstrap block uses to decide it is running under the umbrella (see §7).

  2. Buffer is special-cased. Its tests live under src/Buffer/tests but are gated on PROJECT_IS_TOP_LEVEL and need the real pinpoint_buffer library target. So the umbrella adds the Buffer library first, then pulls in src/Buffer/tests directly. The IMU suite also needs pinpoint_buffer; it guards its own add_subdirectory(Buffer) with if(NOT TARGET pinpoint_buffer) so the umbrella and standalone paths don't collide ("binary dir already used").

  3. A default C++20 standard is set by the shared module. Suites added as sibling directories don't inherit a parent suite's scope-local CMAKE_CXX_STANDARD, so the shared module pins the app's standard (C++20) globally. Individual targets override with STD 17 / STD 20 on pp_add_test where they care.

One configure produces one CTest registry spanning all suites:

cmake -S tests -B build/tests
cmake --build build/tests -j6
ctest --test-dir build/tests --output-on-failure   # every suite, incl. the offscreen qml_ui

4. The shared module — tests/cmake/PinPointTests.cmake

include(PinPointTests) sets up everything a suite needs. It is idempotent (include_guard(GLOBAL)), so the umbrella and a standalone bootstrap can both include it safely.

What it provides:

Thing Detail
Repo layout PP_REPO_ROOT, PP_SRC, PP_BUFFER, PP_CORE, PP_THIRD
Qt prefix The newest ~/Qt/<version>/<abi> carrying a Qt6Config.cmake (macos, gcc_64, msvc2022_64), unless -DCMAKE_PREFIX_PATH / env already set. Compared by version, not sorted, so 6.10 beats 6.9. If none is found it says so rather than silently letting CMake find some other Qt.
Qt + Threads find_package(Qt6 Core Gui) and find_package(Threads) once. Suites needing more (Qml, Bluetooth, OpenCV, Test for QSignalSpy) add their own — cheap and additive.
C++ standard Default C++20 (app/Buffer parity); per-target STD overrides.
pp_find_eigen(<out>) One Eigen locator: explicit -DPP_EIGEN_DIR, then any app build's build/*/_deps/eigen-src (matched regardless of build-dir name), then fetch 3.4.0.
pp_apply_sanitizers(<tgt>) Applied automatically by pp_add_test from the PP_SANITIZE cache var.
pp_require_gtest() Lazily fetches GoogleTest the first time a suite links it — gtest-free suites never pay for it. Called automatically by pp_add_test when GTEST/GTEST_NO_MAIN is requested.
pp_add_test(...) The canonical test-target helper (see §5).

Eigen — the bug this fixed

Before consolidation, src/Analysis/tests globbed build/*/_deps/eigen-src (correct) while src/IMU/tests hardcoded the Linux build-dir name (build/Desktop_Qt_6_11_0-Debug/...). On macOS that path never matched, so eskf_gyro_units_test was silently skipped — not failed, just absent. The single pp_find_eigen() routine removed that whole class of per-suite path drift. Always resolve Eigen through it.

Sanitizers — one knob

All suites share -DPP_SANITIZE:

cmake -S tests -B build/tests-asan -DPP_SANITIZE="address;undefined"
cmake -S tests -B build/tests-tsan -DPP_SANITIZE=thread

Use a separate build dir per sanitizer config (the flags bake into objects). pp_add_test applies them to every target automatically.

Two gotchas that have each cost an afternoon:

  • The Buffer suite does not use this knob. It predates pp_add_test and keeps its own -DPINPOINT_ENABLE_ASAN / _UBSAN / _TSAN options (see src/Buffer/CMakeLists.txt). -DPP_SANITIZE configures the umbrella without touching Buffer's targets — which reads as "TSan found nothing" when TSan never ran.
  • Never set detect_leaks=1 on arm64 macOS. LeakSanitizer is unsupported there, and every test aborts at exit — a whole suite reporting as failing, with nothing wrong. ASan itself is fine; it is only the leak detector.

5. pp_add_test reference

pp_add_test(<name>
    SOURCES <files...>            # required — test .cpp + the production .cpp it needs
    [LINK <libs...>]              # extra link libs (Qt6::Qml, ${OpenCV_LIBS}, pinpoint_buffer, …)
    [INCLUDE <dirs...>]           # extra include dirs (PP_BUFFER is always added)
    [DEFINES <defs...>]           # extra compile definitions
    [COMPILE_OPTIONS <opts...>]   # e.g. -O2 for timing-sensitive tests
    [STD 17|20]                   # CXX_STANDARD for this target (default: C++20)
    [AUTOMOC]                     # the test or its sources contain Q_OBJECT / QML_ELEMENT
    [GTEST]                       # link GTest::gtest_main (gtest supplies main())
    [GTEST_NO_MAIN]               # link GTest::gtest (the test supplies its own main())
    [NO_QT])                      # do NOT auto-link Qt6::Core/Gui (e.g. OpenCV-only math)

Behaviour:

  • By default the target links Qt6::Core + Qt6::Gui and adds PP_BUFFER to the include path (types.h is near-ubiquitous). Pass NO_QT for a pure OpenCV or header-only math test.
  • GTEST / GTEST_NO_MAIN trigger the lazy GoogleTest fetch. Use GTEST_NO_MAIN when the test needs its own main() — e.g. to spin a QCoreApplication for a Q_OBJECT + QTimer class.
  • Every target is registered with CTest (add_test) and gets sanitizer flags from PP_SANITIZE.

Examples (real, from the suites):

# Pure header-only math.
pp_add_test(wrist_angles_test SOURCES wrist_angles_test.cpp)

# OpenCV-only, no Qt, optimised hot loop.
pp_add_test(shaft_tracker_test
    SOURCES shaft_tracker_test.cpp ${ANALYSIS}/shaft_tracker_math.cpp
    INCLUDE ${OpenCV_INCLUDE_DIRS}  LINK ${OpenCV_LIBS}
    COMPILE_OPTIONS -O2  NO_QT)

# GoogleTest + profiler core + a log stub (no whisper).
pp_add_test(pp_profiler_test
    SOURCES pp_profiler_test.cpp ${PROFILER_CORE}
    INCLUDE ${CORE}  LINK Threads::Threads
    DEFINES PINPOINT_PROFILE_BASELINE=1 PINPOINT_PROFILE=1  GTEST)

# Q_OBJECT controller with its own main().
pp_add_test(profiler_controller_test
    SOURCES profiler_controller_test.cpp ${MONITOR}/profiler_controller.cpp ${PROFILER_CORE}
    INCLUDE ${CORE} ${MONITOR}  LINK Threads::Threads
    GTEST_NO_MAIN  AUTOMOC)

Platform-specific bits that don't fit the keyword form (Apple .mm Metal backends, dxgi on Windows) are applied after the pp_add_test call with plain target_sources / target_link_libraries — see the GPU-metrics targets in src/Core/tests/CMakeLists.txt.

6. Running the tests

Everything at once (umbrella)

cmake -S tests -B build/tests
cmake --build build/tests -j6
ctest --test-dir build/tests --output-on-failure

A single suite (standalone — fastest iteration)

cmake -S src/IMU/tests -B build/imu-tests
cmake --build build/imu-tests -j6
ctest --test-dir build/imu-tests --output-on-failure

(The Qt prefix is auto-resolved; pass -DCMAKE_PREFIX_PATH=/path/to/Qt/6.11.x/<abi> only if Qt is installed somewhere non-standard.)

Via presets

tests/CMakePresets.json defines tests, tests-asan, tests-tsan. CMake reads CMakePresets.json from the current directory, so run them from tests/:

cd tests
cmake --preset tests
cmake --build --preset tests
ctest --preset tests

Useful CTest filters

ctest --test-dir build/tests -R shaft        # only tests whose name matches /shaft/
ctest --test-dir build/tests -j6             # run tests in parallel
ctest --test-dir build/tests --rerun-failed --output-on-failure

7. Adding a test to an existing suite

  1. Drop my_thing_test.cpp into src/<Sub>/tests/.
  2. Add one pp_add_test call to that suite's CMakeLists.txt, listing the test plus any production .cpp it must compile (remember: there is no library to link — name the sources). Reach for the right flags from §5.
  3. If the production code touches PpLogStream, add src/Core/pp_log_stream.cpp and src/Core/PpMessageLog.cpp to SOURCES — that is the real application log and it costs Qt and nothing else. Do not pull in pp_debug.cpp: that one still drags in whisper/ggml, because it also installs the message handler and silences whisper, ggml, OpenCV and FFmpeg.
  4. Build + run standalone, then once under the umbrella.

For reference, every suite's CMakeLists.txt begins with this bootstrap so it works standalone and under the umbrella — you won't normally touch it:

# Standalone bootstrap: pull in the shared module when configured on our own.
# Under the umbrella pp_add_test already exists, so this block is skipped.
if(NOT COMMAND pp_add_test)
    cmake_minimum_required(VERSION 3.16)
    project(pinpoint_<sub>_tests LANGUAGES CXX)
    list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_LIST_DIR}/../../../tests/cmake)
    include(PinPointTests)
    enable_testing()
endif()

8. Adding a brand-new suite

Put tests next to the code they exercise, in a tests/ subfolder of that subsystem — src/<NewSub>/tests/. This keeps the relative ../foo.cpp includes short and the co-located stubs/fixtures obvious. (Don't create a single global tests/ dump of all test sources — only the orchestration is centralised, not the test files.)

A tests/ folder does not have to mean a new suite. src/Diagnostics/tests/, src/Export/tests/ and src/Metrics/tests/ hold sources with no CMakeLists.txt of their own — the Analysis suite compiles them by absolute path (SOURCES ${SRC}/Export/tests/swing_doc_test.cpp …), because each links swing_analysis.h or the pack value types and a separate project would duplicate that whole dependency set for a handful of targets. Prefer this when the new tests would pull in the same dependencies as an existing suite; add a real suite only when the dependency set genuinely differs.

  1. Create src/<NewSub>/tests/CMakeLists.txt starting with the bootstrap block from §7 (substitute the project name).
  2. Add your pp_add_test calls.
  3. Register the suite in the umbrella — add its name to the foreach list in tests/CMakeLists.txt (or, if it needs a library like Buffer, add a dedicated add_subdirectory with the same special-casing).
  4. Verify standalone, then under the umbrella.
  5. Add a one-line entry to the suite catalog in BUILDING.md.

If the new suite needs Eigen, call pp_find_eigen(<var>). If it needs a dependency the shared module doesn't find_package (OpenCV, Qt6::Bluetooth, Qt6::Qml), find_package it in the suite — that is additive and cached.

⚠ The QML UI suite is the one target that does not follow step 2, and it is worth knowing why before copying it. pp_add_test cannot express it — that helper uses plain add_executable and hardcodes Qt6::Core/Qt6::Gui, with no hook for a test ENVIRONMENT — so qml_ui_test is spelled out by hand. It also lives in src/Gui/CMakeLists.txt rather than src/Gui/tests/, pulled in by add_subdirectory: qt_add_qml_module names each file's qmlcachegen output directory after the path relative to CMAKE_CURRENT_SOURCE_DIR, so declaring it from tests/ yields qml_ui_test_../theme — which POSIX resolves and ninja on Windows rejects outright. src/Gui is an ancestor of all 157 QML files, so the paths come out clean. Anything else declaring a QML module from a test directory will hit the same wall.

9. Conventions and gotchas

  • Test framework is per-suite, not enforced. Buffer, Core, Update and LaunchMonitor use GoogleTest; Analysis, Audio, Gui, IMU and Pose use a hand-rolled main() + CHECK/CHECK_NEAR macros. Both are fine — match the suite you're editing.

  • AUTOMOC is per-target now. The old suites set a global CMAKE_AUTOMOC ON; with pp_add_test you pass AUTOMOC on the targets that have Q_OBJECT / QML_ELEMENT. Forgetting it shows up as undefined-symbol link errors for the class's signals/vtable.

  • A Q_OBJECT header with no paired .cpp must be listed in SOURCES. AUTOMOC moc's a header automatically when it shares a basename with a compiled .cpp (e.g. foo.h ↔ foo.cpp). A pure-interface or test-double header that has no matching .cpp (e.g. an abstract base, or a FakeXxx test double) is not picked up just by being #included — list the .h itself in SOURCES so AUTOMOC generates its moc_*.cpp. Symptom otherwise: undefined reference to vtable / staticMetaObject for that class. (The Update suite lists update_backend.h and fake_update_backend.h for exactly this reason.)

  • QSignalSpy lives in Qt6::Test, not Core/Gui. A suite that spies on signal emissions/ordering must find_package(Qt6 COMPONENTS Test) and LINK Qt6::Test.

  • Compiling a real settings-backed class? Isolate QSettings. The Update policy test compiles the real AppSettings/SessionController (they expose exactly the API the controller touches) rather than maintaining doubles. Because AppSettings uses a fixed QSettings org/app, the test's main() calls QStandardPaths::setTestModeEnabled(true) before QCoreApplication so it never reads or writes the developer's real settings file.

  • Never link pp_debug.cpp — it pulls in whisper/ggml for PinPointDebug::install(), which silences whisper, ggml, OpenCV and FFmpeg at startup. Link src/Core/pp_log_stream.cpp + src/Core/PpMessageLog.cpp instead: that is ppWarn()/ppInfo() itself, and it needs Qt and nothing more.

    Until 31 Aug 2026 the log primitive lived in pp_debug.cpp, so every suite linked a stub that satisfied the linker and threw every log line away — Core/tests/pp_log_stub.cpp (six suites), plus copies in Analysis/tests/imu_test_stubs.cpp and Pose/tests/pose_test_stubs.cpp. All of them now carry the real log, so a test can assert on a line instead of discarding it. Core/tests/pp_log_stub.cpp is kept for anything that deliberately wants silence; nothing in the tree needs it. (Gui/tests/reanalysis_stubs.cpp is a different thing: it stubs the re-analysis worker body, not logging.)

  • C++20 unless you say otherwise. Anything pulling in event_buffer.h → swing_window.h (std::span) needs C++20 — that's the default, so it just works; only set STD 17 to deliberately pin an older standard.

  • Sanitizers need their own build dir. Don't reconfigure an existing dir with PP_SANITIZE — make build/tests-asan / build/tests-tsan.

  • Windows/MSVC: the OpenCV-using suites (Analysis, Pose) need -DOpenCV_DIR=... at configure (the app's auto-probe isn't in the test projects), and CTest needs Qt's bin + OpenCV's bin on PATH or the test exes fail with 0xc0000135 (DLL not found).

  • build/ is gitignored. Build dirs are disposable; never commit them.

  • Never name a lambda emit. It is a Qt keyword macro. The symptom is not "redefinition" but a wall of left operand of comma operator has no effect on every call site, because the macro expands to nothing and push(a, b, c) becomes a comma expression.

  • A find() helper returns a pointer INTO a vector — bind the vector. find(buildXSeries(...), "key") dangles the moment the full expression ends, and the crash is a null deref somewhere later that looks nothing like the cause. club_delivery_test was written that way first and segfaulted; every call there now goes through a named const auto.

  • Make a synthetic swing look like a swing, in TIME as well as geometry. Producers resolve their address reference over a ±250 ms window about the Address event. A fixture compressed into 160 ms puts the TOP inside that window and contaminates the very reference the metric is measured against — which reads as a producer bug and is not one. body_rotation_test builds ~1.6 s at 240 fps for exactly this reason.

  • A decision that reads live app state cannot be tested — extract the decision. Not the same as "add a fake": if a policy reads five things off four controllers, the testable part is the JUDGEMENT, and the untestable part is the GATHERING, which is also the part that is hard to get wrong. decideStandalone() (src/LaunchMonitor) takes seven booleans and returns a verdict; the controller only fills the struct. The same split is why linux_update_logic.cpp exists. The tell that you need it: the policy has already behaved differently on a real machine than it did on paper — the launch monitor's did that twice, once because a precondition was missing entirely and once because the whole path sat behind a setting that was off, and neither was a coding error that any amount of care in the controller would have caught.

  • Enumerate the state space when it is small enough to enumerate. Seven booleans is 128 states; standalone_gate_test walks all of them and asserts exactly one records. Sampling the interesting-looking ones would not have caught a precondition added later without a thought about ordering. Where the combination count is small, exhaustive is cheaper to write than choosing representatives and arguing about the choice.

  • Return a verdict, not a bool, when several things can be false at once. Which reason gets reported is a decision somebody should be able to read and revisit, and it belongs in a test rather than in the order the ifs happened to be written. standalone_gate orders configuration before current state so a user still setting up hears about the setting rather than about capture — and there is a test asserting exactly that precedence, plus one asserting every refusal carries a reason.

  • Pin the REAL captured bytes, not a tidied version of them. gcquad_csv_parser_test embeds the actual FSX2020 row verbatim — trailing comma, CRLF and all — because the format's quirks are the thing most likely to break, and a fixture retyped into what the format "obviously" is tests the author's belief rather than the device. The same reason fake_shot.py copies the exemplar's header byte for byte instead of composing one.

  • Assert SIGNS on synthetic tracks, accuracy on a corpus. A sign is the one thing a synthetic fixture can pin exactly, and it is also the thing that silently grades every swing backwards when it is wrong. upper_body_metrics_test runs its whole sign suite twice — once normally, once through a MIRRORED camera with a left-handed golfer — because a convention that only holds for a right-hander filmed from one side is not a convention.

The session-setup UI suite — session_setup_ui

session_setup_ui_test (declared beside qml_ui_test in src/Gui/CMakeLists.txt; tests, fakes and drivers in src/Gui/tests/setup/) loads the real session-setup and calibration QML over fake cameras, sensors and a fake HackMotion device. session_setup_lint_test reads the same sources statically. Design and results: docs/design/session_wizard_refactor_design.md §7 and §11.

  • Opt-in — not in the full suite or the release gate (2026-10-17). It passes on an idle machine but crashes in Qt's render path on a loaded one: on Windows at the v0.1-beta5 gate, and on Linux the same week (offscreen + RHI got a GL 2.0 context — 43 101 "No GLSL shader code found" warnings, enough to crash QTest's failure formatter — then a render-thread race in QQuickWindow::afterFrameEnd). Configure with -DPP_SESSION_SETUP_UI_TEST=ON to build and register it. session_setup_lint_test still reads its sources and stays in the gate.
  • Run: cmake -S tests -B build/tests -DPP_SESSION_SETUP_UI_TEST=ON, then ctest --test-dir build/tests -R '^session_setup' (about nine minutes, on an otherwise idle machine). One file by hand: QT_QPA_PLATFORM=offscreen QT_QUICK_BACKEND=rhi QSG_USE_SIMPLE_ANIMATION_DRIVER=1 build/tests/Gui/qmlui/session_setup_ui_test -input src/Gui/tests/setup/<file>.qml (windows instead of offscreen on Windows). Test QML is read from the source tree at run time; production QML is compiled in.
  • It cannot make a sound. The real TingPlayer is filtered out of the target and tests/setup/silent_ting_player.h takes its QML name. Keep it that way: the suite completes dozens of calibrations.
  • Offscreen draws nothing. Behaviour cases run headless; the render cases in tst_setup_guide_render.qml skip there and need QT_QPA_PLATFORM=cocoa (or windows) on a visible desktop. SETUP_GUIDE_GRABS=<dir> saves the frames.
  • Every case fails on an unexpected QML warning (testLog). A binding that breaks at run time has no build error; this is what catches it.
  • Never construct AppSettings in a test before QSettings::setPath(IniFormat, UserScope, <scratch>). QStandardPaths::setTestModeEnabled does not move the app's ini, and the constructor now writes (the one-time imu/placement → imu/roles migration).
  • Each test file gets its own settings store (2026-11-11, #171), in this harness and in qml_ui_test. Quick Test gives every tst_*.qml its own engine, so Theme starts fresh, but a shared ini did not: Theme writes the theme, density and the like back through appSettings, the fakes write imuRoles, and the next file read them, so a test could pass in the full run and fail alone. qmlEngineAvailable() now points QSettings at <scratch>/file<N> per file and stops if the scratch directory is invalid. A file that changes the theme or a setting still puts it back in cleanup(), so tests within one file stay independent, and a test that needs a theme sets it.
  • On Windows it runs in a real window (QT_QPA_PLATFORM=windows, set by the ctest environment; by hand use windows in the line above). Offscreen + RHI has no working Quick 3D on Windows (2026-10-08): D3D11 cannot create a swapchain for an offscreen window ("Access is denied", then a warning every frame) and OpenGL aborts. The suite's window shows on screen while it runs — keep it unoccluded, or its render loop and Timers starve.
  • Every test window holds a View3DKeepAlive (in support/CalibDriver.qml, support/SetupDriver.qml and the files that use neither). Qt 6.11.0's DEBUG Quick 3D crashes the render thread when a window's LAST View3D is destroyed — every time, on a plain View3D unloaded from a Loader; Qt's release libraries survive it. That was the suite's crash in the HackMotion calibration file (QSSGRenderCamera::markDirty + 0xfeeefeee). The app holds one in each top-level window for the same reason; src/Gui/viz/View3DKeepAlive.qml has the bisection, src/Gui/tests/probes/view3d_unload.qml checks the app.
  • The test target embeds the shaders the app's QML loads (view gain, topo background, Bayer demosaic) at the app's resource paths. A missing one is an "Empty shader passed to graphics pipeline" warning on a real D3D11 window — and a failed case.

The architecture lint — architecture_lint_test

src/Gui/tests/architecture_lint_test.cpp is the guard for the refactor (docs/implementation/swe-audit/03-target-architecture.md §2 and §11). It is shaped like design_system_lint_test: it reads the source tree at run time (every C++ and QML/JS file under src/, skipping tests/, Testing/, third_party/ and build/), so it needs no rebuild when the code changes, and it is plain C++ on Qt Core, so it runs on every builder without Python.

  • The rules. A1, an #include from an inner ring into an outer one (Platform · Vocabulary · Rules · DomainModel · Application · Adapters), and, since step 2's S1-g (DEC-146), an Adapters-ring file outside src/Gui (or a src/Recording file) including a src/Gui header. A2, QML creating a session- or athlete-scoped C++ type. A3, folder-name parsing outside SwingPaths and SessionIdentity (src/DomainModel/session_identity.*, step 2.1). A4, a session or athlete file name spelled outside its owner — every FILE-SESS and FILE-ATH name of the information model (§8.1), owned by src/Export/session_files.h or src/Export/athlete_files.h (step 2.8b: session.json, session_input.json, diagnostics.json, club_baseline.json, work_ons.json, archived.json, the report's report_meta.json, *_session-report.pdf and report-stills/, .pinpoint-last-swing, trunk_ceremony*.json; athlete.json, tendencies.json, swing_themes*.json, outcome_corrections.jsonl, fault_profile.json); a writer elsewhere calls the owner's name function. A5, P11 (DEC-161; widened in S1-g from the rules directories): file or directory access (QFile, QSaveFile, QDir, QDirIterator, QFileInfo, QTemporaryFile, std::fstream, fopen, atomicWrite, FolderLock) or an include of a repository (the DomainModel interfaces and the stores and file doors of src/Export) anywhere outside src/Export and the Platform ring; src/DomainModel may include its repositories. Path text is not counted (QDir::cleanPath, a temporary QFileInfo(p).fileName(), …), nor an #include <QFile> line. A6, a Q_INVOKABLE, Q_PROPERTY, public slot or signal with a QVariantMap/QVariantList in any header. A7, a function over 300 lines in the Vocabulary and Rules rings. A8, src/Analysis including the ring buffer, device classes or app state. The header comment holds the details and the known limits; the ring table (directory defaults plus a per-file list for the directories that straddle) is in the test.
  • The allow-list. src/Gui/tests/architecture_allowlist.txt holds today's violations, one line per file and rule: path | rule | count | WHY # step N. The tag is the refactor step that removes the entry (# step 1 … # step 8, or a sub-step such as # step 3.0; # platform refactor (capture) or # platform refactor (settings) for the refactor after step 8, 05 §1 / §2; # not domain (§8.3) for a file the information model classes as not domain, kept allowlisted by Mark's answer; # unplanned if nothing places it yet, which is a question for Mark; # kept or # kept (reason) if it is deliberate). A file over its count fails. A file under its count is printed as a note: so the count can be lowered in the same change. The test ends with the size of today's debt by step.
  • Never add a line to make it pass. A new entry is an architecture decision for Mark, exactly as for the design-system allow-list. When a refactor step removes a violation, lower or delete its line in the same commit.
  • A new directory under src/ fails the test until it is given a ring in dirRings(). That forces the choice to be made on purpose.
  • Run: cmake --build build/tests --target architecture_lint_test -j 6, then ctest --test-dir build/tests -R '^architecture_lint_test$' --output-on-failure (about 2 s).

Parity dumps — regrade_ledger --dump and athlete_dump

The refactor's parity baseline (build/refactor/step-G/BRIEF.md §2) records what today's code produces so each later step can show it changed nothing it did not mean to. Two headless tools write the diagnostics and athlete halves of it. Both are built in the tests tree (src/Analysis/tests/CMakeLists.txt), are not registered with ctest because they write into the folders they are given, and share one canonical JSON writer, tools/diagnostics/parity_json.h: keys sorted, numbers %.17g, NaN as null, no timestamps or cache fingerprints, folders as <session> / <athlete>. Two runs on the same data diff byte for byte, even from two scratch copies at different paths.

  • Session level, regrade_ledger --dump <out.json> <session-dir>…. After its usual regrade it writes, per session, the ledger (every condition's tier, counts and corridor; every shot's rows by condition id with state, value, corridor and z) and, per shot, the composed outcome (shotOutcome) and its three factors, strike · direction · flight (SessionDiagnosticsModel::shotFactorsOf, the club mix's own call). Shots are ordered by id, rows and conditions by condition id.
  • Athlete level, athlete_dump --fresh --out <out.json> <athlete-dir>…. Runs the app's work-ons catch-up (WorkOnsController, synchronous) and writes tendencies.json, the club distances and each session's work_ons.json and club_baseline.json. --fresh deletes those caches first so this build derives them again. The tool refuses any folder without a scratch component in its real path.
  • Content. Both need all eight PINPOINT_CORE_* variables (PACK, NORMS, FLIGHT_NORMS, CONTEXTS, SCREENS, DRILLS, GATE_WORDS, REFERENCES) set to src/Resources/diagnostics/*.json. The tools carry no Qt resources, so without the flight table no flight would be read. Each dump records the SHA-1 of every content file.
  • Order for a parity run. Copy the sessions under /mnt/swingdata/scratch/ (cp -R, never the live library or the corpus). Then run regrade_ledger --tag parity --dump on every session, and then athlete_dump --fresh on each athlete folder. --help on either tool gives the JSON shape.
  • The writer is pinned by parity_json_test (ctest): sorted keys, full precision, run keys dropped and folders tokenised.

The step-2 migrations — migrate_step2_test and pps_migrate_step2

pps_migrate_step2 (tools/migrate/, refactor step 2.8a) runs the four step-2 data migrations M1–M4: the athlete record from the settings into athlete.json, a SessionId and session.json for every old session, the golfer's input out of diagnostics.json, and the Wrist ★ onto the athlete's record. The tool's header (tools/migrate/pps_migrate_step2.cpp) has the whole story. It is a dry run by default. --write refuses a library that is not under /mnt/swingdata/scratch/ unless Mark's approval flag is given.

  • The tool is built in the tools tree (root CMakeLists.txt, PINPOINT_BUILD_STORAGE_TOOLS): cmake --build build/tools-parity-ninja --target pps_migrate_step2 -j 6. On the cabin PC, tools/studio/studio-build.cmd migrate builds it in the same tree. That is a mode of its own, so a failure there never fails tool or both.
  • The test is in the tests tree (Analysis suite): cmake --build build/tests --target migrate_step2_test -j 6, then ctest --test-dir build/tests -R '^migrate_step2_test$' --output-on-failure (under a second). It compiles the tool's engine (tools/migrate/migrate_step2.*) and drives it on a fixture library in a QTemporaryDir. The fixture's settings file is its own, and QSettings::setPath points ppSettings() at it, so the real ones are never reached. The fixture holds legacy sessions (two on one day whose name order is not their start order, one whose name and shots disagree on the type, one with no swing), a renamed athlete, two uuids in one folder, two machines' records, an app-backfilled session.json (J-19), golfer input in the older keys (one entry for a shot that is gone), golfer input only a backup holds, and the ★ setting.
  • What it checks. The dry-run plan, before and after Mark's decisions. That a dry run writes nothing: every file of the fixture is hashed before and after. Every athlete-id decision is honoured or refused (this machine's id, the other machine's, or a uuid the folder's swings record), and M1 blocks when no swing records the id it would settle (J-58). The guards: a write outside the scratch roots, a held folder lock, the app running and backups inside an athlete folder are each refused, and nothing is written. Every path is compared resolved, so a symlink out of scratch is refused, whether it is the library or an athlete folder in it. An archive mirror outside scratch is refused, a lock in it refuses the write, and its old session.json is backed up. A failed review write, or a failed M3 check, leaves diagnostics.json byte-identical. A scratch write, read back through the app's own doors (AthleteStore, SessionStore, SessionDiagnosticsFiles::golferInput, the reviews). The backups' manifest. And that a second run plans nothing and leaves every file as it was.
  • Dry runs on live trees. build/refactor/step-2/2.8a/dry_run_proof.sh <label> <library> [<settings>] runs the tool twice. It proves the run wrote nothing: a listing of path, size and mtime before and after; the SHA-256 of every file the tool read, which the tool lists with --read-list; and the settings file and PinPoint plists, hashed and diffed.

Parity baseline — tools/parity/

The runner around the parity dumps. It takes one baseline run under a label, and compares two runs. Each refactor step re-runs it and diffs against the baseline taken at the commit Stage 0 lands on (build/refactor/step-G/BRIEF.md §2). The scripts are standard-library Python, run with python3 -I, and each has --help. A run reads the library and the corpus and writes only under /mnt/swingdata/scratch/parity/<label>/ (C:\PinPointStudio\scratch\parity\<label>\ on the cabin PC). It refuses any destination whose real path (symlinks resolved) is not inside that root, and it refuses a label that is already used.

  • The set, parity_set.json (written by parity_select.py, which reads the library and the corpus without writing them). It holds eight whole sessions, one per category, as paths relative to the library root: camera + IMU + launch monitor (18 Aug), launch monitor only (4 Aug), the Premier launch monitor (2 Nov), the untaped 6 iron (9 Sep), Jess and Nick (20 Oct), Chris, and one multi-club session. The file says why each one was chosen. It also lists the 61 pinned-pose swings of corpus/pose2, each with its pinned face-on pose and, for the 34 two-camera swings, its pinned DTL pose from corpus/pose2_dtl. ViTPose on CUDA is not deterministic, so both views must be pinned. Re-run parity_select.py only to change the set on purpose. A category with no candidate stops it.
  • Mac half, parity_baseline.py <label>. It copies the eight sessions to <label>/lib/. It then runs regrade_ledger --tag parity --dump over them, then athlete_dump --fresh over their athlete folders, and deletes the copies (--keep-copies keeps them). --athlete-files (step 2.8b) also copies the plain files directly in each athlete folder — on a migrated copy, M1's athlete.json — so a run after the step-2 migrations measures the library they leave; off by default, so a baseline copies only the sessions, as it always did. Output goes to out/diagnostics.json, out/athlete.json and out/run_mac.json (commit, binary hashes, exit codes, seconds per step), with the tools' logs in out/logs/. Build regrade_ledger and athlete_dump in the tests tree at the commit under test first. About 50 s in all: the copy takes 22 s, the session dump 18 s and the athlete dump 5–6 s.
  • Analysis half, on the cabin PC only (it is a 61-swing sweep). First build swinglab_run with tools/studio/studio-build.cmd tool. Then run ssh developer@GOLFSIMPC.local "cmd /c C:\Users\developer\Projects\PinPointStudio\tools\parity\run_analysis_golfsimpc.cmd <label>". It runs parity_baseline.py --half analysis: swinglab_run --out --doc-blocks with --pose (and --dtl-pose) per swing, in place in build\Release-Installer. It writes out/analysis.json, one digest per swing, and out/run_analysis.json (per-swing seconds and exit codes), then deletes the result trees. It takes about 6–8 s a swing. The digest is result.json minus three things: the wall times (analysis.timings, and skeleton3d.diagnostics.ms and .plane.branchMs, the only fields that moved between two runs of one swing), the source path, and the pose arrays, which become {count, sha1}. It also keeps runmeta's verdict (ok, error, score, …). --doc-blocks adds the swing document's summary block and this pass's imuIntegrity/captureIntegrity verdicts, as a write-back of the same analysis would store them. The verdicts are reached as reanalyzeSwingDir reaches them, and the source document goes through the production writer (RecordingRepository::create) and is read back. The cabin's swinglab_run must be built with that option. A swing fails, and the run exits 1, when it has no result.json, a non-zero exit, or a result.json with no summary (the doc-blocks write failed). Its runner log is kept as out/logs/<id>.runner.log. parity_baseline.py --self-test checks that rule. --only <id> runs a subset, for a one-swing smoke test on the Mac.
  • Compare, parity_diff.py <labelA> <labelB>. It compares analysis.json, diagnostics.json and athlete.json.
    • Whole numbers are compared exactly when both sides are whole: a JSON integer, or a float with no fraction. That covers counts, ids and ms or µs timestamps, which a relative tolerance would let move by milliseconds. 1 and 1.0 are equal.
    • Other numbers are equal within a relative tolerance, --rtol 1e-9 by default, or an absolute --atol 1e-12 near zero.
    • inf equals only the same-signed inf, and NaN only NaN. Neither ever passes the tolerance.
    • States, labels and types are compared exactly.
    • A key or a file in one run only is reported.
    • Arrays of objects are matched by their identity (session, swing, shotId, conditionId, id, key, …), so one extra shot does not shift every row after it. An order change is reported once.
    • The output is a Markdown list, first by field (path, count, largest relative change, one example) and then by swing or session, with an empty box on each line for "explained". --out writes it to a file.
    • Exit 0 means no difference and 1 means at least one. Exit 2 is a usage error, or nothing compared: both runs hold none of the three files, or a file is empty or not JSON. So a run that never wrote its outputs cannot pass.
    • parity_diff.py --self-test checks these rules on 27 built-in cases.
  • Order for a step's check. If tools/parity/ is not committed, copy it to /mnt/swingdata/scratch/parity/_tools/ and run C:\PinPointStudio\scratch\parity\_tools\run_analysis_golfsimpc.cmd instead; the .cmd finds its Python beside itself. Run both halves under the same label at the step's commit. Then run parity_diff.py <baseline> <label> --out <step folder>/parity-differences.md. Delete the run folder once its differences are judged, keeping the baseline.

10. File map

Path Role
tests/CMakeLists.txt Umbrella — builds all suites in one configure
tests/cmake/PinPointTests.cmake Shared infra: Qt prefix, pp_find_eigen, sanitizers, lazy gtest, pp_add_test
tests/CMakePresets.json tests / tests-asan / tests-tsan presets (run from tests/)
tests/MIGRATION.md Historical record of the consolidation + remaining cleanup
src/Analysis/tests/ 84 — by far the largest. Wrist-assessment engine, the stage mechanism and one test per stage, segmentation/metrics, shaft + clubhead + ball tracking, pose helpers, scoring and norms, the diagnostics/characteristic packs, orientation filters, IMU driver parse, the shot arbiter, and the Export round-trip. Reach for ctest -R <pattern> here, not the whole suite.
src/Buffer/tests/ Lock-free ring, timeline merge, watchdog, thread-policy, fuzz (8, GoogleTest, links pinpoint_buffer). Not migrated to pp_add_test — own gtest fetch, own PINPOINT_ENABLE_* sanitizer options.
src/Core/tests/ Resource profiler (core, concurrency/TSan, compile-out), OS + GPU metrics, stats log, profiler controller (7, GoogleTest); the standard club table and the Height field (2, GoogleTest); the two persistence mechanisms and the one worker pool (3, hand-rolled main, Qt Core only): atomic_write_test — the one atomic write (atomic_write.h, R-PERS-2): a file replaced whole, a missing folder made (or refused with intoExistingFolder), the original byte-identical after an unwritable folder, a read-only file or a refused rename, no temporary left, the streaming forms (a writer callback and AtomicFile held across calls), text mode; moved here from Diagnostics/tests/pack_io_test with the helper (refactor step 2.0), which now pins only pack_io's forwarding call — and folder_lock_test — the advisory folder lock (folder_lock.*, R-PERS-6): two lock objects (held → Locked, naming the holder, then free once let go), a lock left by another machine freed only by age (stale after 60 s), a missing folder not made, and (step 2.8b) lockOrJoin joining a hold the same thread already has — never another thread's, while a plain lock() on the same thread still answers Locked — and a hold taken through a symlink whose folder vanishes mid-hold leaving no stale entry behind (the registry key kept from acquisition) — and work_pool_test — the one worker pool, its serial lanes and the post-to-owner receiver (work_pool.*, R-CON-5; refactor step 3.1, D15): a lane keeps order (60 jobs on four workers, never two at once); two lanes do not block each other; a cancelled lane drops its waiting jobs and releases their values at once while its running job finishes, and a submit after cancelling a running lane waits behind the running job (the lane stays serial); destroying the pool with a job running and jobs waiting never runs the waiting ones, releases their values, and returns once the running one finishes; priorities (a higher priority first, then the foreground lane, then the earliest; a Low job with its thread set to LowPriority — Qt's stored value, not the OS's — and the worker back at NormalPriority after it, D24); the receiver drops a result whose session is not loaded, is another instance of it, or whose shot is gone, drops a superseded ask, and keeps only the shot's exact latest, telling the owner of every drop but the first two so its batch settles; a job posted through the pool arrives on the receiver's thread, delivery stays first in, first out when an owner spins an event loop inside its deliver, a throwing job still posts (its error set), and a released session's running job is dropped at the receiver; the process-wide counter never repeats a number across sessions or threads. Also clean under ThreadSanitizer (-DPP_SANITIZE=thread)
src/Gui/tests/ TimelineLabels, chart metrics, SwingSeriesModel, ShotListModel, ReanalysisController, QML reactivity (6). Step 2.3: athlete_repository_test (AthleteStore, the Athlete and AthleteController on a temp library and a scratch INI — swing_export_developer_guide.md §15) beside athlete_session_guard_test (no switch mid-session; an edit never emits currentAthleteChanged), both from athlete_session_guard_tests.cmake, sharing athlete_test_library.h. Step 2.7b extends three: charts_traces_presenter_test (shotNumberAfterAGap: after a trashed swing the kicker reads "Shot 9 · 8 of 15", the against-the-previous label "shot 8", a shot before the gap "Shot 3 of 15"), charts_delivery_presenter_test (golfersDayNameDateType: under name-date-type the day's sibling is pooled and named "session 1" by its n-of-day; the kicker and the shape marks carry the swing number; golfersDay checks the scatter's sessionLabel and shotNo) and report_lm_test (section I: the launch monitor's rows, left-out list and notes by swing number after a gap).
src/IMU/tests/ Impact detector, ImuIoWorker, ESKF gyro-unit pin (3)
src/LaunchMonitor/tests/ LastShot.CSV parsing (columns by name not position, metric and imperial units as FSX2020 declares them, closure rate in dps or rpm, the derived values, malformed/torn/short input), shot attribution, and the standalone gate exhaustive over all 2^7 precondition states (3, GoogleTest; the pairing test writes its own fixtures into a QTemporaryDir and needs Qt6 Test)
src/Pose/tests/ v2 temporal ball tracker + its live/offline parity, BallDetector throttle contract, heatmap decode, pose-model selection (5)
src/Audio/tests/ Acoustic onset detector (1)
src/Diagnostics/tests/, src/Export/tests/, src/Metrics/tests/, src/Identity/tests/, src/DomainModel/tests/ Sources only — no suite of their own. Compiled by the Analysis suite (counted in its 84) because they link swing_analysis.h / the pack types. See below. The last two (refactor step 2.1): identity_test — AthleteId, SessionId, ShotRef (src/Identity/identity.h): each reads back the text it writes and nothing else (no folder name or path reads as one; no constructor from text, by static_assert), hashing, order, a QVariant round trip — and session_identity_test — SessionIdentity: a legacy folder name read under every naming pattern against the names SwingPaths composes, D3's name-based id against golden values from Python's uuid.uuid5, n-of-day by start time (by folder name when a day lacks start times — the app's order, not the per-type _NN: step 2.7a's N12, N13), the local day in a given time zone, the label. Step 2.2 adds session_repository_test (src/Export/tests/) — SessionStore, the adapter of SessionRepository: session.json's keys, a newer schema read-only, resume (today's session of the same type, same SessionId), End and the empty discard, a legacy folder read without a write, enumerate, one in-progress session per athlete, the allocator never reissuing a number, D7's library change and a held FolderLock refusing the write; and (step 2.8b, section J, J-18) a crash-left session ended by the next Start gets the latest of its swings' capture ends, marked endInferred, no end when none records a capture time, and an End is never marked; and (section G, D11: DEC-158) SessionStore::fileSaysInProgress, the tools' check: a session being recorded is in progress, an ended one, a legacy one, an unreadable file and no folder are not, and the check writes nothing. swing_paths_test's allocator sections (N, A, R, B, E, K) moved into it with the allocator; swing_paths_test keeps the names, sanitise and the trash. Step 2.5 adds golfer_input_test (src/Diagnostics/tests/, audit H-04's missing test, on the diagnostics model's link line): a focus, a run, a screen answer, a miss, a shot intent and a correction live in session_input.json and the shots' reviews, and survive diagnostics.json being deleted; a legacy session (its input in diagnostics.json's older keys only) reads identically, keeps the keys byte for byte through a regrade (setRebuildCache, as regrade_ledger runs it), takes an edit in the new home and holds a cleared answer explicitly; an envelope written before step 2.5 reads as its moved copy does; a newer envelope is read-only and the golfer told; session_input.json keeps unknown keys and refuses a newer schema; the older keys survive the cache being deleted mid-activation and a catch-up activation; a club-baseline freeze read off the GUI thread gives the synchronous freeze's baseline, grades nothing before it, and a golfer command or a re-activation during it waits for it or lets it go. Step 2.7a adds shot_number_test (src/Diagnostics/tests/, the same link line): after a trashed swing and one with no document, the ledger's "shot N" — the header's label, the review badge, every readout — is the swing number, a session with no gap reads as before; under the name-date-type pattern a session still has its day (a live panel freezes its club baseline over the day before: F-06), and two sessions of one day are placed by their session files' start times, which the tendencies "as of" a session follow.
src/Update/tests/ Linux updater pure logic (version compare, arch-aware AppImage asset selection, GPG VALIDSIG parse, placeholder-key refusal), PlatformTarget arch tokens, and UpdateController state-machine + relaunch session-safety policy via a FakeUpdateBackend (3, GoogleTest; policy test needs Qt6 Qml + Test)
tools/migrate/tests/ Sources only. migrate_step2_test (refactor step 2.8a), compiled by the Analysis suite: pps_migrate_step2's engine on a fixture library (see "The step-2 migrations" in §9).

See also: the per-subsystem developer guides in this folder (e.g. shot_detector_developer_guide.md, resource_profiler_developer_guide.md) for what each suite actually asserts.