Chapter 68 The CnaTests Architecture

cmake/UnitTests.cmake recursively globs C++ test sources from three roots: non-renderer modules, renderer modules, and the repository-level tests directory. The result becomes exactly one executable, CnaTests. There is no GoogleTest executable per framework module.

The broad glob is narrowed by nine explicit list(FILTER ... EXCLUDE REGEX) operations. They remove standalone Glide ABI programs and minimal-link probes, conditionally remove five FFmpeg video tests, remove Net and GamerServices trees when networking is disabled, exclude POSIX-process tests on Windows, Emscripten, and Android, and select only the configured renderer-specific test directory. One exclusion group even checks itself with a fatal error if Net sources survived a Net-disabled configure. Filtering is therefore part of the build contract, not housekeeping.

GoogleTest cases are exposed to CTest with gtest_discover_tests(... DISCOVERY_MODE PRE_TEST). Discovery runs the built binary at test time, so the complete CTest list does not exist at configure time. Its working directory is pinned to the source root after cases failed to find tests/assets when CTest launched them from the runtime-output directory. Both details explain why a source-only count and a ctest -N count answer different questions.

The ordinary configuration enters one chosen renderer family. Alpha.1 also accepts a compatible semicolon-separated CNA_GRAPHICS_RENDERERS set; the selected singular identity remains the default. Renderer-local source filtering tests membership in CNA_RENDERER_IDENTITIES, so a multi-renderer CnaTests build retains the suites for every compiled family rather than only the default. Metal and Glide still contribute some portable policy suites even when their native implementation is gated. No supported tuple therefore implies “all renderers,” and a result that omits both the compiled set and the active runtime identity cannot be interpreted safely.

There are three layers of renderer gating:

1. 1.

   source filtering decides which compiled renderer families’ GoogleTests enter CnaTests;

2. 2.

   CMake conditions decide which standalone executables and CTests are registered;

3. 3.

   runtime pre-flight may return the project-wide skip code 77 when no display is usable.

That last result is deliberately SKIPPED rather than FAILED. It keeps headless machines from turning absence of a display into a renderer defect, but it also means a green CTest summary may contain no executed pixel comparison. Reports must retain skip counts and the display setup.

Files named *_test.cpp under renderer examples usually do not contain GoogleTest at all. Per-renderer CMake macros build them one executable at a time, each with its own main(), then optionally register it with CTest. Many consume the same public fixture from modules/graphics/examples, allowing one renderer to expose a defect while the others serve as controls.

Building and registering are independent acts. Some renderers build far more executables than they register; others add Python audit registrations that are not executables. Labels are also uneven: GraphicsSmoke is cross-renderer, while several large renderer families label only one or two of hundreds of registrations. A command such as ctest -L EasyGL therefore cannot be assumed to select the EasyGL corpus.

Several failures concern process-global state: SDL shutdown, the audio mixer’s global owner, GamerServices dispatch, and network peers. CNA builds small helper executables for these cases and embeds their absolute paths into CnaTests. A GoogleTest case spawns the helper and judges its result, gaining a clean address space and making crashes or shutdown ordering observable without poisoning later cases.

Other helpers are diagnostic tools rather than isolation tests: an XNB audio metadata dumper, XWB inspector, FNA-reference dump, and Devices microbenchmark. Their presence beside harnesses does not make them part of every test run. The governing question is always who invokes the target and under what configuration.

cna_strict_xna_api_leak_check is excluded from the normal build. Its CTest command invokes the build system for that target and is marked WILL_FAIL TRUE. The test passes only when the compiler rejects the forbidden API use. This inverted polarity turns a namespace and surface rule into executable evidence: successful compilation would be the regression.

Fourteen module probes apply the complementary idea to linking. Each builds a minimal consumer of one module alias, then a link-closure test inspects the generated link.txt against a module-specific forbidden-library expression. The probe asks whether the public target works; the closure gate asks whether it works for the right architectural reason.

A reliable test report separates four stages:

Discovery

Which sources and runtime-expanded GoogleTest cases were present?

Selection

Which filter, label, renderer, and feature options chose the cases?

Execution

Did the intended executable, display, GPU, browser, or compatibility layer actually engage?

Verdict

Which oracle declared pass, fail, or skip?

This model explains most apparent contradictions in CNA’s historical test prose. A source can exist but be filtered out; a target can build but not be registered; a CTest can register but skip; a program can pass while running through the wrong graphics translation layer. The rest of this Part examines the numbers and oracles that make those stages auditable.