Chapter 69 Counting Discipline

At v0.1.0-alpha.1 @ 1bb2145d, static inspection gives the following representative counts:

These figures describe different populations. Adding them is meaningless, and selecting the largest does not make it the most authoritative. A test source can define many cases; a parameterized definition can expand at runtime; one executable can become thousands of CTest entries; one standalone executable can exist without any CTest registration.

The source population is a sorted union, not a sum of unchecked directory totals:

{ find modules -path ’modules/*/tests/*’ -name ’*.cpp’ -type f
  find tests -name ’*.cpp’ -type f; } | sort -u | wc -l

Macro counts search only declaration starts, distinguishing TEST, TEST_F, and TEST_P. This is still a source-definition count, not a run count. Nine INSTANTIATE_TEST_SUITE_P sites and platform conditionals can change runtime expansion.

Renderer registrations require another derivation. A lexical search finds 1,720 direct helper call sites under renderer CMakeLists.txt files. Helper definitions, wrapper expansion and nested CMake conditions mean that number is deliberately described as call sites, not registered or executed tests.

Singular builds enter one selected family; multi-renderer builds enter only the explicitly named, compatible set. Platform gates and incompatible native dependency graphs prevent a universal all-family configuration. Summing every family’s registration sites therefore creates a number that no supported build exposes as one CTest suite.

CTest discovery adds a second barrier. PRE_TEST obtains case names by running CnaTests --gtest_list_tests; configure-time source inspection cannot know the complete expanded list. Build features, conditional compilation, and parameter instantiation all affect it. The authoritative count for a particular build must come from that built artifact and must name its renderer and feature set.

Not every disputed number is inherently unknowable. The canonical cache value table contains 50 public renderer identities, while modules/renderers contains 46 implementation families after excluding common infrastructure. Four identities share implementations with other public modes. Thus “50 identities, 46 families” resolves the apparent conflict instead of choosing one number and calling the other wrong.

This pattern is widely useful: define the noun before reconciling the count. An identity is a user-selectable contract; a family is a source implementation. They need not be one-to-one.

Static inspection cannot establish:

• •

  the CTest count for a configured and built tree;

• •

  how many parameterized GoogleTests expand or actually run;

• •

  active standalone registrations after nested conditions;

• •

  pass, failure, timing, or skip totals;

• •

  whether any pixel assertion reached a display;

• •

  image-oracle divergence counts;

• •

  line, branch, or function coverage—no coverage tool is configured.

CNA documents several other kinds of “coverage”: API-surface estimates, mappings from source files to suites, and feature checklists. None is measured code coverage. A percentage without that qualifier is more misleading than no percentage at all.

A durable result can fit in one sentence if every noun is present: “On commit X, the Y renderer build with features Z discovered N CTest entries; it ran R, skipped S, and failed F under host H.” A source audit uses a different sentence: “At commit X, this command found N declarations matching definition D.”

Three historical comments claimed approximately 247, 330, and 600 renderer registrations, each using a different unstated meaning; the tag tree contains 1,720 direct helper call sites across configuration-dependent families. Comments rot, while reproducible definitions can be rerun. The discipline is therefore not to avoid numbers, but to make every number carry its derivation.