Impact and pilot measures

MapToc's impact thesis is that a reviewed evidence record can reduce repeated reconstruction while improving the quality of the program history available to teams and evaluators. A pilot must test both sides of that claim.

Measure the whole workload

The relevant comparison includes source contribution, AI review, correction, retrieval and final report preparation. Faster drafting is not a net saving if maintaining the archive takes more effort than it removes. Teams should record the work that shifts between roles as well as the total time.

Check traceability and usefulness

Review a sample of approved items. Can a reader find the source? Does the source support the wording? Are uncertainty and contradictory evidence visible? Test these questions separately from the number of records captured.

Test handover with real questions

A new team member or evaluator can be asked to explain a consequential decision using the available record. Observe what they can establish, where they still need interviews and what remains unclear. A richer archive should not create false confidence about missing evidence.

Look for learning that reaches a decision

Track whether a lesson is consulted when a team revises delivery or designs a subsequent program. The existence of a lesson entry is not the same as evidence that it was useful or improved an outcome.

Interpret pilot results carefully

Before starting, agree the comparison tasks, baseline, participating roles and criteria for a useful result. Small pilots can identify usability problems and promising signals. They do not establish sector-wide savings or prove that MapToc improves development outcomes. Any later public claim should explain the sample, comparison and limits behind it.

Test institutional decisions, not record volume

In addition to workload and traceability, test whether reviewers can explain a material budget change, identify an unsupported outcome claim and find unresolved issues before a program review. Count relevant actions taken, not just alerts generated.

Compare the proposed workflow with the existing reporting process using agreed tasks and a baseline. Separate time saved drafting from time spent checking AI proposals. Better documentation does not by itself demonstrate better development outcomes.