Research question
AI-generated summaries can change the meaning of a source by removing uncertainty. The central methodological concern is whether an observation has been transformed into a stronger claim.
Source and findings
FRAME distinguishes the description of modifications from investigation of their effects. This distinction is also relevant when evaluating summaries of implementation records. Wiltsey Stirman, Baumann & Miller (2019), FRAME
Interpretation
A source quotation supports traceability, not truth. AI summaries must preserve uncertainty and distinguish observation from causal interpretation.
Assessing a summary
A useful appraisal separates three questions. First, does the cited passage exist in the stated document? Second, does the summary preserve what that passage says, including uncertainty and attribution? Third, is the original evidence sufficient for the intended inference? A summary can pass the first test and fail the others. For example, a field observation that attendance “seems” higher is not equivalent to a measured change in attendance.
For research use, an assessment protocol could compare generated summaries with independent human judgements on omissions, altered meaning and unsupported conclusions. Error rates should be reported by claim type, document quality and language. This is a proposed appraisal approach, not a validated scoring method. Its purpose is to make the inferential steps visible rather than replace substantive judgement.
Limits of the evidence
This is a methodological application of reporting principles, not an empirical test of an AI model.