Grounded Answer Citation Audit

Grounded Answer Citation Audit

Audience: knowledge base owners, RAG prompt reviewers, and support ops
Confidence: High

Review Scope

Question reviewed: “Can we reduce onboarding time by 30% after switching to the new help center flow?”
Answer under review: A drafted response that cited three internal sources and presented a numeric claim, a causal claim, and an operational recommendation.

Verdict

The answer is not fully grounded as written.

• The cited sources support that the new help center flow was launched, that article completion improved, and that agents reported fewer repeat contacts.
• The sources do not support the specific claim that onboarding time was reduced by 30%.
• The sources also do not isolate the help center flow as the sole cause of the improvement.
• The recommendation can be retained only if rewritten as a hypothesis or removed if the goal is a factual summary.

Evidence Summary

Source-by-source check

Evidence-to-claim mapping

Assumptions

• The cited dashboard export is assumed to be the same data source used in the answer draft.
• “Onboarding time” is assumed to mean time-to-complete onboarding tasks, but the answer does not define the metric.
• The review assumes no additional, uncited measurement study was available.
• No evidence was provided that any external communication, production change, or approval step occurred beyond the cited materials.

Risks and Gaps

Recommendation

Revise the answer before reuse. Use only claims directly supported by the cited evidence, and downgrade unsupported numerical or causal statements to cautious language.

Safer rewrite pattern

• Replace: “Onboarding time dropped by 30%.”

• With: “The available sources show improved completion and fewer repeat contacts after launch, but they do not prove a 30% onboarding-time reduction.”

• Replace: “The new flow caused the improvement.”

• With: “The improvement is consistent with the rollout, but the current evidence does not isolate causation.”

• Replace: “Roll this out to all teams immediately.”

• With: “Consider broader rollout after validating the metric definition, measurement window, and causality.”

Action Plan

1. Define the metric

   • Confirm what “onboarding time” means.
   • Specify the measurement window and baseline.

2. Validate the calculation

   • Recompute any percent change from the source data.
   • Verify whether 30% is actually supported.

3. Separate correlation from causation

   • Check whether other onboarding changes occurred during the same period.
   • Identify confounding factors.

4. Add citation coverage

   • Attach the exact source line or chart for each claim.
   • Cite the baseline and post-launch comparison explicitly.

5. Rewrite the answer

   • Keep supported operational observations.
   • Remove unsupported precision and definitive causal wording.

6. Escalate only if evidence is sufficient

   • If the response is used for a regulated or customer-facing decision, require human review with the metric definition and source export attached.

Open Questions

• What exact definition of “onboarding time” is being used?
• What time period is the 30% figure based on?
• Is the analysis comparing equivalent cohorts before and after launch?
• Were any other onboarding process changes deployed at the same time?
• Can the dashboard export be traced back to a reproducible query?
• Is the recommendation intended as a factual summary or a decision proposal?

Final Assessment

Groundedness score: 2.5 / 5

Bottom line: The answer is usable only after removing the unsupported 30% metric and replacing causal language with evidence-bounded wording. The cited sources support an improvement trend, but not the specific performance claim or rollout recommendation as written.

This sample illustrates a review-ready grounded-answer citation audit using a realistic evidence check, separation of evidence and assumptions, and operator-facing next steps; it does not imply any external actions were taken or any regulated decision was approved.

This sample illustrates the skill's output format. Names, metrics, and operational details are illustrative unless the artifact explicitly analyzes public information.