Claude Certified Associate, Foundations (CCAO-F)

Claude summarizes a market report and states that product revenue increased by 18%. What should an analyst do before using that figure in an executive presentation?

1. A. Accept it because the response sounds confident.
2. B. Ask Claude to repeat the figure.
3. C. Verify it against the source and confirm the period and definition of revenue.
4. D. Replace it with the phrase “increased significantly.”

Correct answer: C

A material numerical claim should be traced to its source, including the applicable time period and metric definition. Tone and repetition do not establish accuracy.

A legal team uses Claude to extract renewal dates from contracts. Select TWO methods that provide the strongest evidence of reliability.

Select 2 answers that apply.

1. A. Compare results with a human-verified test set containing varied contract formats.
2. B. Approve the workflow after one correct result.
3. C. Track extraction accuracy and analyze failures by contract type.
4. D. Judge quality mainly by how polished the table looks.

Correct answer: A, C

A representative verified test set measures performance, while segmented error analysis exposes systematic weaknesses and edge cases.

Claude proposes three cost-saving ideas. Two are supported by internal data; one relies on an unstated assumption. What is the best action?

1. A. Keep all three for completeness.
2. B. Remove or label the unsupported idea, investigate the assumption, and retain traceable evidence for supported claims.
3. C. Make the third idea sound more certain.
4. D. Treat all ideas as equally reliable.

Correct answer: B

Each material recommendation should be evaluated against its evidence. Unsupported assumptions must be surfaced and investigated.

A summary is factually accurate but omits a major risk described in the source. How should the output be evaluated?

1. A. Correct because every included fact is true.
2. B. Acceptable if it is concise.
3. C. Incomplete because evaluation includes coverage of material information, not just factual correctness.
4. D. Reliable because no fact was invented.

Correct answer: C

Validation should assess completeness and relevance as well as factual accuracy. A material omission can mislead decision-makers.

Claude provides citations for a research brief. What is the most important next step?

1. A. Assume a citation proves every sentence in its paragraph.
2. B. Open the cited sources and confirm they exist, support the claims, and are sufficiently authoritative.
3. C. Count the citations.
4. D. Replace all citations with one general link.

Correct answer: B

Citation presence is not enough. The reviewer must validate existence, entailment, and source quality.

A team evaluates a classifier only on simple, well-written customer messages. What is the main weakness?

1. A. The test set is too representative.
2. B. It may not reveal performance on typos, ambiguity, mixed languages, rare classes, or other real-world cases.
3. C. The evaluation contains too many edge cases.
4. D. The results will always underestimate performance.

Correct answer: B

Evaluation data should reflect realistic variation and important edge cases; otherwise reported quality may not generalize.

Select TWO. Which checks are most useful when Claude transforms a spreadsheet into an executive summary?

Select 2 answers that apply.

1. A. Recalculate a sample of important figures from the original data.
2. B. Check that conclusions distinguish evidence from inference.
3. C. Accept totals if formatting is polished.
4. D. Validate only the spelling of headings.

Correct answer: A, B

Independent calculation catches numerical errors, while separating evidence from inference prevents unsupported conclusions.

Claude’s answer matches the provided document, but the document itself is obsolete. What does this show?

1. A. Grounded output is always correct.
2. B. Validation must include source currency and authority, not merely faithfulness to the supplied text.
3. C. The prompt needs a warmer tone.
4. D. The answer should be longer.

Correct answer: B

Faithfulness to a bad source can still produce a bad decision. Source quality, currency, and authority are part of validation.

Which evaluation finding most clearly indicates a fairness concern?

1. A. The output is two paragraphs longer than requested.
2. B. Error rates are materially higher for one relevant user group under comparable conditions.
3. C. The response uses bullet points.
4. D. The model takes longer on large files.

Correct answer: B

A systematic quality disparity between groups can signal a fairness issue and warrants investigation into data, design, and impact.

A team changes its prompt and average accuracy rises, but performance on critical high-risk cases falls. What is the correct conclusion?

1. A. The new prompt is better because the average increased.
2. B. The change needs further review because aggregate improvement can hide unacceptable regression in critical segments.
3. C. Critical cases should be removed from testing.
4. D. Average accuracy is the only relevant measure.

Correct answer: B

Evaluation should reflect business impact and segment-level performance, especially where errors carry disproportionate risk.

A generated comparison favors Vendor A. How should a reviewer check whether the conclusion is robust?

1. A. Ask Claude whether it is sure.
2. B. Inspect the criteria, source evidence, missing data, weighting assumptions, and whether the same method was applied to every vendor.
3. C. Accept it if Vendor A is well known.
4. D. Check only grammar.

Correct answer: B

A robust comparison requires consistent criteria, traceable evidence, transparent assumptions, and attention to missing data.

Select TWO. A useful evaluation rubric for a client-facing summary should include which dimensions?

Select 2 answers that apply.

1. A. Factual accuracy and source support.
2. B. Coverage of material points and suitability for the intended audience.
3. C. Whether the response used the maximum possible length.
4. D. Whether every sentence contains an adjective.

Correct answer: A, B

Accuracy, traceability, completeness, relevance, and audience fit are meaningful quality dimensions; arbitrary stylistic counts are not.

Why should evaluation examples include cases where the correct action is to say there is insufficient information?

1. A. To make the test easier.
2. B. To test whether Claude can avoid unsupported answers when evidence is incomplete.
3. C. To reduce the number of source documents.
4. D. To encourage longer responses.

Correct answer: B

Reliable systems should recognize evidence gaps rather than fabricate a confident conclusion.

A response is accurate, complete, and well sourced but violates the requested format required by a downstream system. Is it acceptable?

1. A. Yes, content is the only quality dimension.
2. B. No; format compliance is part of task correctness when downstream use depends on it.
3. C. Yes, if the prose is elegant.
4. D. No, because every answer must be a table.

Correct answer: B

Correctness includes explicit task constraints. A structurally invalid result can break the workflow even when its content is sound.