Expert report review

What errors does a reviewer find in a handwriting examiner's report?

A handwriting examiner's report can be checked item by item. NIST IR 8282r1, published in May 2021 by an expert working group sponsored by NIST and the National Institute of Justice, lists 17 kinds of information the case record and report should contain, among them the inventory of documents, the propositions, the limitations and the error rates. A reviewer compares the opposing report with that list and with Rule 26(a)(2)(B) and marks what is missing or overstated.

A hand holds a red pencil over a blurred report page beside a face-down signature card

What should a handwriting examiner's report contain?

Under Recommendation 3.2 of NIST IR 8282r1, 17 kinds of information, including the documents examined, the propositions, assumptions, methods, observations, conclusions, limitations, error rates and any review of the conclusions.

NIST IR 8282r1, Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach, is dated May 2021 and was written by the Expert Working Group for Human Factors in Handwriting Examination. Recommendation 3.1 calls for a report for every handwriting examination unless a documented laboratory policy exempts it. Recommendation 3.2 lists what the case record must contain and asks that written reports detail all relevant analyses and comparisons accurately and clearly.

The items can be read in four groups. The first identifies the case: the submitter, the examiners and the laboratory, the request for each document, an inventory of the documents with their condition, and a curriculum vitae for each examiner. The second records what framed the work: the background information given to the examiner, the propositions used and the assumptions made, each with a note that the opinion may change if they change.

The third group describes the work itself: the instruments and methods, with the range of possible conclusions and the definitions of terms; the procedures for each document and any deviation from standard methods; the observations, with bench notes; and the evaluation of those observations under each proposition. The fourth states the result and its limits: the conclusions, the limitations of the examination, potential sources of error with relevant error rates or a statement that none is known, the charts and other data generated by the examination, whether the examiner disagreed with a reviewer, and any other statements required by the accreditation body or the laboratory.

In a federal civil case a retained examiner's report must also meet Rule 26(a)(2)(B), which requires "a complete statement of all opinions the witness will express and the basis and reasons for them" and "the facts or data considered by the witness in forming them."

Does the report say exactly which documents were examined and in what condition?

It should. The inventory in Recommendation 3.2 describes each submitted document, its condition and how it is identified, and the limitations item records what the documents allowed the examiner to do.

The inventory answers the first question in any review: was the questioned signature examined on the original, on a photocopy, on a fax or on a scanned image. A report that refers to the signature on a contract when the examiner received a copy of the contract describes an object the examiner never saw.

The limitations item is where the consequence belongs: the report should say which observations the copy prevented. The reviewer checks that the conclusion does not rely on a feature the copy could not show.

The same check applies to the known writing. Each known signature or sample should be listed with its document, date and form, original or copy, so that the reader can see which comparisons rest on originals and which do not.

Is the known writing identified and its source stated?

The report should list each known sample and state the assumptions made about it, such as its genuineness. NIST IR 8282r1 requires assumptions and their basis to be recorded, with a note that the opinion may change if they change.

If a sample was supplied by one party and its genuineness was assumed, Recommendation 3.2 asks the examiner to record that assumption and its basis. A report that lists known signatures without saying who supplied them, when they were written and why they are taken as genuine leaves the reviewer unable to test the starting point of the comparison.

In federal civil cases the source of the samples can be obtained from counsel's side as well. Rule 26(b)(4)(C) protects most communications between a party's attorney and its expert, but not those that "identify facts or data that the party's attorney provided and that the expert considered in forming the opinions to be expressed" or "identify assumptions that the party's attorney provided and that the expert relied on."

The reviewer lists each sample, its date relative to the questioned writing, its type (signature, cursive text, hand printing), and whether it was collected for the case. Gaps in date or type go into the review as limits on the comparison.

Does the conclusion rest on stated propositions?

Recommendation 2.5 of NIST IR 8282r1 calls for examinations based on at least two mutually exclusive propositions, stated in the conclusions, the reports and the testimony.

A proposition is a statement about who wrote the questioned item, such as that the named person signed it, or that someone else did. Recommendation 2.5 asks the examiner to evaluate the evidence under at least two such propositions and to state them. Recommendation 3.2 puts the propositions into the case record with a note that the opinion may change if they change.

A report that states only one proposition, for example that the signature is not genuine, does not show what the observations were weighed against. The reviewer asks which alternative the examiner considered: a genuine signature written under unusual conditions, a simulation, a tracing or a disguised signature by the named writer.

The evaluation item of Recommendation 3.2 then asks for the interpretation of the combined observations under each proposition. A report that lists similarities and differences and moves straight to a conclusion has skipped that step.

Is the conclusion stated more strongly than the method supports?

Recommendation 2.3 bars stating or implying that a specific person, to the exclusion of all others, wrote the questioned writing. In United States v. Hidalgo (D. Ariz. 2002) the court excluded the examiner's ultimate opinion on authorship.

Recommendation 2.3 of NIST IR 8282r1 says that forensic document examiners must not state or imply that a specific person, to the exclusion of all others, wrote the questioned writing. A report that says the named person "and no one else" wrote the signature, or that the match is certain, goes past that line.

The committee note to the 2023 amendment of Federal Rule of Evidence 702 adds a legal test: "each expert opinion must stay within the bounds of what can be concluded from a reliable application of the expert's basis and methodology."

United States v. Hidalgo, 229 F. Supp. 2d 961, was decided by Judge Martone of the District of Arizona on November 6, 2002. The government's examiner, William J. Flynn, was to testify that one defendant wrote most of the entries on a loan application and probably wrote a W-2 form. The court granted the motion to exclude opinion testimony that a defendant wrote the questioned writing and allowed the examiner to describe handwriting characteristics, his method and the similarities and dissimilarities he found. "It would then be left to the jury to make the ultimate finding of identity or non-identity," the court wrote.

Courts differ on this point, and Hidalgo is one district court's ruling. The reviewer records the exact wording of the opposing conclusion, the scale the report says it uses and the definition the report gives for each term on it.

What if the report reaches no conclusion or an inconclusive one?

The report should give the reasons. Recommendation 3.2 asks for reasons when no conclusion is reached, and Recommendation 4.6 asks quality control to track inconclusive and insufficient opinions.

An inconclusive result is a result. The conclusions item of Recommendation 3.2 calls for a full statement, with associations qualified, exclusions stated and reasons given if no conclusion is reached. A report that says only that no opinion could be formed leaves the reader unable to tell whether the obstacle was the questioned item, the known writing or the method.

The reasons matter to the next step in the case. If the obstacle was a copy, the original may be obtainable. If it was too few known samples or samples from the wrong period, more can be collected. If the report gives no reason, the reviewer lists the possible obstacles and the materials that would remove each.

Recommendation 4.6 asks that quality control track inconclusive and insufficient opinions and that test materials include those categories. A laboratory that follows it can say how often its examiners reach no conclusion, and the question can be put to the opposing examiner.

Does the report state its limitations and error rates?

It should state the limitations of the examination and the potential sources of error with relevant error rates, or say that none is known. Recommendation 3.5 adds that examiners should never state that errors are impossible.

Recommendation 3.2 lists limitations and error rates as separate items. Recommendation 3.5 asks examiners to be prepared to describe the steps taken to reduce the risk of error and never to state that errors are impossible. Recommendation 3.6 asks them to know the underlying science and the reported error rates and to be able to explain them.

Published studies give the reviewer figures to compare with. In Hidalgo the court discussed proficiency studies by Professor Kam that involved more than one hundred forensic document examiners and forty-one non-professionals, and noted that "the false positive rate for professionals was 6.5% compared to 38.3% for non-professionals."

A report with no limitations section, or one that describes the method as infallible, does not meet Recommendations 3.2 and 3.5. The reviewer quotes the passage and lists the error rate studies that the examiner should have addressed.

Was the conclusion reviewed, and what was the examiner told about the case?

The report should say whether a second examiner reviewed the conclusion and whether the examiner disagreed with the reviewer. The case record should show the background information given to the examiner, under a documented protocol for managing it.

The review item in Recommendation 3.2 records whether the examiner disagreed with a reviewer, if a review took place. A report that is silent on review leaves open whether anyone checked the work. Recommendation 3.3 adds that the examiner who did the examination and wrote the report should be the one to testify.

Recommendation 2.2 says that laboratories doing handwriting examinations must use a documented protocol for managing contextual information as part of their quality management system. The background information item of Recommendation 3.2 puts what the examiner was told into the case record.

The reviewer compares the background information with the conclusion and records statements in the request that go beyond the documents to be compared, such as an accusation that the signature is forged or a summary of the other evidence in the case.

What does IAE do when the opposing handwriting report needs a review?

IAE prepares a review of the opposing handwriting examiner's report, or a report on the client's questions, for federal and state cases. The document is issued in up to 5 business days after the format and cost are confirmed.

The IAE handwriting page covers whether the person named on a will, contract or check signed it and whether the signature shows signs of simulation or tracing. A review checks whether the report states each opinion with its basis and the facts or data considered, as Rule 26(a)(2)(B) requires, and whether the examiner followed ANSI/ASB Standard 070 on originals, comparability, distortion and alternative explanations. Absent a stipulation or court order, a rebuttal report is due within 30 days after the other side's disclosure under Rule 26(a)(2)(D)(ii).

The client sends the opposing report and the case materials and describes the task. The secretariat confirms receipt the same day. Within one business day IAE confirms the format, the cost, the deadline and the specialist from its registry, and the client approves the candidate and the CV before work starts. The draft is checked for facts, and the signed PDF is issued within up to 5 business days after the format and cost are confirmed. Cases with tight deadlines have a 48-hour rush.

Registry members passed the council's collegial attestation under the published Criteria Matrix. The Member ID in the document opens the specialist's category, specialization, attestation date and record status on the status check page or by QR code, and the specialist confirms in the document that he or she has no interest in the outcome of the case.

Materials are sent through the form on the contact page or by email and are not passed to third parties without the client's consent. A review is evidence that the court weighs together with the other materials, and neither IAE nor the reviewer guarantees the outcome. IAE is not a law firm and does not advise on case strategy.

Sources

Read also

All articles

Expert report review

Ready to discuss your case?

Send the materials: the secretariat will confirm the format, cost, deadline and the specialist within one business day.

Expertise request

Received

Cost and timeline estimate

WorkReview of an opinion
TimelineUp to 5 business days
ExpertIAE registry member