Concept
Kumho Tire and Nonscientific Experts
Reliability and relevancy
Kumho Tire and Nonscientific Experts
Kumho Tire Co. v. Carmichael (1999) finished what Daubert and Joiner began. It held that the judge’s gatekeeping duty reaches all expert testimony based on technical or other specialized knowledge, not only testimony that counts as science, and that the judge has broad latitude to decide how to test reliability in the case at hand.1 Rule 702 now says the same in its text. Amended in 2000 to capture the approach of Daubert and Kumho Tire, the rule admits “scientific, technical, or other specialized knowledge” only if the testimony is the product of reliable principles and methods and the expert has reliably applied them to the facts of the case. Neither condition carries a science-only qualifier.2
On July 6, 1993, the right rear tire of a minivan driven by Patrick Carmichael blew out. One passenger died and others were severely injured. The Carmichaels sued the tire’s maker and distributor, together called Kumho Tire, in a diversity action claiming that the tire was defective, a products liability claim of the kind the Restatement (Third) of Torts: Products Liability surveys.3 They rested their case in significant part on the deposition of Dennis Carlson, Jr., an expert in tire failure analysis who intended to testify in support of their conclusion.4
Carlson reasoned by elimination. He inspected the tire by sight and touch for four signs of abuse: proportionately greater tread wear on the shoulder, grooves left by the beads, discolored sidewalls, and marks on the rim flange. When fewer than two of the four appeared, he concluded that a defect had caused the tread to separate from the tire’s steel-belted carcass.5 This tire showed some of each sign to a limited degree, along with inadequately filled puncture holes, yet Carlson testified that none of the symptoms was significant. The extra shoulder wear, he said, appeared mainly on one shoulder, while an overdeflected tire would show equally abnormal wear on both.6 His conclusion therefore rested on an unstated premise: that inspection by sight and touch could show the tire had not been abused even though it displayed some of the very signs he looked for.7
The district court excluded the testimony. It first rejected the plaintiffs’ claim that Carlson was exempt from Daubert-style scrutiny because his analysis was technical and not scientific, and it simply found that none of Daubert’s four criteria was satisfied. On reconsideration it recognized that the reliability inquiry should be flexible and that Daubert was meant neither to be exhaustive nor to apply in every case, but it still excluded Carlson, who had failed to satisfy either Daubert’s factors or any other set of reasonable reliability criteria.8 Despite his qualifications, the court doubted the methodology he used to analyze the data from his inspection.9 The Eleventh Circuit held that a trial judge may ask Daubert’s questions only of an expert who relies on the application of scientific principles, and not of one who relies on skill- or experience-based observation. The Supreme Court disagreed and reversed, finding no abuse of the district court’s discretion.10
The Court’s reasoning answers four questions, one for each subsection below: whether the gatekeeping duty reaches a nonscientific expert like Carlson, how freely the judge may use Daubert’s factors, what benchmark measures a nonscientist’s reliability, and when experience alone can support an opinion.
Gatekeeping for All Experts
Kumho Tire answered the question Daubert left open. Daubert spoke of scientific knowledge because that was the nature of the expertise in the case before it, and it did not settle whether the same screen applies to engineers, mechanics, and others who are not scientists.11 The Court held that the trial judge’s gatekeeping obligation covers testimony based on technical and other specialized knowledge as well as testimony based on scientific knowledge.12 In its words, “a Daubert-style [reliability] scrutiny” is appropriate for all types of expert testimony.13
Courts and commentators had offered competing answers. The Eleventh Circuit confined the inquiry to science. A third position, which one commentator called the most sensible, held that the proponent of nonscientific testimony must demonstrate the reliability of the expert’s underlying assumptions, but that the judge need not necessarily use the Daubert factors, which derive from a classic scientific model. The judge should ask more flexible questions: whether, in the real world, members of the public routinely turn to this profession for services other than testimony, and whether a feedback loop alerts a practitioner when she has erred.14 The Court essentially endorsed that third position.15
The Court gave three reasons for rejecting a science-only duty.
The first reason is textual. Rule 702 draws no relevant distinction between scientific knowledge and technical or other specialized knowledge, and it makes clear that any of them may be the subject of expert testimony. Daubert had located the standard of evidentiary reliability in the rule’s word “knowledge,” not in the adjectives that modify it, so the standard follows whatever knowledge the witness offers.16
The second reason concerns why the screen exists at all. The Rules give experts a freedom that other witnesses lack, the freedom to offer opinions that do not rest on firsthand knowledge or observation. Daubert explained that this freedom assumes the opinion has a reliable basis in the knowledge and experience of the expert’s discipline, and the Rules grant it to all experts, not just scientific ones.17 A tire analyst who explains why a tire failed on a road he never saw uses the same freedom as a chemist explaining a reaction. Because the freedom is the same, the showing that justifies it must be the same too.
The third reason concerns the jury. Experts of all kinds tie observations to conclusions through what Judge Learned Hand called “general truths derived from . . . specialized experience.” Whether the testimony turns on specialized observations, their translation into theory, a theory itself, or its application in a particular case, it often rests “upon an experience confessedly foreign in kind to [the jury’s] own.” The judge’s effort to assure that such testimony is reliable and relevant helps the jury evaluate that experience, whether the knowledge is scientific, technical, or otherwise specialized.18 Carlson’s own method shows why a label makes a poor trigger for review: it joined a physical inspection to a rule of thumb about how many symptoms to expect, and neither part becomes more or less checkable because someone calls it science or craft.
The Court therefore concluded that Daubert’s general principles apply to every kind of expert matter Rule 702 describes. Where the factual basis, data, principles, methods, or application of an expert’s testimony are called sufficiently into question, the judge must determine whether the testimony has “a reliable basis in the knowledge and experience of [the relevant] discipline.”19 Universal gatekeeping does not mean universal factors. Daubert’s list neither necessarily nor exclusively applies to all experts or in every case, and the next subsection takes up how the judge chooses among them.20
The extension did not erase the differences among experts. Purportedly scientific experts must meet a somewhat higher standard than other experts, and experts in recognized fields that rely primarily on experience seem to need only the standards of their own profession; courts applying Daubert and Kumho Tire to routine medical practitioners do not appear to hold them quite to the standards of rigorous science.21 Nor does the extension doom experience-based testimony. In United States v. Hankey, decided after Kumho Tire, a police officer’s testimony about a gang’s code of silence was admitted because its reliability rested on years of experience and personal knowledge.22
The reach extends beyond workshops and police stations. The Daubert goal, to limit expert testimony to relevant opinions based on reliable analysis and methodology, applies beyond the laboratory, and courts have applied it to economists, a field in which disagreements are commonplace. Because expert analysis often decides antitrust outcomes, and courts may rule as a matter of law on issues once left to the trier of fact, the effects on antitrust litigation have been substantial.23 Where excluded testimony was a party’s only route to a genuine issue of material fact, exclusion can expose that party to summary judgment under Rule 56(c).24
An exam stem that stresses that the witness is a mechanic, appraiser, or police officer and not a scientist invites the science-only answer, and that answer is wrong. Rule 702 reaches technical and other specialized knowledge, so the judge screens the testimony for a reliable basis; the proponent need not satisfy every Daubert factor to pass.
Flexible Use of Factors
Once every expert must show a reliable basis, the next question is whether every expert must show it by the same measures. Kumho Tire said no. The trial judge “must have considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable,” so a trial court should consider the specific factors identified in Daubert where they are reasonable measures of the reliability of the testimony.26 Where they are not, the judge may rely on other reasonable reliability criteria.27 The list is a toolbox: the judge picks up the tools that fit and sets down the rest.
The Court grounded that flexibility in Daubert itself. Daubert described the Rule 702 inquiry as “a flexible one,” said that its factors do not constitute a “definitive checklist or test,” and tied the inquiry to the facts of the particular case. The factors may or may not be pertinent, depending on the nature of the issue, the expert’s particular expertise, and the subject of the testimony. The Court would neither rule them in nor out for all cases and all time, or for subsets of cases sorted by category of expert or kind of evidence, because too much depends on the circumstances of the case at issue.28 It likewise rejected a schematism that segregates expertise by type while mapping certain kinds of questions to certain kinds of experts.29 The Third Circuit’s approach shows the same openness: district courts there must examine all factors relevant to whether particular scientific evidence is reliable, including those in Daubert and Paoli and any other inquiry the court finds relevant.30 The Advisory Committee Note to Rule 702 adds that no single list of relevant concerns will suffice for all cases.31
Flexibility also cuts against factors that appear to be satisfied. A claim made by a scientific witness may never have been peer reviewed simply because the application at issue never interested any scientist, so the absence of review proves little. And general acceptance shows nothing about reliability where the discipline itself lacks it, as with theories grounded in generally accepted principles of astrology or necromancy.32 A judge who treats the factors as a scorecard can err in both directions, by faulting a trade for lacking journals and by crediting a discredited field for its followers.
Where a factor does fit, the judge may translate it into the field’s own terms. The Court’s example was a perfume tester able to distinguish among 140 odors at a sniff, a witness whose expertise rests purely on experience; for that witness it will at times be useful to ask whether the tester’s preparation is of a kind that others in the field would recognize as acceptable.33 The commentator’s two questions serve the same end, because a profession that serves the public outside court and learns from its own errors gives the judge an outside check on the expert’s method.34 The Advisory Committee Note lists further indicators, including whether the expert used the same care as experts use outside of court and whether the expert, even one relying solely on experience, can adequately explain how the methods lead to the conclusions.35
The first judge let a missing factor stand in for a judgment about reliability that was never made, although a repair trade may check its work in ways that leave no journal trail. The second judge looked for the checks the trade does use. Flexibility changes the yardstick without lowering the demand for proof, so the second judge may still exclude the mechanic if the answers show that the method rests on nothing beyond the mechanic’s own say-so.
The latitude extends from which measures to apply to how the judge proceeds. The law grants a district court the same broad latitude in deciding how to determine reliability that it enjoys in its ultimate reliability determination, and abuse-of-discretion review covers both.36 Without that authority, a judge could neither avoid unnecessary reliability proceedings in ordinary cases where an expert’s methods can properly be taken for granted nor require appropriate proceedings in the unusual or complex cases where cause for doubt arises. The Rules themselves seek to avoid “unjustifiable expense and delay,” so a full inquiry for every routine expert would work against their purpose.37 The judge’s tools include a hearing outside the jury’s presence, which she will often find necessary, and, when partisan experts in an important case reach completely different conclusions, a court-appointed expert under Rule 706.38
Kumho Tire itself shows proportion at work. The district court examined the transcript in some detail and considered the respondents’ defense of Carlson’s methodology before it ruled, and the Supreme Court held that the doubts that triggered the inquiry were reasonable, as was the court’s conclusion.39
The latitude has a boundary, and Justice Scalia’s concurrence marks it. The discretion the Court endorses, he wrote, is not discretion to abandon the gatekeeping function or to perform it inadequately; it is discretion to choose among reasonable means of excluding expertise that is fausse and science that is junky.40 Because the Daubert factors are not “holy writ,” the failure to apply one of them in a particular case may be unreasonable, and hence an abuse of discretion.41 The boundary runs in both directions. A judge who demands journal publication from a mechanic has chosen an unreasonable measure, and a judge who never looks at the measured false-positive rate of a laboratory technique whose reliability is disputed has skipped a pertinent one.
Flexibility carries a cost. Because the Court would not rule the factors in or out for categories of experts, two judges may choose different reasonable measures for similar testimony, and review for abuse of discretion will uphold each choice if it was reasonable.42
Exam answers go wrong in two directions. One treats Daubert’s list as a checklist every expert must pass, so that a missing publication or error rate sinks a mechanic or an appraiser, which is the schematism the Court rejected.48 The other treats flexibility as license to admit an expert on credentials or to skip the inquiry, which the concurrence forecloses.49 A sound answer names the field, selects the measures that fit it, and explains why the judge’s way of testing the expert was reasonable.
Intellectual Rigor Standard
Rule 702 requires reliable principles and methods and their reliable application, but its text does not say what reliability looks like for an expert whose field has no laboratory and no journals.51 Kumho Tire supplied a benchmark drawn from the expert’s own working life. The gatekeeping requirement exists to ensure the reliability and relevancy of expert testimony, and the Court described its objective as making certain that an expert, “whether basing testimony upon professional studies or personal experience, employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.”52 The Advisory Committee Note to the 2000 amendment lists the same idea among its considerations: whether the expert used the same care in developing the opinion as experts use outside of court.53
The benchmark suits a judge who is not an expert. The trial judge must often evaluate reliability in a field in which she has no training, and polling the relevant community for general acceptance is no longer enough.54 The rigor comparison lets her do the job without deciding whether the field itself is right. She asks only whether this witness did here what the profession ordinarily requires. The yardstick comes from the expert’s own field, so it fits an engineer or a mechanic whose practice produces no journal articles, and it measures what the expert did in this case, which a résumé cannot show.
The Court applied the benchmark to a specific opinion, not to a field. Carlson’s supporters stressed that qualified experts can often identify tire abuse by visual and tactile inspection and that an expert may draw conclusions from extensive and specialized experience. The Court accepted both points. It added that the question before the trial court was specific, not general: whether this expert had sufficient specialized knowledge to assist the jurors “in deciding the particular issues in the case.”55 The issue was not tire inspection in general. It was whether to rely on visual and tactile inspection, combined with Carlson’s particular way of analyzing what the inspection revealed, to conclude that a defect had caused this tire’s tread to separate from its carcass.56 A field’s general respectability cannot vouch for every use an expert makes of its techniques; the proponent must connect the particular method to the particular conclusion.
The record showed where the connection failed. On bead grooving, Carlson testified that most tires have some groove pattern, that where he suspected an abnormal one he would ideally look at a lot of similar tires to know its significance, and that he had not looked at many tires like this one, a concession that his simple visual-inspection method did not suffice for that sign.57 His signed report before the first deposition had reached the no-abuse conclusion for a different reason, that the rim flange impressions were normal, and not because two of four signs were absent. It also reported a remaining tread depth of 3/32 inch, while the opposing expert’s apparently undisputed measurements put the depth anywhere from .5/32 to 4/32 inch, with greater wear along both shoulders than along the center.58 That last finding sits uneasily with Carlson’s explanation of the shoulder wear, since he had said that an overdeflected tire would show abnormal wear on both shoulders and that the extra wear on this tire appeared mainly on one.59
Nothing outside Carlson’s own testimony filled those gaps. The Court found no indication that other experts in the industry use his two-factor test or that tire experts normally make the very fine distinctions, such as the symmetry of comparatively greater shoulder wear, that his theory required, and no one pointed to an article or paper validating his approach.60 No one argued that Carlson, were he still working for Michelin, would have told his employer in a report that a similar tire was similarly defective on the same grounds.61 That last observation is the benchmark in pure form: would the courtroom opinion survive the expert’s own professional standards? What remained was Carlson’s claim that his method was accurate, and on that claim Kumho Tire returned to Joiner. Nothing in Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence connected to existing data only by the ipse dixit of the expert, and a court may find too great an analytical gap between the data and the opinion.62 Ipse dixit is Latin for an assertion that rests on the speaker’s say-so alone.
The rigor comparison and the ipse dixit bar work as a pair. The comparison asks what a careful professional would have done and what corroborates the expert, and the bar excludes the opinion when nothing does. Corroboration need not come from the lab. Where a proponent relies on its own experts, one court required them to explain precisely how they reached their conclusions and to point to some objective source, such as a learned treatise, a professional association’s policy statement, or a published article, showing that they followed the method as practiced by at least a recognized minority in the field.63
The missing support did not make exclusion automatic, and the exclusion did not decide who was right. The district court’s first order simply found that none of Daubert’s four criteria was satisfied, but on reconsideration the court recognized that the inquiry is flexible and rested the exclusion on Carlson’s failure to satisfy Daubert’s factors or any other set of reasonable reliability criteria; the Supreme Court held that conclusion within the court’s lawful discretion.64 The ruling also shows where reliability review stops. Despite Carlson’s qualifications, the district court found his testimony outside the range where experts might reasonably differ and the jury must choose among conflicting views, even though the evidence is “shaky.”65 The exclusion tested how Carlson reached his conclusion; it did not decide what made the tire fail. A later section on admissibility versus weight develops that boundary.
The engineer and Carlson may be equally credentialed. What differs is what stands behind the opinion: the engineer’s steps match his working practice, other practitioners use his method, and written sources describe it, while Carlson’s two-factor test appeared only in his own testimony.
An exam stem that describes decades in the trade, an inspection by sight and touch, a personal rule that no other practitioner is shown to use, and an insistence that the method works is describing Carlson. Two answers go wrong. One admits the opinion because the witness is experienced and qualified, which treats a résumé as though it were a method. The other excludes the opinion automatically because nothing was published and no error rate was measured, which treats Daubert’s factors as the checklist Kumho Tire rejected. The sound answer compares the courtroom work with the field’s own practice, looks for outside corroboration, and recognizes that an assurance of accuracy with nothing behind it can be excluded.
Experience-Based Experts
Rule 702 admits testimony based on technical or other specialized knowledge gained through skill, experience, training, or education, including knowledge that is less than formal.71 Experience can therefore ground an expert opinion. The Advisory Committee Note says that nothing in the 2000 amendment suggests that experience alone may not provide a sufficient foundation for expert testimony.72 The Note attaches a condition, however. A witness who relies solely or primarily on experience “must explain how that experience leads to the conclusion reached, why that experience is a sufficient basis for the opinion, and how that experience is reliably applied to the facts.”73
The condition follows from a problem of proof. A scientific method comes with studies and published results a judge can inspect; an experienced practitioner’s judgment lives in the practitioner’s memory, where no one else can see it. The Hankey officer’s testimony was admitted because it rested on years of experience and personal knowledge, yet the question remains whether simple experience means reliability.74 In both Joiner and Kumho Tire the Court stated that an expert must present more than ipse dixit, and a witness who asks the court to admit an opinion merely because she has relevant education or experience offers exactly that.75 The explanation does for experience what the Daubert factors do for science: it gives the judge something to evaluate other than the witness’s assurance.
A witness who answers every question about method with “I used my nose. That’s all I can do” has described an act, not a method.76 Kumho Tire’s own example was a perfume tester able to distinguish among 140 odors at a sniff, and it suggested asking whether such a witness’s preparation is of a kind that others in the field would recognize as acceptable.77 A foundation that answers the Note’s three questions looks different from the tester’s reply.
Qualification and reliability are separate questions. Qualification asks whether the witness has the skill, experience, training, or education to testify as an expert, and the reconstructionist passes easily. Reliability asks whether this opinion rests on a sound basis, and a witness who offers an opinion merely because she has relevant experience offers ipse dixit.82 The explanation links the two by showing the path from this person’s experience to this opinion.
An exam stem built on this doctrine tempts two wrong answers. The first admits the opinion because the witness is a twenty-year veteran, treating qualification as though it settled reliability. The second excludes a mechanic or an investigator because nothing was published and no error rate was measured, treating laboratory factors as mandatory for craft knowledge, which the flexible approach of Kumho Tire does not require. The sound answer asks the three questions and looks for whatever the witness can show beyond personal assurance. Where the witness can show it, the opinion comes in, and the opponent’s quarrel with the conclusion becomes a matter of weight for the jury, a boundary a later section develops.