Concept
The Daubert Factors
Reliability and relevancy
The Daubert Factors
Rule 702(c) demands that expert testimony be the product of reliable principles and methods, but the rule's text does not tell a judge how to measure reliability.1 Daubert filled that gap. In evaluating the validation of an expert's methodology, the Court said, trial judges should consider, among other things, whether the methodology is testable and has been tested, whether it has been subjected to peer review and publication, whether it has a known error rate, whether accepted standards govern its use, and whether it is generally accepted.2 Each of the five receives its own treatment below, beginning with testability, the factor the Court appears to have emphasized most heavily.3
Each factor turns one habit of science into a question that a nonscientist can ask. Science proposes claims that evidence could contradict, exposes its results to outside readers, measures how often a technique misleads, follows standards that keep the work consistent, and lets a community of practitioners accept or reject what survives. A judge can ask whether an expert's method shows those habits without becoming a scientist, and that suits the judge's role. Chief Justice Rehnquist, writing separately in Daubert, accepted that Rule 702 gives judges a gatekeeping responsibility but denied that it imposes on them either the obligation or the authority to become amateur scientists.4
The five factors guide judgment and do not form a test. None is indispensable or dispositive, the list is not exhaustive, and the factors are merely considerations to be weighed concerning the offered evidence.5 On remand, the Ninth Circuit read them the same way: as illustrative rather than exhaustive, and not equally applicable, or even applicable at all, in every case. Its governing question was whether the analysis behind the experts' testimony falls within the range of accepted standards governing how scientists conduct their research and reach their conclusions.6
Two consequences matter on an exam. First, a missing factor does not compel exclusion, and a satisfied factor does not compel admission. Soundly based evidence so new that it has no track record should fare better under Daubert than under Frye, a result possible only because no single factor controls.7 Second, the listed factors are not the only road to reliability. They are ways of showing that Rule 702's requirements are met, and other ways remain open so long as they do not flout Daubert or Kumho Tire, both of which are flexible by their own terms. Purportedly scientific experts must meet a somewhat higher standard than other experts.8 Kumho Tire extended the Daubert standard to all expert testimony presenting technical or specialized material, a development taken up in a later section.9
Testability
The testability factor asks two questions: whether the expert's theory or technique can be tested, and whether anyone has tested it.11 The questions do different work. A theory fails the first if no conceivable evidence could show it wrong, and no further effort can repair that. A theory that fails only the second can be repaired by running the test, which is why a missing test counts against the proponent without ending the inquiry.
Daubert linked testability to falsifiability, the idea that a theory's scientific status depends on whether evidence could show it false. Chief Justice Rehnquist confessed that he was at a loss to know what that meant and suspected that some federal judges would be too.12 The idea is plainer than its name. A falsifiable theory takes a risk: it predicts something specific, and the evidence can come out against it. The claim that a drug raises the rate of a birth defect takes that risk, because it predicts that defects will turn up more often among children whose mothers took the drug than among children whose mothers did not.
A method that has faced a test capable of contradicting it has earned some trust, because each failed attempt at refutation is a reason to think the idea may be sound. A method that has never faced such a test leaves the judge nothing to weigh except the expert's assurance. The factor also fits the judge's limited role. The judge does not decide whether the theory is true; the Daubert test is the soundness of the expert's methodology, not the correctness of the expert's conclusions.13 Any expert can be asked, in plain terms, how a mistake in the theory would be discovered and whether anyone has looked.
The Bendectin litigation shows the factor at work. In Daubert itself, the plaintiffs answered the defendant's motion with the testimony of eight experts, whose conclusions rested on test-tube and live animal studies, on pharmacological comparisons of Bendectin's chemical structure with that of substances known to cause birth defects, and on a reanalysis of previously published epidemiological studies.14 Of the three, the epidemiological studies test the hypothesis most directly in people. On remand, the Ninth Circuit explained that such a study had to show that children whose mothers took Bendectin were more than twice as likely to develop limb reduction birth defects as children whose mothers did not. The plaintiffs' epidemiologists made vague assertions of a statistically significant relationship, but none stated that the relative risk exceeded two, so their studies would not help the jury and would only confuse it if offered to prove causation. A relative risk below two, the court added, tends to disprove legal causation, because it shows that Bendectin does not double the likelihood of birth defects.15
That result marks a boundary. Passing the testability factor carries a proponent no further than the next question, and the Ninth Circuit asked a separate one: whether the proof satisfied the requirement of fit, meaning that it showed children of mothers who took Bendectin were more than twice as likely to suffer defects.16 The animal studies and structural comparisons raise a different problem, whether those experts' opinions were sufficiently supported by the animal studies on which they purported to rely, which a later section takes up under the analytical gap.17
An exam stem often pairs an impressive credential with a confident conclusion. The testability question ignores both and asks whether some result could have contradicted the method, because the inquiry looks to the soundness of the methodology and not to whether the conclusion is correct.18
Peer Review
Peer review moves the inquiry from whether a method can be checked to whether anyone besides its proponent has checked it. The Daubert considerations include whether the methodology has been subjected to peer review and publication,20 and the factor keeps a role under Daubert, though no longer a determinative one.21 Outside readers have no stake in the result, which puts them in a better position than the author to find a flaw. The Ninth Circuit accordingly treated acceptance for publication in a reputable journal, after the usual rigors of peer review, as a significant indication that other scientists take the research seriously, meaning that it meets at least the minimal criteria of good science.22
Presence has its limits. Peer review and publication do not guarantee that a study's conclusions are correct; much published research is greeted with intense skepticism and is not borne out by further work, and the test under Daubert is the soundness of the methodology, not the correctness of the conclusions.23 The minimal criteria form a floor, and a published method still faces the questions of error rate, standards, and fit.
The Bendectin remand shows what silence looks like. Despite years of controversy, the Ninth Circuit observed, no one in the scientific community except the defendant's experts had deemed the plaintiffs' studies worthy of verification, refutation, or even comment, as if there were a tacit understanding that the work was litigation and not science.24 The court drew that inference only because the ordinary explanations for nonpublication were unavailable. A study may be too recent or of insufficiently broad interest to be published, but the plaintiffs' experts had otherwise been well published, so those reasons did not apply.25
Absence of review shifts the proponent's task without ending the proponent's case. Without peer review, the proponent must show validity another way, including expert testimony that explains how the experts reached their conclusions and shows that they followed the scientific method as practiced by at least a recognized minority.26 The Bendectin plaintiffs offered only their experts' unadorned assertions that their methods matched standard scientific procedure, together with those experts' testimony in other cases. Those materials showed that the experts had relied on animal studies, chemical structure analyses, and epidemiological data, but they neither explained the methodology the experts followed nor pointed to any outside source that validated it.27 Neither the methodological showing nor the fit showing was made.28
Where the absence has an explanation, the factor loses force. Kumho Tire observed that it might not be surprising for a claim made by a scientific witness never to have been peer reviewed, because the particular application at issue may never have interested any scientist.29 The same reasoning reaches technical and experience-based experts. The Ninth Circuit cautioned that the Daubert factors may not apply at all in a given case,30 and Kumho Tire suggested that, for an engineer's experience-based method, the judge might ask instead how often the method has produced erroneous results or whether it is generally accepted in the relevant engineering community.31
Secrecy supplied a third explanation in United States v. Chiaradio. The district court held a Daubert hearing, a proceeding outside the jury's presence in which the judge considers data and arguments for and against admissibility,32 and found the government's file-sharing program, EP2P, reliable because it had no error rate, had never yielded a false positive, and produced results that could be independently verified. The program's source code had not been peer reviewed, but the court held that the government's need to keep the code secret satisfactorily explained the absence.33 The two cases differ in the way that decides the factor. In the Bendectin litigation no explanation was available and nothing else supplied the validation; in Chiaradio a reason existed and other objective indicators carried the showing.
An exam stem that mentions publication is testing which way the fact cuts. An expert whose method has been published and debated has cleared a minimal threshold, yet the judge still weighs error rate, standards, and fit, because a published study can be wrong.34 An expert whose method is unpublished is not excluded for that reason alone; the judge asks whether the absence has an explanation and, if it has none, whether the proponent has shown by other objective evidence that the method follows accepted scientific practice.35 A later section on litigation-driven opinions takes up the related question whether the work was done for the lawsuit.
Known Error Rate
The third consideration asks how often a technique gives the wrong answer. Daubert's list includes whether the methodology has a known error rate,37 and the Court cited a Seventh Circuit decision, United States v. Smith, that surveyed studies of the error rate of spectrographic voice identification.38 Another statement of the factor refers to the potential (or known) rate of error,39 wording that covers both error that has been measured and error that a technique is capable of producing. An error rate counts wrong results, including false positives, in which the technique reports a link that is not there.
Courts ask about error because an opinion can be no more trustworthy than the method behind it is accurate, and a judge cannot see accuracy from the bench without data. The judge can still ask how often the method has been wrong. Kumho Tire gives the example of an engineering expert's experience-based methodology: in certain cases it is appropriate for the trial judge to ask how often that methodology has produced erroneous results.40
The error-rate question also limits what an expert may tell the jury. Some applications of science rest on rigorous principles: blood testing, blood typing, and the application of genetic and DNA principles, if properly conducted, produce highly refined and dependable results. Handwriting analysis, ballistics testimony, hair identification, syndrome evidence, and the psychology of eyewitness identification rest on more speculative principles and greater subjectivity, and although often admitted, they stand on less certain footing. Experts in those fields tend to overstate their conclusions, expressing a confidence the methodology does not support.41 A firearms examiner who tells the jury that his error rate is zero and that the bullet found at the scene “matches” the defendant's gun fits the pattern: his judgment is subjective and so carries a rate of error, and no one has established that each gun makes a unique marking on a bullet, so he cannot validly testify to a match.42
A claim of no errors can survive when evidence supports it. The Chiaradio court's finding that EP2P had no error rate and had never yielded a false positive shows the factor working in a proponent's favor.43 That claim came out of a hearing at which the court found it supported, because the program's results could be independently verified. The examiner's claim of a zero rate rests on his own say-so about a method whose uniqueness premise has never been established. The difference in foundation, and not the bare number, separates the two.
As with the other considerations, error rate feeds a flexible inquiry in which no factor is dispositive,44 and the factor concerns the technique's record and leaves aside whether the expert's conclusion in this case is correct.45 On an exam, an expert who claims a zero error rate, a match, or an identification to the exclusion of all others is the cue to ask what validation supports so strong a claim. The answer sets the limit on what the expert may say, a point the later section on staying within the method develops.
Standards and Controls
The fourth consideration asks whether accepted standards govern the use of the methodology.47 Standards here are the written rules for how a technique must be performed, and the factor raises two questions: whether such standards exist, and whether this expert followed them. The questions fall under different parts of Rule 702. Whether a field has standards goes to reliable principles and methods under Rule 702(c).48 Whether the expert followed them goes to Rule 702(d), which requires that the opinion reflect a reliable application of the principles and methods to the facts of the case.49
Written standards matter because they fix a technique's steps before anyone knows what result the proponent wants. With a standard in place, the judge has an outside benchmark for measuring the work, and a departure can be recognized as a departure. Without one, the technique is whatever the analyst chooses to do, and the judge is left with the analyst's own assurance. Standards also give an error rate its meaning, since an error rate describes a technique performed under defined conditions.
DNA analysis shows why application matters. Its principles produce highly dependable results if properly conducted,50 and the word “if” carries the weight. A technician can put DNA from a single sample in two lanes of a gel and obtain a perfect but false match.51 Controls and written steps are what let a court check that a mistake of that kind did not occur in the case before it.
A party can locate a standard in a learned treatise that falls within the hearsay exception in Rule 803(18), and the parties may cite such standards when they litigate the Rule 702(d) issue.52
Neither pole decides admissibility alone, because no factor is indispensable or dispositive.53 Compliance with a standard supports reliability without showing that the standard rests on validated science or that the technique fits the disputed issue. The absence of a formal standard deprives the proponent of one objective indicator and leaves the other considerations to carry the showing. A standard that exists but was ignored belongs to Rule 702(d), and a later section on proper procedure develops it. When an exam stem describes a laboratory that streamlined its protocol, or one that has none, ask first whether the field has a governing standard, which goes mainly to the method, and then whether this expert followed it, which goes mainly to application.
General Acceptance
Frye made general acceptance the whole test, and the logic behind it explains the factor's smaller role today. A judge facing a scientific dispute can admit everything from qualified experts and let the jury sort it out, decide for herself whether the proof is valid science, or defer to the scientific community by asking the proponent to show that scientists generally agree with the proof. Frye most clearly resembles the third course: the judge excludes only because other scientists do not accept the proof, and she can tell the expert that the verdict of other scientists, and not her own opinion, controls.55
Daubert keeps acceptance as the fifth factor on its list56 but demotes it. General acceptance and peer review both keep a role, though no longer a determinative one, and the Court declined to define the relevant field rigidly, to require any particular degree of acceptance, or even to specify what exactly must be accepted. Widespread scientific disapproval of the evidence, at any level, counts as a negative factor in assessing its validity.57
The change sorts evidence differently. Soundly based evidence so new that it has no track record should fare better under Daubert than under Frye, while unsoundly based evidence with substantial support in its own field should fare better under Frye.58 Under Frye, acceptance was the proof of reliability; under Daubert, it is one piece of evidence about reliability.
Acceptance cannot be the whole test, because a rule that admitted only accepted methods would hold sound new science out until others had certified it. A method that only a minority of scientists accept can still come in. The Ninth Circuit on remand said that a party's use of a methodology accepted by only a minority of scientists would be a proper basis for impeachment at trial,59 and it described the proponent's required showing as one that the experts followed the scientific method as practiced by at least a recognized minority.60 The proponent still must explain the methodology and point to some outside source that validates it, because unadorned assurances do not suffice.61 Thin support then becomes a subject for cross-examination.
The opposite limit comes from Kumho Tire Co. v. Carmichael, which observed that the presence of the general acceptance factor does not help show that testimony is reliable where the discipline itself lacks reliability, as with theories grounded in generally accepted principles of astrology or necromancy.62 Agreement among practitioners vouches for a method only when the field checks its claims against evidence. A room of astrologers who share the same principles has multiplied one untested assertion, and counting them adds nothing. The judge must therefore ask whether the community doing the accepting tests its claims before giving its agreement any weight. Kumho Tire also shows the factor traveling beyond hard science: for an engineering expert it may be appropriate to ask whether the method is generally accepted in the relevant engineering community, and even of a perfume tester whose expertise rests purely on experience it can be useful to ask whether his preparation is of a kind that others in the field would recognize as acceptable.63
An exam stem that says every practitioner of a technique endorses it describes only one factor, and the tempting answers on both sides revive Frye. One answer admits the opinion because the field agrees, treating acceptance as sufficient. The other excludes an opinion because the field has not yet embraced the method, treating acceptance as required. The sound analysis weighs acceptance with testing, review, error rate, and standards, and asks first whether the discipline doing the accepting is itself reliable.