Concept
Additional Reliability Indicators
Reliability and relevancy
Additional Reliability Indicators
Rule 702(c) asks whether the testimony is the product of reliable principles and methods, and the five Daubert considerations are the best-known tools for answering that question.1 They are not the only tools. The Advisory Committee Note to the 2000 amendment adds several more. It asks whether the methodology was developed especially for litigation, in which case it is less likely to be reliable; whether alternative explanations are adequately addressed and accounted for; and whether the expert, including an expert who relies solely on experience, can adequately explain how the methods lead to the conclusions.2 It also asks whether the expert “is being as careful as he would be in his regular professional work” when issuing the opinion.3
These questions fit the setting in which expert opinions are actually offered. A retained expert can work backward from the result a client needs, pass over an obvious rival explanation for the effect observed, or stretch an accepted premise past what it can carry. Each added indicator targets one of those patterns, and each gives the judge a question that can be put to an expert in any field without first mastering the science.
Daubert itself called the reliability considerations flexible and fixed the focus of the inquiry “solely on principles and methodology, not on the conclusions that they generate.”4 The added indicators work the same way. A judge may weigh them alongside the original five when the facts call for it, and none operates as a rule of exclusion.
This section takes three indicators in turn. The first asks where the opinion came from, whether independent research or work built for the testimony. The second asks whether the expert dealt with the obvious rival explanations. The third looks beyond the expert's own work, at how the technique relates to methods already shown reliable and at the uses it has outside the courtroom. The Note's questions about care and about explaining the method return in the sections on Kumho Tire and experience-based experts. An answer that finds all five Daubert considerations satisfied has therefore not necessarily finished the Rule 702(c) analysis, because a method created for the case, or an opinion that ignores the obvious rival cause, can still fail.
Litigation-Driven Opinions
A judge who cannot audit the science can still ask where an opinion came from. On remand in Daubert, the Ninth Circuit framed the question as whether the experts are “proposing to testify about matters growing naturally and directly out of research they have conducted independent of the litigation, or whether they have developed their opinions expressly for purposes of testifying,” and it called that one very significant fact.5 The Advisory Committee Note carries the concern into Rule 702 practice: a methodology developed especially for litigation is less likely to be reliable.6 The inquiry looks at the opinion's history because an opinion that is the output of a scientist's ordinary work and one assembled after a party went looking for support differ in what they show about how the method was chosen.
Testimony based directly on legitimate, preexisting research unrelated to the litigation is the “most persuasive basis” for concluding that an expert's opinions were derived by the scientific method.7 Such research exists whether or not anyone ever hires the expert, so it shows how the method was used before any party had a stake in the answer.
The indicator raises the demand for proof without operating as a rule of exclusion. That an expert testifies for money does not necessarily cast doubt on the reliability of the testimony, since few experts appear in court merely as an eleemosynary gesture. But a court judging whether proposed testimony amounts to good science may not ignore that a scientist's normal workplace is the lab or the field, not the courtroom or the lawyer's office.8
Some fields are the exception. Fingerprint analysis, voice recognition, DNA fingerprinting, and other scientific endeavors closely tied to law enforcement may have the courtroom as a principal theater of operations, and for such disciplines the fact that the expert developed the expertise principally for litigation will obviously not be a substantial consideration.9 Where the courtroom is a normal workplace, litigation origin says little about whether the method is sound.
When the opinion does not rest on independent research, the proponent must come forward with other objective, verifiable evidence that the testimony is based on scientifically valid principles. One means is proof that the research and analysis behind the conclusions have been subjected to normal scientific scrutiny through peer review and publication; as one commentator quoted by the court put it, “the ultimate test of [a scientific expert's] integrity is her readiness to publish and be damned.”10 Pre-litigation research and peer review are the two principal ways to make the showing. Where neither exists, the proponent may use its own experts, who must explain precisely how they reached their conclusions and point to some objective source, such as a learned treatise, the policy statement of a professional association, or a published article in a reputable scientific journal, showing that they followed the scientific method as practiced by at least a recognized minority of scientists in their field.11
The plaintiffs in Daubert could not make that showing. Their experts had relied on animal studies, chemical structure analyses, and epidemiological data, but they neither explained the methodology they followed nor pointed to any external source to validate it. The court had only the experts' qualifications, their conclusions, and their assurances of reliability, and under Daubert that was not enough.12 On remand the Ninth Circuit again found the plaintiffs' proof was not valid science: without peer review, the proponent had to show validity another way, and that showing was not made.13 Litigation origin did not by itself bar the testimony. It removed the most persuasive proof of reliability and left the plaintiffs to supply other objective evidence.
Independence is evidence about how the work was done, and it does not make the conclusions correct, because the focus stays on principles and methodology and not on the conclusions they generate.14 A toxicologist whose earlier research used an untested technique still faces the other Rule 702(c) considerations. An expert retained first and asked afterward to run new analyses is not barred from testifying. If that expert explains precisely how the analysis was done and points to an objective source showing that a recognized minority of researchers in the field use the method, the proponent has supplied the validation that independence would otherwise have provided.15
Exam facts that flag this indicator include analysis prepared at counsel's request and a study never published or reviewed outside the case. The origin counts against the proponent and shifts the task to showing validity by other objective means, and it does not by itself exclude the opinion. Facts that point the other way include research that predates the suit and a field closely tied to forensic work. The next indicator turns from where the opinion came from to whether the expert dealt with the obvious rival explanations.
Alternative Explanations
A causal opinion is an inference, and the Advisory Committee Note asks whether the expert has adequately addressed and accounted for obvious alternative explanations of the effect.17 The last indicator asked where the opinion came from; this one asks what the expert did with the rival possibilities. It matters most in disputes about why something happened, where more than one answer is plausible.
The expert observes an effect, such as a cancer, a fire, or a birth defect, selects a cause, and tells the jury that the cause produced the effect. Because many effects have more than one possible source, proof that the favored cause was present does not show that it was the operative one. An expert who skips the rivals leaves the jury no reason to prefer the conclusion over the alternatives, and the inference becomes a guess. The indicator tests that reasoning from the outside, so a judge without scientific training can apply it: did the expert confront the obvious rivals?
The question is whether the expert dealt with the alternatives, not whether the expert chose the cause the judge would have chosen. The focus of the reliability inquiry stays on principles and methodology, not on the conclusions they generate.18 The Advisory Committee recognizes, though, that a conclusion departing sharply from that of the expert's peers is reason to ask whether the principles and methodologies have been faithfully applied, because an unreliable application of a reliable methodology results in an unreliable opinion.19 An expert who works through the plaintiff's other risk factors and explains why the defendant's product still contributed satisfies the indicator even if the opposing expert insists the other factors explain everything. That disagreement goes to which reliable account the jury accepts, since the judge decides whether the experts are reliable and the jury decides whether they are correct.20
The Ninth Circuit's remand opinion in Daubert shows the failing case. Unlike the other plaintiffs' experts, who spoke in terms of probabilities, Dr. Palmer concluded that the plaintiffs' injuries were in fact caused by Bendectin rather than another cause, although the record refuted the notion that anyone can tell what caused the birth defects in any given case.21 The court found that he offered no tested or testable theory explaining how, from the limited information he had, he eliminated all other potential causes of birth defects, or how he alone could state as a fact that Bendectin caused the injuries. Agreeing with the Sixth Circuit, the court observed that he did not testify on the basis of the collective view of his scientific discipline, nor did he take issue with his peers and explain the grounds for his differences: “no understandable scientific basis is stated. Personal opinion, not science, is testifying here.” His testimony was inadmissible as a matter of law under Rule 702.22 A claim that specific is where unaddressed alternatives do the most damage, because saying that one cause produced this injury says that every other cause was excluded. The court also noted that the plaintiffs' experts did not seek to differentiate these plaintiffs from the subjects of the statistical studies, so the studies had to stand or fall on their own.23
Medical experts often build the accounting into their method through differential diagnosis, which lists the possible causes of a patient's condition and reasons toward the one that remains after the others are ruled out. Acceptance of the method does not excuse skipping its steps. In Guinn v. AstraZeneca Pharmaceuticals LP, the Eleventh Circuit called differential diagnosis a scientifically accepted methodology and still found the expert's opinion unreliable, because the expert had not adequately considered possible alternative causes of the plaintiff's weight gain and diabetes.24 The result tracks Rule 702(d), which requires that the opinion reflect a reliable application of the principles and methods to the facts of the case.25 A sound method applied without considering the rival causes has not been reliably applied.
The Note speaks of obvious alternatives, and the word sets the scope. The judge asks whether the expert confronted the competing causes the record makes plausible, such as a lung cancer plaintiff's smoking. Demanding that the expert eliminate every conceivable cause would require a certainty that science does not offer and would push the judge toward deciding the merits. Like the other indicators, this one feeds the Rule 702(c) judgment and does not alone decide admissibility. A stem that gives the plaintiff a competing risk factor and then describes an expert who never mentions it is signaling this indicator. The silence counts against the proponent, who must show that the opinion rests on more than the expert's say-so, and the court need not decide which cause was in fact responsible. How a court reviews a reliability ruling, and when a gap between data and opinion is itself the flaw, is the subject of General Electric Co. v. Joiner in the next section.
Related Methods and Outside Uses
Rule 702(c) asks whether the testimony is the product of reliable principles and methods, and the Third Circuit has given trial courts an open-ended list of factors for answering that question.27 In Meadows v. Anchor Longwall & Rebuild, Inc., the court stated that a trial court may consider several factors in evaluating whether a methodology is reliable, “including but not limited to” eight: a testable hypothesis; peer review; the known or potential rate of error; the existence and maintenance of standards controlling the technique's operation; general acceptance; the relationship of the technique to methods established to be reliable; the qualifications of the expert witness testifying based on the methodology; and the non-judicial uses to which the method has been put.28
The first five track Daubert's considerations. The last three go beyond them. The qualifications factor asks about the witness and not the method; in Meadows the district court found the engineer qualified, and the parties did not contest that finding.29 The other two, the technique's relationship to established methods and its non-judicial uses, ask what the method itself has going for it and are this section's subject. Because they come from the circuit's list and not from the Supreme Court's, a court that weighs them is using the flexibility of the reliability inquiry, not applying a command of Rule 702. In Reiff v. Convergent Techs., the court restated that the Third Circuit requires district courts to examine all factors relevant to reliability, including those listed in Daubert and Paoli and any other inquiry the district court finds relevant.30
A judge cannot test a new technique from the bench, but a technique built from, or checked against, a method already established as reliable draws support from that method's record. The support reaches only as far as the overlap. The step that is new is where untested error can enter, so the proponent must explain why that step is sound. And the established method must in fact have been shown reliable, since resemblance to a method of doubtful standing proves nothing.
The non-judicial uses factor asks whether the method has a life apart from lawsuits. A method used only to produce courtroom testimony is checked by no one but the opposing lawyer, while a method that people rely on to design structures, diagnose patients, or run processes has been exposed to the consequences of its errors outside the courtroom. This differs from the litigation-origin indicator. That one asks where this expert's opinion came from; this one asks whether the method has a record of its own.
Technical subjects add signals of their own. In cases involving subjects like engineering, trial courts may consider relevant literature, evidence of industry practice, product design, and accident history in evaluating reliability.31 These do work comparable to peer review in laboratory science. Engineering knowledge often lives in handbooks, shared practice, and service performance more than in journals, so the questions become whether the people who do this work do it this way, whether the approach has been described and criticized in print, and what the record of failures says about it. An accident history resembles a field test, because it shows how often experience has contradicted the method.
The consultant's formula borrows credibility from no established method and has no use outside the lawsuit, so both indicators weigh against reliability, and the proponent must supply validation by other objective means. The engineer's analysis draws on an established method, has outside uses, and finds support in the literature and in practice, so the indicators favor reliability. Neither result is final. The consultant's opinion is not excluded automatically, because the indicators are inputs to a flexible judgment. The engineer's opinion must still fit the platform actually at issue and reflect a reliable application of the calculations to the facts of the case.32
These indicators measure reliability and leave correctness to the jury: a judge decides whether the experts are reliable, and the jury decides whether they are correct.33 An exam stem describing calculations engineers use every day for design work, supported by the literature, therefore supplies facts that favor the proponent on this point only. A stem describing a technique invented for the case, used by no one else and unrelated to any validated method, supplies the opposite facts, which count against the proponent without ending the analysis. A common error is to present either indicator as a Supreme Court requirement, when both are circuit-level considerations that a court may weigh together with Daubert's five.