Higher Order Evidence

Defining Higher Order Evidence

Higher order evidence is evidence about the status, reliability, or quality of one’s own cognitive processes, as opposed to first-order evidence, which bears directly on some claim about the world. While first-order evidence concerns whether a proposition p is true, higher order evidence concerns whether one’s way of arriving at a belief about p is trustworthy, impaired, or biased.

Typical sources of higher order evidence include learning that:

• One is tired, drunk, or under strong emotional influence.
• One has made reasoning mistakes in similar cases.
• Competent peers strongly disagree with one’s judgment.
• The methods or instruments one used are unreliable.

The central issue is how such evidence should rationally affect our confidence in first-order beliefs. Higher order evidence seems to matter, yet integrating it with ordinary evidential reasoning raises subtle puzzles about rationality, introspection, and disagreement.

Central Examples and Puzzles

Philosophers often use stylized examples to illuminate the distinctive role of higher order evidence:

• The Drunk Doctor: A doctor, while sober, examines a patient and concludes they do not have a certain disease. Later, after several drinks, the doctor reconsiders and reaches the same medical conclusion. The higher order evidence that she is now intoxicated appears to undermine the rational status of the belief, even though the first-order medical evidence and conclusion are unchanged.

• Peer Disagreement: Two mathematicians, equal in training and ability, independently work through a proof. One concludes that the theorem is true, the other that it is false. Each then learns of the other’s contrary verdict. This disagreement is not directly about the mathematical proposition itself; rather, it is evidence that at least one of them has made a mistake in reasoning.

• Bias and Cognitive Limitations: A person learns about systematic cognitive biases (such as confirmation bias, motivated reasoning, or overconfidence) and discovers that people with their political or religious orientation are especially prone to such errors. This information may serve as higher order evidence that their strongly held views are less reliable than they had assumed.

These cases generate several puzzles:

1. Deflation vs. Stability: When faced with higher order evidence that one’s reasoning might be flawed, should one reduce confidence in the first-order belief, even if the object-level evidence is strong? Or can one rationally remain confident?

2. Self-Undermining Evidence: Higher order evidence can sometimes seem to undermine itself. For instance, learning that “I am generally unreliable about probability” appears to be evidence against the reliability of the belief that “I am generally unreliable about probability.”

3. Double-Counting or Miscounting: If all evidence is ultimately just evidence, is higher order evidence already implicitly accounted for in a rational agent’s first-order probabilities? Or does it provide a distinctive kind of input that standard models mis-handle?

Major Theoretical Approaches

Philosophical responses to higher order evidence often fall into several broad camps.

1. Conciliatory Views

Conciliationists maintain that higher order evidence should usually lead agents to reduce confidence in their first-order beliefs, especially in cases of peer disagreement. Learning that others, no less competent, disagree with you is evidence that you may have erred, and rationality requires “splitting the difference” or at least significantly moderating your confidence.

Proponents argue that this view:

• Respects the epistemic significance of others’ perspectives.
• Fits intuitive judgments about symmetric disagreement.
• Helps explain why arrogance or dogmatism can be epistemically defective.

Critics object that strong conciliation can lead to:

• Excessive skepticism when disagreements are widespread.
• A “bootstrapping” of doubt: the more one encounters disagreement, the less confidence one is rationally allowed to have, even in well-supported conclusions.

2. Steadfast Views

Steadfast theorists contend that agents can, in many cases, rationally retain much of their original confidence despite higher order evidence. They emphasize that one’s own assessment of the first-order evidence can justifiably carry greater weight than the bare fact of disagreement or potential bias, especially when one has carefully reviewed the evidence.

Supporters argue:

• Rationality sometimes permits standing by one’s evidence-based judgments, even in the face of disagreement.
• Not all apparent peers are genuine epistemic equals; asymmetries in evidence or reasoning may be hidden.
• Excessive conciliation may require agents to abandon well-supported scientific or moral commitments too easily.

Opponents respond that steadfastness risks ignoring serious indicators of one’s fallibility and may license an overly confident or insulated attitude toward dissent.

3. Skeptical or Revisionary Approaches

Some philosophers highlight the potentially destabilizing implications of higher order evidence. If humans are systematically biased, overconfident, and reasoning under opaque social and psychological influences, then we possess extensive higher order evidence that many of our beliefs—especially about controversial or value-laden topics—are less reliable than we assume.

On more radical readings, this can support forms of epistemic skepticism or motivate significant revision of our belief-forming practices. Others treat this as a call for institutional or methodological reforms (e.g., blind review, pre-registration in science, or structured disagreement) to counteract the negative implications of higher order evidence.

4. Meta-Evidential or Bayesian Treatments

Another approach maintains that higher order evidence is not sui generis but is best understood as additional, ordinary evidence about the world and about one’s own reliability. On this meta-evidential or Bayesian view:

• Information about your cognitive situation (e.g., being tired, discovering disagreement) simply updates your probabilistic model.
• There is no need for special principles beyond the general rules of rational belief revision.

Advocates claim this preserves the unity of evidence and avoids ad hoc constraints. Critics argue that standard formal models often struggle to capture intuitive features of higher order evidence, such as its ability to rationally require suspending judgment even when first-order evidence is strong.

Connections and Ongoing Debates

Higher order evidence sits at the intersection of several areas in epistemology and beyond:

• Disagreement and Epistemic Peerhood: Debates about when two agents count as “epistemic peers” and what rational responses to disagreement should be are among the most active contexts for discussing higher order evidence.
• Reflection and Self-Knowledge: Higher order evidence raises questions about how well we can know our own cognitive states and reliability, and whether rational agents should “trust themselves” or systematically correct for recognized biases.
• Pragmatic Encroachment: Some views explore whether practical stakes interact with higher order evidence—for instance, whether awareness of possible error should matter more when decisions are high-risk.
• Philosophy of Science and Methodology: Scientific practices such as replication, blinding, and statistical correction for multiple testing can be understood as institutional responses to collective higher order evidence about scientific error and bias.

Current discussions focus on whether a unified theory can both:

• Respect the intuitive force of higher order evidence (e.g., in the drunk doctor and peer disagreement cases), and
• Avoid an unmanageable slide into skepticism or stalemated conciliation in the face of pervasive disagreement and bias.

The topic continues to be central for understanding how reflective agents should respond not only to the world’s evidence, but also to evidence about their own cognitive limits.