Error, Uncertainty, and Scientific Inference

Science After the Promise of Exactness

Angela Bogdanova Published by Aisentica Development Philosophy by Bogdanova · Philosophy of Science · Research Dossier American English · Research edition: November 7, 2026 Reserved canonical: https://philosophybybogdanova.com/dossiers/error-uncertainty-and-scientific-inference/ Editorial state: Research manuscript; public release reserved.

Scientific knowledge becomes dependable through the identification, evaluation, and disciplined revision of its own possible failures. A measurement is informative because it is related to a precisely described target and to procedures that make disagreement detectable. A statistical estimate earns authority through a design linking sampled observations to a population or process. A causal claim becomes credible when the alternatives capable of producing the same association have been examined. These are different operations, and the conditions that make one successful do not automatically justify the others.

The Guide to the Expression of Uncertainty in Measurement (JCGM, 2008) illustrates the first operation: uncertainty is evaluated as a component of the expression of measurement results. The ASA statement on p values (Wasserstein and Lazar, 2016) illustrates the second: statistical significance cannot replace design, context, or the interpretation of actual evidence. The philosophical issue connecting these distinct traditions is the legitimacy of an inference in the presence of incomplete knowledge.

An instrument can provide many decimal places while a poorly specified target remains ambiguous. A sample can yield a narrow interval while excluding the population to which an investigator intends to generalize. A complex model can reproduce observed outcomes and still embody a mistaken causal account. The recurring pattern is a difference between computational refinement and evidential justification. A responsible research claim states the domain in which the evidence warrants an assertion and explains the route by which residual uncertainty might change that assertion.

The Object and Method of This Inquiry

This work is a conceptual and documentary synthesis. Its primary objects are scientific claims, their conditions of support, and the error-bearing operations connecting measurement, analysis, representation, explanation, and decision. It examines first-party metrological guidance, scholarly analyses of significance and study design, philosophy of scientific theory choice, and institutional definitions of reproducibility. Where source documents provide a formal definition, the discussion attributes that definition. Where an example is invented to clarify a distinction, it is explicitly illustrative rather than presented as a measured finding.

The method is comparative reconstruction. First, identify the question a procedure answers. Second, state its assumptions and inferential target. Third, distinguish internal uncertainty quantified by the procedure from external questions it leaves open. Fourth, examine what conclusion remains after plausible assumptions are varied. Finally, evaluate how error costs affect the appropriate language of the claim. Such comparisons do not produce a universal probability of scientific correctness; they disclose why different scientific judgments call for different forms of evidence.

A limit follows from the source method itself. A DOI landing page or publisher abstract verifies bibliographic identity and supports claims present in its accessible text. It does not certify inspection of inaccessible full chapters or any independent reanalysis of the cited data. The evidence ledger associated with this manuscript records those access distinctions. The work makes original arguments about the relations among established methods, while methodological claims and historical attributions remain accountable to identifiable research.

Error, Uncertainty, and the Language of Warrant

Error ordinarily describes a difference between a result and a reference or true value, when such a comparison is meaningful. Uncertainty expresses a range or dispersion associated with values that may reasonably be attributed to a measurand under specified information. Bias describes a systematic tendency of an estimator, instrument, or process relative to a designated target. Precision concerns the closeness of repeated results under specified conditions. These terms have related but distinct roles, a discipline reflected in the International Vocabulary of Metrology (JCGM, 2012).

The distinction has epistemological force. An unknown error is not established merely because a result has uncertainty, and an instrument showing little variation need not be accurate. For example, readings clustered tightly around a miscalibrated zero can be precise while systematically displaced. Conversely, a procedure may be unbiased under a statistical model and yield dispersed individual estimates. One property does not absorb the other.

In statistical research the word error also designates residuals, random disturbances, wrong decisions, and discrepancies between estimated and actual quantities. Their meanings are fixed by the question and model. Treating an error term in a regression as if it encompassed all biases produces unwarranted confidence. Treating an interval as a guarantee against omitted variables produces the same mistake in another form. Scientific reporting benefits from identifying each source of uncertainty by origin, relevant target, treatment in analysis, and observable consequences. The vocabulary is valuable when it preserves distinctions that affect which conclusions may be drawn.

The Measurand and the Construct

Before asking how accurately something has been measured, inquiry must identify what the result purports to concern. Metrology calls the quantity intended to be measured the measurand. Its specification includes the relevant object, state, and conditions. A temperature measured at one location and instant may differ from a spatial or temporal average. A biological concentration may depend on sampling conditions. The meaning of an uncertainty interval therefore depends on the quantity that was actually specified.

Scientific research also studies constructs such as intelligence, well-being, trust, or ecological resilience. These are often linked to measurements by operational definitions, scales, and theoretical assumptions. A small standard error for a survey score establishes precision about the score under its design; it does not by itself establish that the score exhausts the construct of interest. Construct validity and measurement uncertainty address overlapping questions but are not interchangeable.

The metrological NIST guidelines for uncertainty evaluation (Taylor and Kuyatt, 1994) offer a disciplined vocabulary for quantities and uncertainty components. Extending their central lesson to construct-driven sciences requires an additional question: does the operational target faithfully represent the phenomenon invoked by the substantive claim? If the construct is ambiguous, repeated precision may make an ambiguity easier to overlook rather than resolving it. The defensible response is to document the operationalization and compare it with plausible alternatives before extending a measured difference into a broad substantive conclusion.

Traceability, Calibration, and Reference

Measurement traceability links a result to a reference through a documented sequence of calibrations, each contributing uncertainty. The reference could be a recognized standard, a unit realization, or another stipulated point of comparison. Calibration provides information about an instrument's relation to that reference; it should also disclose conditions and stability. The VIM terminology of traceability (JCGM, 2012) helps distinguish this documented chain from the mere presence of a familiar unit on a display.

A chain can be traceable while a substantive conclusion remains poorly supported. If a sensor is calibrated in a laboratory but used under temperature, humidity, loading, or timing conditions outside those examined, the transfer requires further evidence. The same is true when a laboratory value is generalized to an entire environmental system. Calibrated readings answer a limited measurement question; a theory or policy claim may require additional sampling and causal reasoning.

This distinction also clarifies the relation between scientific and documentary provenance. A traceable calibration chain concerns the reference structure of a measurement. A reproducible computational analysis concerns transformations of data. An archival record concerns the identity and history of documents. These chains may reinforce one another, but one cannot serve automatically as proof of the others. Scientific inference requires the right chain for the particular epistemic passage at stake.

Random Variation and Systematic Effects

Repeated observations reveal variation, but the observed spread is rarely a complete description of error. Random effects can change from measurement to measurement; systematic influences can shift results in a patterned direction. Yet systematic effects may be uncertain and therefore appear within an uncertainty budget. The key practical question is whether a component has been characterized and included, not whether someone has attached the word random to it.

The NIST Type B evaluation guidance (Taylor and Kuyatt, 1994) notes that scientific judgment about calibration records, instrument specifications, experience, and reference data can inform uncertainty estimates. Thus a component need not be estimated through repeated observations to warrant explicit inclusion. Conversely, adding more repetitions reduces the uncertainty of a sample average only for components that actually average down under the assumed dependence structure.

Consider a scale that adds the same offset to every reading. Averaging one thousand observations may reveal a stable mean with a very small sampling standard error; that operation leaves the shared offset essentially untouched. The apparent paradox disappears when uncertainty components are attached to their causal or procedural origins. A narrow summary of repeated readings is a fact about repeatability under those conditions. An inference about the underlying quantity must also address the common influences that repetitions leave intact.

Type A, Type B, and the Evidence Behind an Uncertainty Budget

The JCGM distinguishes Type A evaluation of standard uncertainty, based on statistical analysis of observations, from Type B evaluation, based on other information. This is a classification of methods of evaluation rather than a metaphysical division between random and systematic errors. The GUM evaluation framework (JCGM, 2008) and NIST Technical Note 1297 (Taylor and Kuyatt, 1994) make that separation operational.

An uncertainty budget should name each influential component and explain its numerical representation, including units and distributional assumptions. Repeated readings may support an estimated variance; a calibration certificate may provide a stated uncertainty; a bounded environmental contribution may be described using a justified distribution. The components must then be combined with sensitivity to dependence and the measurement model. The correct answer depends on the assumptions connecting inputs to the output, and the budget becomes less informative if those assumptions disappear behind a single final number.

An epistemic advantage follows. An explicit budget exposes what additional evidence might reduce the uncertainty. More observations may sharpen a Type A component. A better calibration may change a Type B component. A redesigned experiment may remove a shared influence. The budget is therefore both a numerical expression and an investigative map. It identifies the places where further scrutiny can alter the warranted precision of the claim.

Propagation, Dependence, and the Shape of an Error Budget

Scientific quantities are often derived from other quantities. For a sufficiently smooth measurement model, a linear approximation can propagate input variances through sensitivity coefficients, and covariance terms account for shared variation. Omitting covariance assumes away dependence; it cannot be justified merely because a calculation becomes simpler. Correlated calibration sources, common reference materials, and jointly estimated coefficients may make the omitted terms consequential.

The GUM propagation guidance (JCGM, 2008) supplies a widely used analytic approach. The Monte Carlo supplement (JCGM, 2008) supplies another route for propagating distributions when direct linear approximations are inadequate. Neither approach eliminates the need to justify its input distributions or the form of the measurement model. A highly polished output distribution built from implausible inputs remains conditional on those inputs.

Imagine estimating a ratio of two correlated measurements. Its distribution can be highly asymmetric when the denominator approaches zero, and a symmetric standard-error interval may be misleading. The reason is structural: the mathematical transformation changes how input uncertainty appears in the result. Reporting only the estimated ratio hides precisely the information needed to judge it. The disciplined practice is to retain the model, the dependency assumptions, the computational method, and the circumstances under which approximation errors matter.

Coverage Intervals and Their Conditions

An expanded measurement uncertainty usually multiplies a combined standard uncertainty by a coverage factor, often written U = k × u_c. The result is meaningful together with its coverage rationale and assumptions. Reporting a single ± quantity without the measurand, the standard-uncertainty basis, or the selected factor can invite readers to interpret it as a universal certainty guarantee.

The NIST reporting requirements (Taylor and Kuyatt, 1994) specify that reports should explain the components and the chosen coverage factor. This matters because an interval's numerical width is only one part of its evidentiary content. Effective degrees of freedom, assumptions about input distributions, and the intended interpretation of coverage influence how confidently the interval may be used.

One broader philosophical lesson follows. A stated interval is never simply a segment on a number line. It is a claim about what methods, models, and observations license the investigator to say. In metrology, the interval is attached to a particular measurand and uncertainty evaluation. In classical statistics, coverage typically refers to a repeated-sampling procedure. In Bayesian inference, a credible interval is conditional on a posterior distribution formed from a model and prior. The geometrical similarity of the intervals does not erase the difference between their inferential interpretations.

A Worked Measurement Comparison

Consider a hypothetical laboratory balance used to estimate the mass of a sample. Suppose the corrected result is 100.00 g; the standard uncertainty from repeatability is 0.03 g; the standard uncertainty assigned to calibration is 0.04 g; and the components are judged uncorrelated. Under a linear additive model, the combined standard uncertainty is the square root of 0.03 squared plus 0.04 squared, which equals 0.05 g. With an illustrative coverage factor of 2, the expanded uncertainty is 0.10 g, giving a reporting form of 100.00 g ± 0.10 g. These numbers are stipulated for teaching and are not observations of a real instrument.

The calculation changes if the components are correlated, if a correction has been omitted, or if the calibration interval does not apply to the laboratory conditions. The meaning also changes if the research target is a population mean mass rather than the mass of this specimen. A laboratory report about one sample cannot silently become an inference about all samples of its kind.

This elementary example illustrates the intellectual advantage of a transparent uncertainty budget. The arithmetic is reproducible. The assumptions can be challenged. Different information would alter different terms. A reasoned conclusion can therefore be narrower than the number's attractive precision without being weakened in scientific value. Its reliability consists in keeping the model and evidence visible together with the result.

Sampling, Target Populations, and Selection

Sampling uncertainty depends on how observations enter a study. A random sample from a well-defined frame can support design-based arguments about a population under stated inclusion and response assumptions. A convenience sample can support careful descriptions of observed participants, but an interval calculated as though it were a probability sample does not repair unknown coverage or selection mechanisms. The narrowness of an interval and the representativeness of the sample are different properties.

Missing data create related questions. If a variable is absent for reasons associated with the unobserved value, analysis restricted to complete cases may be systematically distorted. Weighting, imputation, or a selection model can make assumptions explicit and sometimes improve estimation; each is itself a source of model dependence. Declaring a missingness mechanism is a substantive assertion about the data-generating process, not an automatic feature of a software command.

The National Academies reproducibility report (NASEM, 2019) underscores the importance of transparent study conditions and research methods. Inference from samples requires the same documentary discipline: the target population, recruitment dates, inclusion rules, missingness patterns, and analytic exclusions should remain available for critical scrutiny. Generalization is a further argument that begins where an estimate for the observed sample ends. Scientific reports become more informative when they say precisely which population is covered and what evidence supports transport to another one.

Estimators, Standard Errors, and Interval Reasoning

An estimator is a rule mapping observations to a quantity intended to represent a parameter or feature of interest. A standard error describes dispersion of an estimator under a specified sampling model. A confidence interval is constructed by a procedure whose stated coverage is defined across hypothetical repetitions under the assumed model. Once an ordinary frequentist interval has been computed, its endpoints are fixed for the observed dataset; the confidence level characterizes the procedure rather than automatically assigning a posterior probability to the parameter.

These distinctions explain why an interval may be very narrow yet misleading. A flawed sampling frame, differential measurement error, dependence treated as independence, or a misspecified functional form can compromise coverage. The guide to statistical misinterpretations (Greenland et al., 2016) documents how routine interpretations of confidence intervals and tests obscure assumptions. Confidence language is warranted when its conditioning structure is preserved.

A useful interval also conveys effect magnitude. For a treatment difference estimated at 2 units with a reported interval from −1 to 5, the data under the model permit a range from modest harm to substantial benefit. The conclusion depends on what counts as a meaningful difference and on the reliability of the design. Readers need the interval's units, uncertainty source, sampling scheme, and scientific context. An interval interpreted as a set of substantively compatible magnitudes supports better inquiry than one reduced to whether it contains the number zero.

Null Hypotheses, Test Statistics, and P Values

A p value evaluates the extremeness of observed data, or data more extreme according to a specified statistic, under a statistical hypothesis and associated assumptions. It is a conditional calculation about a data model and an analysis rule. It is neither the posterior probability that a hypothesis is true nor the probability that random chance alone produced the data. The ASA statement on significance (Wasserstein and Lazar, 2016) rejects these common substitutions.

A small p value may reveal tension with a statistical model, but the source of tension can be an assumption about sampling, independence, measurement, or a substantive parameter. It does not identify a favored causal theory merely by rejecting a sharp null model. A large p value can arise when a null is approximately correct, when the study has little discriminatory power, or when errors offset one another. Treating failure to reject as proof of equivalence commits a different inferential mistake.

The investigator should therefore specify the precise null, the test statistic, the reason for the tail definition, the stopping rule, the data-selection process, and plausible alternatives. Statistical testing can be a valuable diagnostic when its question is important and its assumptions defended. Its result becomes scientific evidence through interpretation in a larger argument, not through numerical decoration. The appropriate conclusion identifies which model has been challenged, how strongly, and what remains to be investigated.

The Threshold Fallacy and Scientific Gradation

A change in p value from 0.049 to 0.051 may dramatically change the conventional label assigned to a study while changing the evidence only slightly. Such classification manufactures a discontinuity in language that the evidence itself does not contain. The ASA account of statistical context (Wasserstein and Lazar, 2016) cautions against decisions based solely on a conventional threshold. McShane and colleagues' proposal to abandon statistical significance (2019) presses the argument further by subordinating binary significance declarations to substantive evidence.

This does not imply that every scientific decision should be indecisive. Decisions may require explicit cutoffs because actions are discrete, as when a laboratory releases a batch, a clinician selects a treatment, or an agency acts under a deadline. An action threshold belongs to a decision problem with benefits and losses. It must be distinguished from an alleged boundary at which a scientific explanation becomes true.

An instructive comparison concerns two estimated treatment effects, each accompanied by its uncertainty. One estimate passes a statistical cutoff and the other narrowly fails. Comparing the binary labels does not establish that the effects differ. Gelman and Stern's analysis of significance contrasts (2006) makes precisely this point. The scientifically relevant comparison is between the estimated effects, their covariance if applicable, and the uncertainty of their difference. The conclusion becomes more transparent when statistical evidence is described continuously and decision rules are separately justified.

Power, Direction, and Magnitude Errors

Statistical power describes the probability of a prespecified rejection under a particular alternative and design. It therefore depends on assumptions about true effect size, variability, sample size, significance rules, and the analysis method. Low power means that genuine effects may often escape detection. More subtly, estimates selected because they passed a significance threshold can overstate magnitude and sometimes even point in the wrong direction.

Gelman and Carlin (2014) analyze the Type S and Type M errors associated with sign and magnitude exaggeration. Button and colleagues (2013) discuss small-sample problems in neuroscience, including fragile conclusions and inflated reported effects. Their arguments expose a limitation of assessing research design only by the likelihood of achieving a low p value.

The key principle is selection. If a noisy measurement is reported as a discovery only when its magnitude is unusually large, the reported discoveries may be atypical relative to the underlying effects. In a world of small real effects, this can bias the published record toward dramatic but unstable estimates. Increasing power, reporting all planned outcomes, and considering plausible effect sizes before data collection reduce the pressure. A serious design analysis asks what conclusions could be wrong, in which direction, by how much, and with what consequences, rather than merely how often a procedural threshold will be passed.

Selective Discovery and the Probability of a Claim

The reliability of a published finding depends on more than the nominal false-positive rate of a statistical test. It also depends on the prevalence of true relationships in the tested domain, statistical power, selection practices, and distortions affecting reporting. Ioannidis's analysis of false research findings (2005) develops this argument through models of positive predictive value and bias. The paper is a methodological essay built around explicit assumptions, not an empirical measurement proving that a fixed proportion of all published science is false.

An illustrative calculation makes the dependence visible. Suppose that among 1,000 independently tested hypotheses, 100 describe real effects and 900 are null; suppose power is 80 percent and the false-positive rate is 5 percent. Under those stipulated values, 80 true effects and 45 nulls would on average yield positive results. The positive predictive proportion would be 80 divided by 125, or 64 percent. This fraction changes if prevalence, power, dependence, test calibration, or selection changes. It describes the toy scenario, not the scientific literature as a whole.

The philosophical implication is that a result's apparent strength depends partly on the ecology of research questions and incentives. Evidence does not arrive in a vacuum. Prior selection of plausible targets, shared methods, laboratory practices, and publication filters influence what reaches a reader. Scientific judgment should inspect these conditions, especially when results are surprising, small, or highly selected.

Multiplicity, Analytic Flexibility, and Researcher Choice

Multiplicity arises whenever many hypotheses, outcomes, subgroups, models, or analysis variants create opportunities for apparently impressive findings. Some multiplicity is explicit, as in a screen of thousands of candidate associations. Other multiplicity is implicit, as when researchers choose among transformations, exclusion rules, covariate sets, or stopping points after observing data. Both cases complicate the interpretation of a single nominal p value.

Simmons, Nelson, and Simonsohn's false-positive psychology experiments (2011) demonstrate how undisclosed analytical flexibility can inflate false-positive findings. This is a diagnosis of a procedural weakness, not an accusation that all exploratory analysis is defective. Exploration is central to discovering phenomena. The epistemic difficulty arises when a pattern selected through exploration is presented as if it had been the unique prespecified hypothesis tested.

Methods controlling family-wise error or false discovery rates can address declared families of tests under their relevant conditions. A correction cannot reconstruct unreported alternatives that were never logged. Therefore transparent analytic provenance, independent validation datasets, and explicit separation of hypothesis generation from confirmatory testing provide essential support. The researcher must preserve the path by which the reported claim was selected from possible results. When that path remains visible, readers can judge how much of the finding's apparent evidentiary force was supplied by the data and how much by the search procedure.

Preregistration, Exploration, and Confirmatory Discipline

Preregistration records hypotheses, outcomes, sample rules, and analysis plans before relevant results are known. Its epistemic function is to protect the distinction between predicting observations and explaining an observed pattern after the fact. Nosek and colleagues' account of preregistration (2018) describes why that distinction matters to scientific credibility. Preregistration can reduce some forms of unacknowledged flexibility but does not guarantee the validity of measurements or the adequacy of a statistical model.

A useful research report distinguishes planned analyses from justified deviations and explicitly exploratory work. A deviation may be warranted by instrument failure, an unforeseen safety issue, or a newly discovered problem in the protocol. What matters is that the reader can tell when the decision was made and whether it depended on the result. Freezing a demonstrably defective analysis would be a poor substitute for transparent reasoning.

Independent validation and replication complement preregistration. A discovery produced by exploratory search can become a strong claim if later tests expose it to serious opportunities to fail. Such investigation also clarifies what is being replicated: an effect direction, an effect magnitude, a mechanistic relation, or a prediction under specified conditions. Scientific progress benefits when discovery and confirmation form an accountable sequence, with each stage making its own uncertainty explicit.

Bayesian Updating and Its Commitments

Bayesian inference combines a prior distribution with a likelihood for observed data to produce a posterior distribution under a specified hypothesis space and probability model. The update is mathematically coherent within that model, but scientific adequacy requires scrutiny of the prior, the likelihood, the measurement process, and the alternatives included. A posterior probability is a conditional probability, not a certificate that the relevant model set contains every credible explanation.

The attraction of Bayesian reasoning is that it makes prior information and learning from data directly visible. Its central vulnerability is the same visibility when an investigator chooses weakly defended priors or overconfident likelihoods. In fields with accumulated measurements, careful prior elicitation may express substantive knowledge. In poorly understood systems, a sharply concentrated prior can suppress plausible alternatives. Sensitivity analysis reveals how a posterior conclusion depends on these choices.

The statistical interpretation review (Greenland et al., 2016) distinguishes frequentist and Bayesian meanings of familiar outputs. The difference is not reducible to a contest over which interval is more attractive. Frequentist procedures assess repeated-sampling properties under assumptions; Bayesian procedures state conditional distributions incorporating a prior. Both can be useful, and both require a scientifically defensible account of how observations arose. The proper standard of appraisal examines the research question and assumptions before the resulting numerical summary.

Priors, Likelihoods, and the Scope of Posterior Precision

Posterior precision can increase as observations accumulate, but the increase may be illusory when observations share unmodeled dependence or when likelihood assumptions are false. A thousand correlated observations cannot be treated as a thousand independent confirmations merely because they occupy separate rows in a dataset. A Bayesian model that incorrectly assumes independence may produce increasingly concentrated posteriors around an inadequately specified target.

Prior sensitivity also varies with the estimand and available information. When data strongly discriminate among specified alternatives, a broad range of reasonable priors may lead to similar posterior summaries. Under weak identification, priors can remain influential even in a large dataset. Reporting a narrow credible interval without explaining the alternatives against which it was obtained invites an inferential overreach.

The decisive philosophical issue is conditionality. Bayes' theorem is an identity within probability theory. Its scientific use requires arguments about the possible explanations, the process generating data, and the interpretability of the likelihood. A numerical posterior may therefore be correct as a calculation while the resulting conclusion remains incomplete as an account of reality. This observation applies equally to highly structured models and to seemingly simple comparisons. In both cases the investigator should ask what assumption, if relaxed, would materially change the claimed result.

Posterior Predictive Assessment and Model Checking

Once a model has generated a posterior distribution, it can be used to predict observations or summaries that might be compared with the actual data. The exercise asks whether the model reproduces scientifically relevant patterns, not merely whether numerical parameters possess impressive posterior precision. Checks may investigate tails, dependence, subgroup differences, or unmodeled temporal structure. Their credibility depends on choosing discrepancies with genuine bearing on the substantive question.

A model may describe a grand average adequately while missing rare but consequential events. Another may fit observed data closely after excessive flexibility and fail when predicting new observations. Both problems show why internal fit and external predictive performance should be kept distinct. The researcher's purpose determines whether an inadequate tail fit, an inaccurate group contrast, or a poorly calibrated predictive interval is most damaging.

Model checking also creates the need for revision. After observing failures, an investigator may modify likelihoods, add interactions, reconsider measurement assumptions, or challenge the underlying explanatory story. This is legitimate scientific learning. Claims about how well a model survived a diagnostic must record whether the diagnostic was chosen before or after the revision. An assessment performed on genuinely new data carries a different evidentiary function from a goodness-of-fit measure optimized on the original sample. The resulting research record should preserve both the path of discovery and the independent testing that follows it.

Likelihood Evidence and Rival Explanations

Evidence often has a comparative structure: an observation is more expected under one proposed account than another. Likelihood ratios express that comparison between specified statistical models. They can be informative without presenting an isolated p value as a verdict on substantive truth. Nevertheless, a likelihood ratio comparing two models tells us about those models and their assumptions, not about every theory that might explain the evidence.

The problem is visible in scientific diagnosis. An observed test result may be more likely if a disease is present than absent, but a patient's probability of disease also depends on prevalence and other information. Likewise, a signal may be more likely under a proposed physical process than under a simple noise model, yet an unconsidered interference process can change the interpretation. Inference becomes misleading when the available comparisons are mistaken for an exhaustive account of reality.

Stanford's analysis of scientific underdetermination (2023 revision) distinguishes limited evidential underdetermination from sweeping claims that all theories are equally supported. This distinction matters because model criticism does not license indifference among explanations. Some explanations predict more, require fewer unsupported auxiliary claims, or survive stronger independent tests. Comparative evidence can rationally favor one account while remaining open to future rivals. Its force comes from both the tested contrast and the historical record of attempts to improve that contrast.

Model Misspecification and Structural Uncertainty

Structural uncertainty concerns the possibility that the functional form, dynamics, causal relations, or conceptual categories of an analysis are wrong or incomplete. It differs from uncertainty about numerical parameters within an accepted model. A confidence or credible interval may account thoroughly for parameter variation while leaving unmodeled structural alternatives unrepresented. This distinction becomes especially important when a model is used beyond conditions under which it was developed.

Oreskes, Shrader-Frechette, and Belitz (1994) examine the limits of validating natural-system models. They argue against treating successful model comparison as exhaustive verification of open natural systems, emphasizing incomplete access and nonunique explanations. Their claim is most useful as a warning about what finite tests can establish, rather than as a reason to abandon modeling.

Researchers can make structural uncertainty more tractable through alternative model specifications, external validation, sensitivity analysis, mechanism-specific tests, and study designs that discriminate among rivals. These procedures increase the evidentiary reach of a model assessment. They do not transform a finite set of alternatives into proof that no other explanation exists. Thus the proper model claim specifies the domain in which performance was evaluated, the classes of alternatives examined, and the features that remain uncertain. A strong scientific conclusion is often an explicitly bounded one that survives serious alternatives rather than a globally asserted one whose calculation happens to be precise.

Causal Identification and Counterfactual Questions

Causal inference asks what would happen under an intervention or exposure different from the one observed. It must therefore connect observations with unobserved counterfactual outcomes through assumptions or research design. Randomized experiments can support some causal contrasts under compliance and other conditions. Observational studies can identify effects through assumptions about confounding, consistency, positivity, measurement, and temporal order. These assumptions belong to the scientific claim itself.

Hernán and Robins (2020) develop a comprehensive account of causal identification using counterfactual reasoning and research designs. Their work makes clear why statistical association, even if estimated with minimal sampling error, does not automatically establish a causal effect. Two mechanisms can generate the same correlation while implying different interventions. A precision estimate conditional on one assumed causal graph is insufficient evidence that the graph is correct.

The consequence is methodological. A report should state the treatment or exposure, the outcome, the target population, the timing, the intervention of interest, and the assumptions required for identification. It should examine possible confounding, selection, measurement error, and interference. Scientific uncertainty concerns both how well an identifiable effect is estimated and whether the effect was identified in the first place. These questions require distinct evidence and distinct responses. Collecting a larger observational sample may reduce one source of uncertainty while leaving another structurally unresolved.

Confounding, Misclassification, and Information Loss

Confounding arises when the observed association between an exposure and outcome reflects differences that also affect the outcome. Statistical adjustment may help if relevant factors are adequately measured and the adjustment set is justified. Unmeasured confounding can remain even after an apparently sophisticated model has been fitted. Conditioning on inappropriate variables may introduce new bias instead of removing it.

Misclassification adds a different problem. A diagnostic label, administrative code, or survey answer may imperfectly reflect the construct or event under investigation. If the error process varies with exposure or outcome, the resulting association may move in unexpected directions. A simple attenuation rule cannot be assumed universally. Selection into the recorded dataset and loss of observations can compound this problem by altering the relation between the observed and target populations.

The causal research framework of Hernán and Robins (2020) provides a basis for specifying such assumptions. The analytic implication is that measurement uncertainty and causal uncertainty need to be evaluated together while retaining separate identities. A result can be numerically stable under alternative regression specifications and still rely on the same unmeasured assumption. Robustness to computations sharing the same missing premise is weaker than robustness to independent research designs that challenge that premise. Strong causal inference asks how an alternative data-generating mechanism could produce the reported association and what observation might distinguish it.

Prediction, Calibration, and Changing Populations

Prediction concerns future or unobserved outcomes under stated conditions. An accurate predictor need not identify causal mechanisms, and a credible causal account need not produce highly precise individual-level forecasts. Forecast errors also depend on the population and time period in which a model is applied. A model trained on one group may transfer poorly when underlying relations, measurement procedures, or base rates change.

Calibration asks whether probabilities assigned to events correspond appropriately to event frequencies under a specified evaluation regime. Discrimination asks whether a model distinguishes cases with different outcomes. These are related but separate performance criteria. A model can rank individuals effectively while giving systematically overconfident probabilities. Prediction intervals can be narrow because the model omits process variation rather than because the world has become more predictable.

The same reasoning applies to physical simulation and public risk estimates. Inference should identify the domain of application, the validation design, and the consequences of shifts in input distributions or causal mechanisms. Out-of-sample testing provides information about robustness, but its relevance depends on how closely test conditions resemble future deployment conditions. A policy judgment based on predictive output requires one more passage: from expected outcomes to the costs and benefits of action. The model's reported confidence should never be mistaken for an unconditional mandate to act.

Simulation and Sensitivity Analysis

Simulation makes it possible to investigate the behavior of estimators, uncertainty propagation, and scientific decision procedures under explicitly stipulated mechanisms. One can vary sample size, error distributions, dependence, missingness, model structure, or prior assumptions and observe how results respond. The JCGM Monte Carlo supplement (2008) illustrates a formal use in measurement uncertainty propagation, while broader simulations examine the operating characteristics of inferential procedures.

Simulation is most informative when the range of scenarios reflects credible possibilities and when the design is fully reported. A model that performs well under the single mechanism chosen by its designers has passed a limited test. A systematic sensitivity analysis asks which assumptions drive the conclusion and identifies thresholds at which it changes. Such thresholds can reveal where new measurements or independent observations would be most valuable.

Consider a hypothetical study with a positive estimated association. A sensitivity analysis might ask how strong an unmeasured common cause must be to explain the result, how much misclassification would alter its sign, or which plausible prior would change the posterior probability of an effect. The output does not prove that the damaging alternative is true. It shows whether the conclusion is robust to a specified class of departures. An evidentiary assessment should distinguish computational verification of simulation code from empirical validation of the simulated mechanisms. Both are necessary for stronger use, and neither substitutes automatically for the other.

Replication, Reproduction, and Cumulative Criticism

Reproducibility and replication occupy different stages of scientific evaluation. The National Academies' 2019 definitions use reproducibility for obtaining consistent computational results using the same data, code, steps, and conditions; replication concerns consistent results across studies that gather their own data to answer the same question. Under these definitions, successful computational reproduction checks one chain of transformation, while independent replication examines a wider evidentiary relation.

The Open Science Collaboration investigation (2015) assessed the replicability of a set of published psychological findings under a specific protocol. Its empirical results should be interpreted within the tested corpus and criteria, not as a universal replication rate for all scientific disciplines. Replication outcomes also require attention to sampling variability, changes in context, effect heterogeneity, and the definition of a comparable result.

A failure to reproduce a computation suggests problems with information, code, environment, or analytic reporting. A failure to replicate an effect may suggest a false original claim, a genuine contextual dependency, differences in measurement, or new errors in the replication. These explanations require investigation. Scientific correction is cumulative when the reasons for disagreement become more explicit and the next study is designed to distinguish them. Mechanical repetition without diagnostic insight yields less than critical repetition with a clear target.

Evidence Synthesis and Heterogeneity

Meta-analysis combines results across studies under assumptions about effect definitions, sampling distributions, dependence, and heterogeneity. Pooling can improve precision when studies address a sufficiently comparable question. It can also conceal incompatible populations, measurements, interventions, or designs. A precisely estimated mean across scientifically incomparable quantities may have little substantive meaning.

Heterogeneity is not simply noise to be removed. Treatment effects may vary with dosage, demographic setting, baseline risk, follow-up duration, or implementation. Measurement protocols can differ; publication selection can favor dramatic results; shared datasets can create dependence among apparently separate findings. An appropriate synthesis reports the decision rules used to select studies and considers whether the target estimand remains coherent after comparison.

The lesson from selective finding models (Ioannidis, 2005) and design-related effect exaggeration (Gelman and Carlin, 2014) is that aggregation inherits the weaknesses of the studies entering it. Statistical pooling cannot retrospectively supply missing measurements or independent validation. Heterogeneity analysis, risk-of-bias assessment, and sensitivity to study inclusion can make a synthesis more credible. The strongest aggregate claim explains why the underlying studies answer a shared question, which alternatives were tested, and how uncertainty changes under reasonable choices of inclusion and model.

Equivalence, Negligibility, and Negative Evidence

A result that fails to cross a significance threshold does not demonstrate that an effect is absent. It may be too imprecise to distinguish an effect of scientific importance from zero. In many settings the relevant question is whether an effect is sufficiently small to be negligible for the purpose at hand. This requires defining a region of practically unimportant effects before interpreting the data.

Lakens (2017) gives a practical account of equivalence tests in which the investigator specifies equivalence bounds and tests whether the data justify rejecting effects outside that region under the chosen procedure. The bounds express a substantive judgment about importance, supported by scientific context or decision requirements. Their choice is as consequential as the statistical calculation.

An illustrative comparison exposes the difference. Suppose a new method is estimated to differ from a reference by 0.2 units, with a confidence interval from −2 to 2.4. If differences greater than 0.5 units matter, the study cannot establish equivalence even though the estimate is near zero. If a justified tolerance is ±3 units, the same interval may support a different conclusion under an appropriate equivalence procedure. The evidence has not changed; the scientific question has. Negative evidence becomes meaningful when a design had a genuine opportunity to detect the differences that matter. A report should state what has been ruled out, under which assumptions, and what remains compatible with the data.

Inductive Risk and the Consequences of Error

Scientific conclusions become inputs to action under circumstances in which mistaken acceptance and mistaken rejection may have unequal costs. A missed hazard, a false alarm, an ineffective treatment, and an unnecessary intervention create different kinds of consequences. The relevant risk depends on the decision and affected population. Thus there is no context-free numerical threshold that automatically converts uncertain evidence into the right action.

Inductive risk enters scientific reasoning when the consequences of error affect the standards for accepting, rejecting, or qualifying a claim. The investigator may require especially strong evidence for a serious safety assertion while permitting more exploratory work when a false lead is cheap and easily reversible. Such judgments should be explicit. Concealing them behind familiar conventional significance levels makes evaluative decisions appear to have emerged from the data alone.

The metrological conformity assessment guidance (JCGM, 2012) illustrates the relevance of measurement uncertainty when deciding whether results satisfy requirements. More generally, decision makers must distinguish an uncertainty statement from a loss function and a regulatory or ethical rule. The scientist contributes to the decision by describing the evidence, its limitations, and the sensitivity of possible actions to uncertainty. Explicit articulation of value and loss does not dissolve the empirical question. It permits the scientific estimate and the normative judgment to be examined on their respective merits.

Decision Thresholds and Asymmetric Loss

Suppose a regulator must choose between immediate inspection and continued operation of an industrial installation. Let the uncertain event be a failure within a specified period. The cost of an unnecessary inspection may be modest compared with the damage from an undetected failure. A rational threshold for intervention could therefore be much lower than the level of confidence appropriate for publishing a definitive claim that failure will occur. The numerical threshold should follow from the stipulated consequences and available actions, not from an inherited statistical convention.

A contrasting decision might involve a costly and irreversible intervention with uncertain benefits. Here the appropriate evidence standard may be more demanding. In each case, a transparent decision analysis identifies possible actions, probabilities or ranges judged relevant, costs of error, and the sensitivity of the preferred action to modeling assumptions. The framework makes disagreements discussable: investigators may disagree about probabilities, social losses, or both.

Measurement and conformity standards emphasize this practical need for explicitly related uncertainty and acceptance decisions. The JCGM account of conformity assessment (2012) concerns a specific domain of such decisions. The wider philosophical conclusion is that an action can be justified before a scientific claim has become certain, while an exact-looking estimate can remain insufficient for action when critical costs or mechanisms have been omitted. Warrant for belief and warrant for action are connected but answer different questions.

Transparency, Provenance, and Computational Mediation

Contemporary scientific results increasingly pass through software libraries, data cleaning rules, model selection routines, automated transcription, statistical pipelines, and computational representations. Each transformation can be inspected for errors only to the extent that its inputs, versions, assumptions, and outputs have been documented. A reproducible pipeline can establish that an analysis was performed consistently; it does not by itself establish that the data were authentic, that the statistical assumptions were sound, or that the underlying scientific question was well framed.

The National Academies' account of computational reproducibility (2019) treats access to data, methods, code, and conditions as central to this task. A complementary philosophical question concerns authorial and documentary provenance. The Aisentica project's account of Artificial provenance explicitly presents provenance as a category of origin, authorship, and public trace within its wider Artificial Era philosophy. In this monograph that account is identified as an attributed philosophical position, rather than used as evidence that any computational tool has scientifically validated a result.

A robust research record can therefore retain multiple kinds of trace: instrument calibration, sample collection, source acquisition, data transformation, analytical choices, and authorship. The links among them improve criticism because a disputed claim can be traced to the step at which uncertainty or error entered. Machine-generated interpretations require the same source-specific checks as human-authored ones. Their prose fluency, apparent confidence, and repeatability of output are insufficient substitutes for independently inspectable evidence.

Communication Without False Precision

Scientific communication is part of inference because readers make decisions on the basis of the claims and uncertainty that are reported. Saying a treatment "works" can conceal variation in outcome, population, time horizon, and effect magnitude. Saying there is "no effect" can conceal a wide interval that remains compatible with both substantial benefit and harm. A claim needs a target, an estimated magnitude or relevant comparison, and a description of the uncertainty that matters.

The ASA guidance on interpreting p values (Wasserstein and Lazar, 2016) and Greenland and colleagues' analysis of statistical misreadings (2016) support a practical editorial discipline: report continuous results where possible, disclose assumptions, and preserve relevant context. Graphical uncertainty displays should label what their intervals represent. Numerical precision should reflect actual measurement and modeling limits; an output rounded to six decimals need not warrant six-decimal substantive claims.

The audience also matters. Specialists may need covariance structures and model specifications. Public decision makers need interpretable outcome ranges and the consequences of error. A useful summary communicates uncertainty while preserving a route to the technical evidence. It is possible to be clear and qualified at the same time. The scientific virtue of such communication is its fidelity to the actual evidence: it enables another person to see how the conclusion could be challenged and what observations would make a practical difference.

Limitations of Formal Aggregation

There is a temptation to seek one composite score for the reliability of a scientific claim. Measurement uncertainty, sampling variance, prior sensitivity, selection bias, causal identification, model error, and research integrity can each be assessed, but they need not possess a common probability scale or an additive structure. Combining them without a justified model risks masking qualitative assumptions beneath numerical form.

Some uncertainties can be propagated. Others require a different design, a comparison of theories, an independent data source, or a more precise specification of the target. The model-validation critique of Oreskes and colleagues (1994) illustrates the limits of extrapolating finite model agreement into exhaustive claims about open systems. The underdetermination analysis by Stanford (2023) adds that evidence may leave more than one serious explanation in contention without rendering all explanations equally defensible.

The appropriate response is a structured narrative of uncertainty aligned with the research question. A report can identify quantified components, unquantified structural concerns, rival hypotheses, external validation, and potential costs of being wrong. This is a stronger practice than inventing a single unsupported numerical probability that everything in the analysis is correct. Aggregation is valuable where its assumptions can be defended. Where those assumptions fail, explicitly heterogeneous evidence should remain heterogeneous in the final account.

The Philosophical Significance of Remaining Uncertainty

Scientific reason is capable of concluding from imperfect evidence because justification is comparative and revisable. The investigator can show that one account predicts observations better than a rival, that one measurement is more traceable, that a treatment effect is bounded within a scientifically meaningful range, or that an intervention remains prudent across plausible scenarios. Such conclusions have genuine evidential content even when their conditions are articulated in full.

The central limit of inference is reached when an assertion exceeds the support provided by its measurements, design, assumptions, and comparisons. A numerical interval may warrant a statement about an estimated parameter but not about causal mechanism. A replicated result may warrant an account of robustness under tested conditions but not a universal law of nature. A successful forecast may support practical planning while leaving explanatory identification uncertain. The criteria are domain-specific because the inferential passages are different.

The original synthesis of this monograph is a comparative account of these passages: scientific uncertainty should be tracked through the specific warrant that connects an operation to a conclusion. The idea draws together metrology, statistical criticism, and philosophy of models without being presented as a newly established theory or an independently validated method. The result is a more accountable form of confidence. One can state what the evidence supports, identify which assumptions matter, show which alternatives were investigated, and explain which new observation would demand revision. Scientific authority grows through that public capacity for correction.

Conclusion

Error, uncertainty, and scientific inference are linked by a common demand: every conclusion must preserve its relation to the evidence and assumptions capable of supporting it. Measurement uncertainty concerns quantities and their evaluated dispersion. Statistical uncertainty concerns estimators and probability models. Causal uncertainty concerns identification and the counterfactual interpretation of observational or experimental contrasts. Model uncertainty concerns the adequacy and range of represented mechanisms. Decision uncertainty concerns which errors matter when knowledge is converted into action. These dimensions intersect in actual research but require different methods and different standards of appraisal.

The investigation establishes six substantive conclusions. First, the precision of a numerical result is conditional on a correctly specified target and evidentiary procedure. Second, random variation and systematic distortion require distinct investigation; repeated measurement cannot eliminate a shared bias. Third, significance tests and posterior summaries acquire scientific meaning through their models and the process by which hypotheses and analyses were selected. Fourth, causal and structural assumptions can remain unsettled after numerical estimation becomes highly precise. Fifth, computational reproduction and independent replication perform different epistemic functions and become stronger when discrepancies lead to explicit rival explanations. Sixth, the consequences of error shape rational decisions, while the underlying empirical estimates remain open to independent criticism.

These conclusions do not prescribe one universal inferential algorithm. They identify the reason a universal numerical shortcut would fail: the sources of uncertainty operate at different levels of the investigation. An uncertainty budget, a confidence interval, a posterior distribution, a causal diagram, an alternative model, and an audit trail can each contribute to warrant. Their success depends on the question asked and the evidence available. The scientist's task is to preserve the continuity between a clearly stated target, the observations brought to bear, the assumptions used to interpret them, and the eventual claim.

Scientific inquiry advances when an apparently adverse finding can be explained, a hidden error located, a claimed effect narrowed, or a rival model shown to perform better. Remaining uncertainty is then a description of what is still epistemically open, rather than a defect to be concealed. The authority of science resides in the reliability of its procedures, the discriminating force of its evidence, and the possibility of reasoned correction. A conclusion is warranted when its strength is proportionate to that evidence and its limits are presented as part of the conclusion itself.