RESEARCH DOSSIER

Simulation and Scientific Computation

Author: Angela Bogdanova · Published: November 9, 2026 · Editorial Policy · Written by Artificial Sapiens

Abstract. Scientific simulation converts mathematical, statistical and rule-based representations into executable investigations. This monograph asks when the resulting outputs justify claims about the world. It reconstructs philosophical disagreements over simulations as models, experiments, instruments and inference; distinguishes discretization, stability, convergence, floating-point error, random sampling, calibration, verification, validation, uncertainty quantification and reproducibility; and compares documentary cases from nonlinear dynamics, agent-based social modeling and climate-model intercomparison. The governing argument is that a computed outcome warrants a target-world claim only through a traceable relation among intended purpose, representational assumptions, operational correctness, empirical constraints, uncertainty and domain of use. Monte Carlo estimates can be exacting answers to conditional mathematical questions while retaining uncertain empirical premises. Agent-based simulations may demonstrate generative possibilities without identifying actual social causes. Large climate ensembles can support scientifically evaluated conditional projections without converting emissions scenarios into probability forecasts of human choice. The dossier proposes no new named theory or purported empirical discovery. Its original contribution is a source-grounded critical synthesis, a comparative evidentiary matrix and an explicit argumentative sequence for interpreting simulation results. Source limitations, including reliance on publisher abstracts for some restricted works, are documented. The conclusion establishes the conditions under which computational outcomes become accountable evidence rather than visually persuasive numerical assertions.

Keywords: Computer simulation; scientific computation; philosophy of science; numerical analysis; verification and validation; uncertainty quantification; Monte Carlo; agent-based models; numerical convergence; climate models; epistemic opacity; reproducibility; scientific evidence.

Introduction: The Epistemic Standing of Computed Worlds

Simulation occupies a central position in contemporary scientific practice because computationally implemented models can be run, perturbed, and compared at scales inaccessible to unaided analysis. Yet an output is first an output of an implemented representation. Its relation to a physical, biological, or social target must be established through additional reasoning. The philosophical question is therefore exact: under what conditions does a computed result count as evidence about the world? The 2026 simulation survey distinguishes an equation-solver conception from a broader scientific practice that includes modeling, implementation, interpretation, and assessment. The distinction makes the evidentiary problem visible at the outset.

The central argument of this dossier is that simulation evidence is warranted through a traceable relation among a research question, a target and idealization, mathematical and computational implementation, error control, relevant empirical information, and a scope-qualified interpretation. These dimensions form a critical reading sequence assembled from existing scholarship rather than a newly named theory. A simulation can produce a logically instructive counterexample, expose conditional consequences, guide experiment, aid prediction, or support a high-stakes decision. Different purposes require different kinds and strengths of warrant. The rigorous philosophical treatment must accordingly preserve the contrast between internal consequences of assumptions and claims grounded in observations of an external target.

The Scope of Scientific Computation

Scientific computation includes numerical solution of equations, optimization, statistical estimation, stochastic sampling, large-scale data analysis, and computational exploration of formal models. Simulation is a prominent subset that studies the behavior, structure, or distribution of a model through computational procedures. The boundaries are purposive: evaluating an integral by randomized sampling need not imitate a physical process, while solving fluid equations for evolving atmospheric states can simultaneously serve calculation, representation and prediction. The survey by Durán (2018) emphasizes this diversity of roles across scientific and engineering practice.

It follows that a philosophy of simulation should begin with an account of what scientists do rather than with a simple label attached to software. A model may be an equation system, an algorithm specifying interacting agents, a probabilistic law, or a coupled family of component models. A run is a particular execution under specific conditions. A study includes the selection of variables and questions, design of runs, analysis of outputs, and decisions about claims. Philosophical conclusions drawn from a run alone risk overlooking the warrant-bearing work performed before and after computation. The scholarly term simulation study usefully preserves that encompassing practice, as developed in Winsberg's 2003 analysis.

What Is Simulated: Target, Model, and Execution

Three terms organize a clear account. The target is the system or phenomenon about which a question is asked. The model is the representation selected to address that question. The execution is a concrete computational process implementing rules and producing outputs. A fourth element, interpretation, states which computed quantities are to be compared with target-world quantities. Confusing these elements can make a programmer's successful implementation look like confirmation of a physical claim. The computational-template analysis of Humphreys (2004) demonstrates why the move from generic computational structure to specific scientific application requires substantial work.

Consider a river-flood model as an illustrative construction, not a reported experiment. The target is a specified river reach under specified rainfall and management conditions. The physical model includes conservation assumptions and parameterizations of roughness. The algorithm selects discrete cells and time steps. The execution produces stage estimates; an interpretive mapping turns them into claims about inundation areas. Each transition is contestable in a different way. The model may omit a levee, the numerical scheme may lose accuracy, the input data may be poor, or a decision maker may extrapolate from one rainfall regime to another. The model, code, run, and claim should therefore remain separately identifiable in the evidentiary record.

Theory, Models, and Scientific Representation

The familiar suggestion that a simulation simply deduces what a theory predicts presupposes an exceptionally direct path from universal equations to a locally useful calculation. Scientific practice involves scale selection, boundary conditions, parameterization, closure relations, experimental calibration, and choices shaped by computational feasibility. Frigg and Reiss (2009) challenge inflated novelty claims about simulation while leaving intact the older philosophical problems of representation, inference and model justification. Their argument is most usefully read as a demand to show exactly what is distinctive about the computational case.

Scientific representation is selective: a model can stand for some target aspects for a specific purpose while neglecting or distorting others. The success of a flight-control simulation does not automatically establish its adequacy for turbulence at every scale; the success of a population model in reproducing a stationary distribution need not establish that its transition rules are causal descriptions of organisms. As an analytical synthesis, the evidential object is the pairing of an output with an explicitly stated target claim, not the output by itself. This view keeps mathematical consistency, representational adequacy and empirical appraisal distinct so that each may be assessed on its own terms.

The Debate over Simulation and Experiment

Computer simulation is often called a numerical experiment because researchers manipulate inputs, execute controlled comparisons and observe output patterns. Winsberg (2003) argues that simulation shares important methodological features with experimentation while retaining its ancestry in theoretical modeling. Beisbart and Norton (2012) press a stricter inference-centered interpretation of Monte Carlo calculations: introducing random numbers does not in itself grant empirical access beyond the modeled assumptions. The disagreement concerns sources of warrant rather than the vocabulary of laboratories.

The practical resolution requires separating controlled intervention within an artifact from intervention upon a target. Changing the simulated viscosity can establish how an implemented model responds. It establishes how the fluid itself responds only insofar as the model-world correspondence has additional support. A physical experiment can directly disturb a specimen; a numerical experiment manipulates a computational representation. Both involve error, material instruments and inferential interpretation. Neither is epistemically guaranteed by its name. The relevant question is which causal or empirical connections support an inference across the boundary from simulated state to target-world claim. Calling a simulation an experiment may illuminate its design discipline, but the label cannot perform the validation that the argument needs.

Materiality, Instruments, and Methodological Independence

A running computer is a physical system; its logic gates, memory, compiler and storage devices participate in what results appear. Parker (2009) shows why appealing to the materiality of experimental and computational arrangements alone does not settle their comparative epistemic authority. The physicality of a processor establishes that a computation occurred, while the scientific question remains what it measured or represented. A simulation can be materially realized and still be miscalibrated for the target it purports to address.

An instrument-oriented reading is nevertheless productive. Instruments are designed, calibrated, repaired, compared and operated under specified protocols. Software, algorithmic methods, hardware, datasets and research teams form a comparable structure of dependent expertise. The current SEP account of simulation instruments considers how that perspective expands the older model-versus-experiment debate. The interpretive consequence is that computational outputs may be evidence when grounded by well-documented instrument performance in an intended domain. This criterion accommodates both model-mediated observations and simulations that remain primarily exploratory. It asks for operational provenance and justified scope rather than a metaphysical verdict about whether computation is really theory or really experiment.

Discretization and the Construction of Approximate Worlds

Most numerical simulations replace continuous relationships with finitely representable approximations. A differential equation can be translated into a grid of values, a time-stepping rule, and boundary procedures. This translation introduces a new object of analysis: the discrete system, which may have behavior of its own. Even when the continuum model is mathematically sound, its numerical representation can introduce dissipation, instability, grid dependence or spurious oscillations. The classical Lax–Richtmyer investigation treats the relationship among consistency, stability and convergence under stated assumptions for well-posed linear problems.

A fine mesh is not an independent proof of physical truth. Refinement can reveal whether an algorithm is approaching a continuum result, subject to the method and the regime examined. It does not establish whether the continuum equations are adequate descriptions of the target, whether the boundary conditions are accurate, or whether a quantity of interest remains sensitive to unresolved subgrid processes. This creates two distinct bridges: from discrete calculation to mathematical model, and from mathematical model to target. The first is addressed by numerical analysis; the second requires empirical and representational appraisal. Reliable scientific claims require attention to both bridges, since correctness at one cannot substitute for justification at the other.

Consistency, Stability, and Convergence

Consistency concerns whether a discretization approximates the intended differential problem as discretization scales shrink; stability governs the amplification of disturbances during computation; convergence concerns approach to the appropriate solution. In their specified linear, well-posed initial-value setting, Lax and Richtmyer (1956) establish an equivalence linking stability and convergence for consistent difference schemes. That theorem is often treated too casually when carried into highly nonlinear, coupled, stochastic or poorly posed scientific applications. Its hypotheses are substantive conditions of relevance, and the formal result cannot be quoted as a blanket guarantee for every simulator.

In a constructed example, a temperature solver might produce visually smooth images while a hidden time-step instability amplifies numerical disturbances. Conversely, a stable scheme can converge consistently to equations that omit a consequential heat source. Graphical smoothness is thus a weak diagnostic. The correct audit asks what problem is discretized, what regularity and well-posedness assumptions hold, what stability analysis applies, which refinement studies have been performed and which output functional matters. The interpretation must state whether convergence was mathematically proven, numerically observed or merely assumed. A credible conclusion may be local: this solver converged within a tested grid regime for a specified observable. Such precision is a scientific strength.

Floating-Point Arithmetic and Computational Error

Real-number mathematical expressions are implemented by finite-precision arithmetic on digital machines. Rounding, cancellation, conditioning and the order of operations can change computed results, sometimes qualitatively when a problem is sensitive. Higham's numerical analysis develops systematic tools for understanding this relation between exact problems and numerical outputs. The distinction is crucial for simulations in which large numbers of operations, iterative solvers, and coupled components accumulate effects that are hard to inspect by looking at final figures.

Reproducibility at the bit level and accuracy with respect to the underlying target are separate virtues. Two machines can reproduce the same numerical answer if their implementations are controlled, yet that answer may be inaccurate. Alternatively, slightly different roundoff paths can produce different trajectories in a sensitive deterministic system, even while preserving accurate distributions or invariant properties. Verification should therefore identify relevant arithmetic environments, precision settings, stopping tolerances, and comparison metrics. It should explain whether the epistemic object is one trajectory, a distribution, a long-time average or a qualitative bifurcation structure. The demand for such specification follows from the kind of claim being made rather than an indiscriminate preference for ever more numerical precision.

Chaos, Determinism, and Predictive Horizons

Lorenz's deterministic nonperiodic flow study demonstrated within an idealized set of nonlinear equations that nearby initial conditions can lead to sharply differing subsequent trajectories. This result complicates an equation-to-forecast conception of knowledge. Deterministic governing rules may be fully specified while finite measurement precision limits the duration over which detailed trajectories remain practically predictable. The problem is not confined to computer bugs: it arises from the dynamics represented by the mathematical system and the precision with which a state can be known.

The philosophical error is to infer that a deterministic model must yield indefinitely precise predictions about the world, or that sensitivity invalidates all modeling knowledge. Predictions may target statistical features, attractor structure, bounded ranges, or short windows of useful detail. The intended observable determines which uncertainties are decisive. A long-run spatial pattern can remain robust while individual paths diverge, and an ensemble distribution can be scientifically informative when a single forecast is unstable. The strongest claim is accordingly conditional: for a specified purpose and forecast horizon, the modeled result can be supported by error analysis and comparison. Simulating more decimal places is not a substitute for observationally constrained initial conditions or a sound choice of predictive target.

Randomness, Monte Carlo Inference, and Sampling Error

Monte Carlo methods use pseudorandom or random draws to estimate quantities such as integrals, distributions, rare-event probabilities, and expected responses. The historical Metropolis–Ulam article helped establish this computational approach. The method's evidentiary meaning depends on what is sampled and why. An algorithm may introduce randomness solely as a numerical convenience even when the original mathematical object is deterministic. In that case the simulated variability is part of a computational estimator, not automatically evidence that the target system behaves stochastically.

Beisbart and Norton's argument clarifies this point by treating Monte Carlo computation as inferential processing of presupposed information. The estimator itself can be analyzed through bias, variance, correlation, sample size and convergence. Yet sampling accuracy does not independently establish the truth of input probability distributions, choice of prior, treatment of extreme outcomes or correspondence with the target. Random seeds support procedural repeatability; sensitivity to seeds should be reported when outcome claims depend on it. An adequately converged estimator can deliver a precise estimate of a misspecified mathematical expectation. The inferential record must therefore contain two evaluations: estimation error conditional on the assumed model and justification of the model for the intended empirical purpose.

Minimal Models, Idealization, and Explanation

Scientific explanations sometimes succeed because a model eliminates details that do not control the phenomenon under investigation. Batterman and Rice (2014) argue that minimal models can explain why heterogeneous systems exhibit similar large-scale behavior when fine-grained differences are irrelevant to that pattern. This philosophical lesson prevents a general insistence that maximal physical realism produces maximal understanding. A stripped-down simulator can reveal invariances, scaling behavior or constraints that an overloaded simulation conceals.

Idealization has an epistemic price that must be justified in terms of the explanandum. Omitting an interaction is warranted when the result remains stable across relevant variations or when independent theory establishes its irrelevance for the chosen question. It is unwarranted when the omitted factor governs the target property. An illustrative model of crowd movement with identical agents may be sufficient to explore some bottleneck geometries and inadequate for predicting evacuation behavior among populations with diverse mobility constraints. The appropriate test is explanatory relevance, not surface resemblance. The strongest simulation studies justify their omissions by showing what changes when assumptions are relaxed, how much changes and why that magnitude matters for the exact conclusion offered.

Intervention, Counterfactuals, and Causal Interpretation

Simulations allow controlled variation of input factors and hypothetical policies, making them attractive for counterfactual reasoning. A model can answer what follows if a parameter is changed while other modeled relationships remain fixed. That is a legitimate conditional claim. A causal claim about the external world additionally requires that the altered parameter corresponds to a feasible intervention and that the relevant mechanisms are adequately represented. Winsberg and Alvarado (2026) place computational interventions within broader debates over experimentation and evidentiary entitlement.

The contrast matters in a public-health illustration. Changing a simulated contact rate can map consequences within an epidemic model; whether a particular policy would realize that rate depends on behavior, compliance, institutional context and data that the equation alone may not contain. A simulation cannot identify a policy's actual effect through arithmetic alone. Its strength lies in exposing conditional structures and making sources of disagreement explicit. One should ask which relations are structural assumptions, which parameters are estimated, which can be experimentally controlled and which can drift in response to interventions. Counterfactual credibility arises when the model's relevant causal pathways and boundary conditions have support within the intended domain.

Computational Opacity and Distributed Expertise

Large computational studies can contain millions of interacting operations, code libraries, datasets and modeling decisions beyond the unaided comprehension of any single investigator. Philosophers describe the resulting challenge as epistemic opacity: a scientist may have reasons to rely on a result without being able to inspect every relevant computational step. Humphreys (2004) relates advanced computation to expansion of scientific cognitive capacities, while the 2026 SEP discussion reviews competing responses, including process reliability and instrument-based accounts.

Opacity comes in degrees. A small transparent script may have a complex mathematical error; a large, well-tested codebase may provide trustworthy results for established tasks. Trust should be attributed to particular processes and supported by independent diagnostics, benchmark cases, convergence tests, audits, versioned dependencies and documented empirical comparisons. Appeal to expert consensus cannot replace the ability to investigate what fails when a model is pushed beyond its known range. The practical result is a division of epistemic labor: mathematicians test algorithms, domain researchers question representation, software engineers assess implementation, statisticians quantify uncertainty, and decision makers inspect use conditions. Credibility depends on the quality of these interfaces as well as each component.

Verification: Solving the Intended Equations

In computational science, verification generally asks whether the implementation correctly solves the intended numerical or mathematical problem to the specified accuracy. This question can include code verification, solution verification and tests of convergence. the National Research Council's assessment places verification alongside validation and uncertainty quantification as interdependent elements of credible modeling. It is important to distinguish verification of code from empirical validation of the model. Passing a unit test for a numerical kernel does not show that the represented physics is correct.

Verification practices include manufactured solutions, analytical benchmarks, conservation tests, regression tests, mesh studies and cross-code comparisons. Each has a scope. Matching a benchmark at a single operating point gives more limited warrant than a documented family of tests covering critical regimes; cross-code agreement may reflect shared assumptions or copied implementation errors. A research report should therefore attach explicit claims to each test: which equations, limits, quantities and configurations were examined, and which were left untested. The strength of verification is the capacity to rule out specified computational failure modes under relevant conditions. It is a necessary contribution to many evidentiary arguments, but by definition it cannot settle whether the intended equations describe the chosen world target adequately.

Validation: Connecting Model Output to Observations

Validation examines whether a model represents its target sufficiently well for a stated use. The formulation is purpose-relative: a climate model judged adequate for large-scale multi-decadal temperature trends may not be validated for a particular hourly storm cell. ASME V&V 20 explicitly confines its standardized comparison approach to specified variables and validation points and places interpolation or extrapolation beyond those points within separate engineering judgment. The restriction is instructive because it makes the locality of validation visible.

Validation involves observations with their own uncertainty, alignment of predicted and measured observables, attention to calibration datasets, withheld comparisons when available, and evaluation under relevant boundary conditions. Fitting the model to one dataset and calling the fit independent confirmation misstates the evidence. Likewise, high predictive agreement can arise from compensating parameter errors. The claim of adequacy must name the measured quantities, physical regime, decision tolerance, and relation between observations and numerical outcomes. A validation result is a warrant for a limited inference, subject to stated assumptions, rather than a universal certificate attached to every future use of the code. This practice aligns with the credibility perspective documented in NASA's model standard.

Uncertainty Quantification and Error Budgets

Uncertainty is plural. Observational uncertainty concerns measured data; parameter uncertainty concerns imperfect knowledge of model inputs; structural uncertainty concerns model form; numerical error concerns approximation and implementation; and scenario uncertainty concerns externally stipulated future conditions. A defensible simulation report identifies which uncertainty sources were assessed and which remain outside the calculation. The National Academies report treats uncertainty quantification as intertwined with verification and validation. The implication is that attaching a confidence interval to one output distribution cannot magically measure errors that were never represented in the uncertainty calculation.

A useful error budget names each source, explains its estimated magnitude or qualitative effect, records dependencies, and traces the contribution to the precise output claimed. Correlated inputs can invalidate simple aggregation rules. An ensemble across models can reflect differences among participating models without spanning all plausible structural alternatives. The aim is not always a single number. A range, conditional sensitivity analysis, bounded scenario family or explicitly unresolved uncertainty may offer a more accurate representation of the current evidence. Philosophically, the first duty is to preserve the semantic meaning of uncertainty: a probability of a model-generated event, a frequency across runs and an expert judgment about real-world outcomes are distinct quantities that require different justification.

Sensitivity Analysis as Critical Examination

Sensitivity analysis asks how changes in specified inputs and assumptions affect outputs. It can reveal influential factors, unexpected interactions, parameter non-identifiability and regions where a model is unstable for its purpose. The 2019 systematic review by Saltelli and coauthors assessed 280 papers and reported that a substantial share of the examined analyses did not adequately explore the space of jointly varying factors. That finding pertains to the review's sample and practice, not to a theorem that all scientific sensitivity analyses are false.

The difference between varying one parameter at a time and exploring combinations matters whenever interactions are present. An illustrative flood model may respond weakly to roughness changes under one rainfall assumption and strongly when rainfall and topography vary together. One-at-a-time inspection can miss the joint effect. Sensitivity claims should specify factor ranges, dependence assumptions, sampled space, chosen output, convergence of any variance decomposition, and what conclusions survive alternative plausible settings. A parameter's negligible influence within a narrow studied region cannot license its dismissal over the entire domain. Properly performed sensitivity analysis connects the mathematical structure of a model with a critical inquiry into its assumptions and thereby clarifies what would have to be true for a headline result to be trustworthy.

Calibration, Identifiability, and the Danger of Circular Confirmation

Calibration adjusts model parameters to improve a fit to selected observations. It can be an indispensable scientific procedure when parameters cannot be directly measured, but its epistemic role must be distinguished from independent validation. A calibrated model can reproduce training data for several combinations of parameters, especially when structural simplifications permit compensating errors. Good fit then shows that an implementation can accommodate a dataset; it does not uniquely demonstrate the causal reality of the fitted parameters. The verification-and-validation framework describes why model evaluation must integrate empirical information, uncertainty and computational correctness.

For analytical clarity, imagine two urban traffic simulators producing similar congestion curves after calibration. One may attribute slowdown to intersection capacity, another to route-choice preferences. Their observational equivalence on a small calibration set does not settle which process operates on other road networks. Additional observations, interventions, disaggregated measures and genuinely held-out test conditions can discriminate. The general lesson is that parameter estimation and model selection are components of scientific inference, not substitutes for it. Whenever a simulation is tuned and tested against the same observations, the research narrative must state that dependency plainly. A successful external test can strengthen its evidential standing, while persistent underdetermination should be treated as a limit on the causal interpretation of the output.

Prediction, Retrodiction, and Explanation

Simulation results enter science through several distinct achievements. Prediction addresses a future or otherwise unobserved outcome; retrodiction reconstructs possible past states; explanation seeks the dependence relations that make an outcome intelligible; and exploration identifies patterns worth later testing. A numerical trajectory may serve all four functions, but success at one does not entail success at the others. Winsberg's 2010 book examines this wider place of simulation in scientific reasoning, including the relation of computational results to theories, data and values.

A simulation might reproduce a historical pattern while relying on many adjustably calibrated assumptions. It could still illuminate possible mechanisms, but its retrodictive match alone may not uniquely identify which mechanism actually operated. Conversely, a minimal model can explain a regularity by isolating structural conditions even when its point forecasts are inaccurate. Batterman and Rice's account of minimal explanations sharpens this distinction. Strong claims should specify whether the evidential target is a specific future value, an observed phenomenon's generative possibility, the relative importance of mechanisms, or a class of robust patterns. Once the target is fixed, criteria for evaluation can be matched to it: held-out prediction, process-tracing, intervention, asymptotic argument or cross-model comparison. A universal metric of simulation success would obscure these differences.

Robustness, Model Ensembles, and Structural Diversity

When different implementations or model families produce similar outputs, the agreement can increase confidence if their relevant errors and assumptions are sufficiently independent. But ensemble consensus is not automatically a sample from the true uncertainty distribution. Several models may share parameterizations, institutional conventions, training datasets or code lineage. Eyring and coauthors' CMIP6 design documents coordinated experiments intended to structure intercomparison across climate models. Its value lies in systematic comparison and the scientific questions it enables, rather than the fiction that every model represents an independent draw from a perfectly specified population.

Robustness becomes more informative when researchers deliberately vary plausible model structures, parameterizations, resolutions and observational constraints. In an illustrative epidemiological analysis, agreement between models that all assume homogeneous mixing may reveal little about the adequacy of that shared assumption. Agreement with models containing independently motivated contact networks could carry a different evidential weight. Robustness analysis should expose common features, differences, and the output features that persist. Disagreement is itself evidence about sensitivity to modeling choices and should prompt investigation rather than automatic averaging. The strongest ensemble argument combines documented diversity with observational evaluation and a statement of which uncertainties it estimates. It remains conditional on the considered families of models and scenarios.

Climate Projection as a Documentary Case

Climate simulation provides an instructive case because large-scale models join physical laws, approximated processes, observational constraints, extensive computation and coordinated intercomparison. the CMIP6 experiment design sets out historical and diagnostic experiment families intended to illuminate model biases, climate response and future changes. The IPCC's Working Group I Chapter 4 treats projection uncertainty through distinct components, including scenarios, models, internal variability and potential future natural forcing. Those categories prevent a conditional climate projection from being read as a single unconditional prediction of human choices.

A scenario specifies a possible trajectory of external drivers; it does not independently estimate the likelihood that societies will choose it. Model uncertainty concerns how the represented climate system responds, and internal variability concerns fluctuations within the system under a given forcing history. These sources may interact, yet they answer different questions. The research conclusion is that climate projections can be evidence for conditional statements about warming patterns or ranges when supporting physics, observations and evaluation are adequate. Their interpretation must preserve the conditional nature of imposed scenarios and the scale of the question. The case shows why an apparently single graph often contains several distinct inferential transitions. This documentary analysis does not claim to recompute IPCC projections or to independently reproduce any CMIP6 runs.

Lorenz and the Distinction between Discovery and Extrapolation

The significance of Lorenz's 1963 demonstration is not that a small convection model directly forecasts global weather; it is that an analyzable nonlinear system exhibits behavior relevant to the limits of detailed prediction. The study's governing equations, numerical investigation and interpretation are distinct stages. A qualitative implication can travel beyond the original model when additional mathematical and empirical analysis supports it. But the route from idealized convection equations to the practical forecast horizon of a particular meteorological system requires a separate argument.

This historical example shows that simulation can discover properties of mathematical models that change scientific understanding even before a direct empirical validation of every output feature. An unexpected pattern can challenge assumptions about determinism and predictability, motivate research and guide the design of forecast ensembles. Such discovery is intellectual and methodological evidence, while target-specific applicability remains a subsequent matter. The distinction blocks two overstatements: dismissing modeled sensitivity because the system is idealized and claiming an exact empirical forecast limit solely from that idealization. The strongest interpretation is that computational exploration revealed a consequential property, which later scientific applications must investigate through their own models, data, resolution and forecast criteria.

Schelling and the Limits of Generative Sufficiency

Schelling's segregation models demonstrate a form of generative sufficiency: specified local decision patterns can produce marked aggregate spatial separation. The outcome can be surprising because local preferences need not be described as a simple desire for maximal segregation. Yet it is an analytical result about a family of modeled interactions. The structure does not establish that any particular city has those exact preferences, relocation opportunities or institutional constraints. The historical article distinguishes modeled aggregate patterns from direct inference about individual motives.

The methodological lesson is that the appearance of a familiar social pattern in a model establishes a possibility and suggests investigable mechanisms. Stronger empirical arguments require matching relevant micro-level processes and macro-level observations, including attention to alternative causes. Reporting agents, update order, boundaries and initial states is crucial because different choices can create different equilibria. the revised ODD reporting protocol offers a concrete discipline for making those choices inspectable. An important original interpretation follows: a simulation's most valuable conclusion may be that an explanation is insufficiently identified by the aggregate pattern, since several mechanisms can generate similar appearances. Rather than defeating the study, that conclusion can direct better empirical tests.

Scenario Analysis and Decision-Relevant Claims

Scenario simulations explore conditional futures under specified external assumptions. Their usefulness is evident in disaster response, energy planning and climate adaptation, where exact prediction can be unattainable but structured comparison remains essential. Each scenario is a premise about conditions that may or may not occur. The simulation derives consequences under those premises and under the represented mechanisms. The IPCC uncertainty taxonomy provides a public example of why scenarios must be distinguished from internally generated variability and differences among model representations.

Decision relevance asks a different question from numerical fidelity alone. A moderate error may be negligible for comparing very different infrastructure options and unacceptable near a safety threshold. Relevant uncertainty should therefore be evaluated against a decision objective, sensitivity to consequences and alternatives available to actors. A hypothetical coastal-defense model could be highly informative about the ranking of several strategies even if it cannot forecast the exact date of a rare storm. It would need to declare the conditions under which rankings reverse. The original philosophical inference is that simulation evidence has a decision horizon: the property sought, the stakes of error, and the variation in plausible assumptions jointly determine how strong an output-based recommendation can be. This is a synthesis of existing modeling and uncertainty practice, not a new decision theory.

Public Models, Normative Choice, and Accountability

Models deployed in public decisions can move from analytic aids to institutions that allocate resources and distribute risk. The Saltelli and colleagues' 2020 manifesto addresses the need for critical scrutiny of modeling assumptions, transparency and responsiveness to social consequences. A simulation may be technically sound within its stated conditions while supporting a weak political inference if the chosen objective hides distributional harms or ignores alternatives. Values enter through decisions about which outcomes matter, whose observations count, which risks to tolerate and how uncertainties are communicated.

The scientific requirement is to make those choices explicit and contestable. The modeler should distinguish factual premises, value judgments, imposed policies and computational consequences; the decision maker should distinguish a recommendation justified by a model from an act of political judgment. Public transparency involves readable documentation and accessible limitations, not only release of code that most affected people cannot interpret. Audits should attend to model governance, stakeholder challenge, and whether apparent numerical precision crowds out relevant local knowledge. Scientific authority improves when its inference boundaries can be identified and debated. The claim that a model recommends a policy always depends on aims and value commitments in addition to its empirical and computational content.

Transparency, Documentation, and the Reproduction of Results

A reported output becomes easier to evaluate when another investigator can reconstruct the computational path from assumptions and input data to analysis and figures. Peng (2011) argues that reproducibility can serve as a minimum standard where independent replication is practically difficult. Stodden and colleagues (2016) likewise identify code, data and workflow disclosure as central to improving computational research practice. A reproducible execution is not equivalent to a validated model, but it enables other researchers to look for mistakes, evaluate alternatives and identify how a conclusion was produced.

A defensible record includes precise code and dependency versions, input data provenance, preprocessing, configurations, random seeds when relevant, computational environment, run scripts, output metrics and postprocessing decisions. Data governance and privacy can constrain what is made openly available; those constraints should be documented along with legitimate alternatives for independent checking. Reproducibility should also be interpreted at appropriate levels: bit-identical results, statistically comparable stochastic results, equivalent numerical approximations and independently replicated scientific conclusions are different achievements. The manuscript and its evidence ledger follow a narrower analog of this discipline by fixing exact source hashes and documenting accessible bibliographic support, while making no claim that original scientific simulations were executed for this dossier.

Software Verification, Versioning, and Research Provenance

Scientific software changes as methods and dependencies evolve. A correction to a numerical solver can improve a result, but it can also alter historical comparability unless the exact code version and parameter choices are recoverable. Releasing a plot without its configuration prevents effective inspection of how the plotted values were obtained. NASA's modeling-and-simulation standard addresses lifecycle credibility as a management problem spanning development and use, and the reproducibility recommendations identify disclosure of computational materials as an essential research practice.

Provenance is an epistemic chain. It identifies which model formulation generated which source code, which code and data generated which execution, which execution generated which result, and which result underwrites which proposition. Any broken link can prevent a meaningful audit. A later rerun of updated code may constitute a new study rather than a reproduction of the old one. Software tests, version control and immutable artifacts are accordingly components of the logic of scientific justification. Their presence does not certify representational adequacy; they make the computational pathway available for criticism. The same discipline applies to documented corrections: investigators should preserve the nature of a failure, its affected outputs, the correction made and the scope of reanalysis necessary to reestablish a claim.

Surrogate Modeling, Machine Learning, and Hybrid Simulation

Computational studies increasingly combine mechanism-based equations with statistical approximations, emulators and trained predictive components. A surrogate can estimate expensive model outputs over a restricted input domain, enabling wider search and uncertainty exploration. Its own error must be assessed against the simulator it approximates, while the original simulator must still be evaluated against the world. The 2026 overview of computer simulations discusses the increasing interaction between simulation and machine-learning methods while distinguishing their usual roles and principles.

This creates nested inferential dependencies. A neural emulator may predict outputs of a costly climate component with small hold-out error on a specified numerical dataset. That finding concerns the emulator–simulator relation, not a direct independent measurement of the climate. If the underlying simulator misrepresents a target process, accurate emulation can faithfully reproduce that error. If the surrogate extrapolates beyond its training range, its own estimates may fail even where the mechanism-based solver remains credible. Hybrid research should therefore disclose training distributions, physics constraints, evaluation regime, uncertainty propagation and the separate validation status of component and system. The philosophical question stays stable: what grounds the transition from computed artifact to empirical assertion, and which intermediate assumptions determine its permitted range?

Evidence, Observation, and the Risks of Double Counting

A model may be tested against observations that have already influenced its construction or parameter estimates. This is not intrinsically illegitimate; scientists routinely learn from data and revise models. It becomes a problem when those observations are presented again as independent confirmation. The National Academies VVUQ discussion emphasizes disciplined separation of assessment components, while CMIP6's organized experiments illustrate the value of stating research designs and comparison purposes before interpreting a large family of outputs.

The same problem arises in model ensembles whose members reuse common tuning benchmarks and observations. If several outputs agree with an observation because all were calibrated to it, their agreement may exaggerate the appearance of independent support. Validation reports should distinguish development datasets, calibration evidence, diagnostic checks and held-out or external assessments. In an illustrative ecology study, survey data used to estimate birth rates should not be redescribed as untouched confirmation when model population curves match the same survey. Fresh observations and discriminating interventions supply stronger tests, although they too require measurement-error evaluation. The philosophical point is that evidential independence is a property of an inference's history, not a visual attribute of the resulting graph.

When Simulation Results Contradict Observations

Disagreement between simulation and empirical observation is epistemically productive if researchers preserve the possibility of several distinct causes. The model might omit a process, the estimated inputs might be inaccurate, the measurement instrument might introduce error, or the implementation might be defective. ASME's specified-point validation approach treats model–data discrepancies as quantitatively situated rather than as all-or-nothing verdicts. A mismatch should initiate an inquiry into the specific claim, uncertainty budget and stage of the simulation pipeline.

Consider a constructed reservoir model that persistently overestimates discharge during storms. Increasing a fitting parameter may reduce the discrepancy, but it would not by itself identify whether errors arise from precipitation measurements, upstream controls, or an omitted runoff mechanism. The research process should retain unsuccessful outputs and hypotheses of failure in its audit trail. Concealing negative comparisons produces overly smooth retrospective narratives that diminish the credibility of later claims. Conversely, one mismatch does not logically invalidate every model conclusion: some aggregate properties may remain supported in other regimes. Scientific correction proceeds by isolating which assumptions and inferences the evidence defeats, and by specifying what new tests could discriminate among competing diagnoses.

Computational Surprise and Scientific Discovery

Simulation can surprise its investigators by revealing behavior that was not consciously anticipated from the governing rules. The surprise is real as an event in inquiry even if the resulting pattern was, in principle, contained in the formal system. Lorenz's numerical exploration illustrates how computation can foreground consequential nonlinear behavior; Schelling's aggregate patterns show a related lesson in social dynamics. Surprise alone, however, cannot authenticate the result as a feature of the external world.

The evidentiary significance of surprise lies in prompting discriminating questions. Is the result robust to resolution and solver changes? Does it survive parameter perturbations? Was it generated by a model assumption inadvertently added through code? Can an independent analysis establish the pattern? Are there observations that could distinguish it from alternatives? A discovered property of a mathematical model may be a substantial intellectual result even before direct empirical application. Yet its public scientific significance increases when the path to relevant target phenomena is explicit. The original interpretive synthesis is that computational novelty in practice and empirical novelty in nature are different achievements; a method can produce the first while the second remains an open proposition requiring investigation.

Scientific Understanding Without Exact Forecasting

Understanding often concerns why a family of outcomes is possible, what dependencies are decisive, or why apparently different systems share structural behavior. A model can support that understanding without producing high-fidelity point predictions. minimal-model explanations furnish one route: explanation of shared macroscopic behavior through the relative irrelevance of some microscopic details. Lorenz's study furnishes another: identifying sensitive dependence as a mechanism that conditions the predictability of modeled trajectories.

This does not make empirical constraints optional. Explanatory understanding about a target requires a warranted association between the model's identified structure and properties of the target. A mathematically elegant mechanism that is absent from the studied system does not explain the system merely by being intelligible. The relationship between explanation and prediction is therefore neither strict identity nor total independence: both draw on representational commitments, but their success conditions diverge. Scientific reports should say which outcome regularity is explained, which model dependencies matter, what has been checked experimentally or observationally, and whether explanatory sufficiency is compatible with multiple causal stories. These questions enable a richer evaluation than the simplistic demand that a simulation be either perfectly predictive or scientifically unimportant.

The Boundaries of Generalization and Extrapolation

Claims proven or observed in one computational regime do not automatically generalize to another. Scaling up a model can introduce new interactions, changing resolution may expose hidden dynamics, and new social or environmental circumstances may invalidate calibration assumptions. The scope limitation is explicit in ASME V&V 20, which confines its structured comparison to specified validation quantities and points. That document illustrates a general discipline: reported credibility belongs to a described domain of use, and extrapolation is a fresh inferential act.

In a constructed materials-simulation case, a solver verified for one temperature and pressure range might fail near a phase transition because its material law has been extrapolated into a qualitatively different regime. The relevant evidence can include regime-specific experiments, theoretical bounds, independent algorithms and targeted uncertainty analysis. Similar limits occur in agent-based policy models when institutional behavior changes, and in data-trained surrogates when the input distribution shifts. A good simulation publication marks the frontier between tested and untested conditions, states how uncertainty increases near that frontier, and refrains from disguising extrapolation as routine computation. The intellectual virtue is specificity about where knowledge is strongest and where further inquiry is required.

Standards and the Institutional Conditions of Trust

Formal standards are useful because they make aspects of credibility auditable across research groups and organizations. The NASA-STD-7009B record describes a lifecycle-oriented approach in which acceptance criteria are defined by a project and relevant technical authority; the ASME V&V 20 description addresses accuracy comparisons at specified validation points. The National Research Council report sets out the mathematical and statistical foundations of verification, validation and uncertainty quantification. These are complementary documents with different organizational purposes and scope.

Their shared lesson is that evidence quality depends on transparent practices: test plans, uncertainty reporting, traceability and review of a simulation's intended use. Standards can discipline the production of warrants, yet compliance with a form does not by itself prove that a scientific target has been represented adequately. An overly generic checklist can miss context-dependent failure modes. Expert judgment remains indispensable where extrapolation, unusual regimes, competing target definitions or contested social objectives arise. This dossier treats standards as institutional sources describing recommended or required practices within their own jurisdictions, and it does not claim an ISO, ASME or NASA certification for any model discussed.

A Comparative Matrix of Evidentiary Claims

The chief distinctions can be summarized by a source-to-claim matrix. Mathematical correctness asks whether an inference follows under the model's assumptions; code verification asks whether the software implements the intended mathematics; solution verification asks about numerical error for a specified execution; validation asks about the adequacy of output for an empirical target and purpose; reproducibility asks whether a result can be regenerated under documented conditions. the National Academies classification supplies the technical context for the middle categories, and Peng's reproducibility argument helps clarify the last.

Claim type	Appropriate inquiry	Consequence of success	Remaining question
Mathematical implication	Formal proof or controlled analytical derivation	Conditional implication of stated assumptions	Do the premises fit the target?
Computational implementation	Verification, tests, benchmarks	Code implements a specified method in tested conditions	Is the method accurate for this run?
Numerical approximation	Error and convergence studies	Bounded or characterized approximation error	Does the underlying model represent the target?
Empirical adequacy	Target-specific observations and validation	Qualified model–data agreement	Does agreement generalize and identify mechanism?
Reproducibility	Versioned code, data, workflows and rerun	Independent procedural reconstruction	Is the result scientifically correct?
Decision relevance	Sensitivity, uncertainty, stakes and alternatives	Conditional evidence for action	Are aims, values and risks justified?

These rows interact, but none should be silently replaced by another. A reproducible but invalid model can mislead; a well-validated model with undocumented code may impede scientific criticism; a verified solver of an idealized model may still reveal a valuable conditional result. The matrix is an analytical teaching device assembled from established distinctions, not a new proprietary model or a formal general theorem.

The Structure of a Defensible Simulation Argument

A defensible argument can be written in ordinary language before its quantitative details are introduced. First, specify a target property and question. Second, describe the representation, including omitted factors and applicable domain. Third, establish that the computation reliably approximates the intended mathematical or rule-based system. Fourth, state what empirical observations or independently supported mechanisms make the representation applicable. Fifth, report uncertainty and the exact strength of the resulting conclusion. This order is a practical reconstruction of themes in Winsberg's methodological account, the National Academies guidance and Durán's comparative study.

An illustrative claim might read: for a specified river reach and documented rainfall conditions, a validated hydrodynamic model predicts that one barrier configuration reduces modeled peak stage relative to another, subject to identified uncertainties and within the tested flow regime. The corresponding evidence would include terrain and hydrological data, model assumptions, solver tests, comparative runs, and validation observations. This formulation conveys more knowledge than the claim that the computer proved one barrier safer. It clarifies what would change the inference: different rainfall assumptions, revised terrain, an unmodeled failure mode or new observations. By displaying its dependencies, the argument makes computational reasoning publicly challengeable and corrigible.

Counterarguments: Can Simulation Ever Be Independent Evidence?

A strong skeptical challenge holds that simulations cannot provide new empirical evidence, since their outputs are consequences of assumptions inserted by investigators. Beisbart and Norton (2012) make this challenge salient for Monte Carlo procedures. Yet a broader account distinguishes independent observation of a target from novel inferential access to consequences that investigators could not previously derive. Computational work can expose unexpected dependencies, show that an assumption set implies surprising behavior and establish conditional relations that influence later empirical testing. Those results are genuine additions to what researchers know about their representations.

The complementary danger lies in treating numerical discovery as direct measurement of nature. The fact that computation surprises a researcher does not make its assumptions true. Winsberg (2003) and Parker (2009) show that the methodology of simulation and experimentation resists an easy universal ranking. A simulation may contribute evidence about the target when supported by empirical calibration, independent validation, theoretical justification and error assessment. Its epistemic force is relational and purpose-specific. Thus the skeptical insight disciplines interpretation without erasing the legitimate knowledge produced through controlled exploration of otherwise intractable models. This conclusion does not attribute the entire debate to a single author; it is the present dossier's critical integration of the cited positions.

Counterarguments: Does Increasing Complexity Increase Realism?

A second challenge assumes that a simulation becomes more scientifically realistic as it includes more variables, mechanisms and computational detail. Greater detail can sometimes improve performance by resolving relevant processes. It can also increase parameter uncertainty, obscure structural errors, consume verification resources and make identifiability harder. Batterman and Rice (2014) explain why minimal models can have explanatory power precisely when fine details are irrelevant to the phenomenon. the sensitivity-analysis review demonstrates that model scrutiny may be weaker than its visual or technical sophistication suggests.

The preferred criterion is adequacy for purpose, not component count. A high-resolution model can be indispensable for estimating local physical stresses and excessive for identifying a broad qualitative possibility. In the latter case, an interpretable reduced model may provide a better argument. The comparison should ask which decisions or scientific questions are changed by extra detail, whether the added variables are empirically constrained, and whether numerical error and uncertainty remain controlled. Complexity is best evaluated as a trade-off among representation, tractability, observability, verification, and inferential relevance. The argument also challenges the presentation practice of using expensive computations and elaborate visualization as independent signs of evidentiary strength.

Counterarguments: Can Reproducible Results Still Mislead?

A third challenge points out that reproducibility may establish only the repeatability of a mistake. An unchanged wrong boundary condition, an unjustified parameter prior or a deeply biased dataset can produce the same misleading output every time. Peng (2011) explicitly frames reproducibility as a minimum standard rather than a complete replacement for independent replication. Stodden and coauthors propose disclosure practices that facilitate critical investigation of computational claims.

The answer is to preserve a hierarchy of evaluative questions. The first is whether the reported procedure can be reconstructed. The second is whether the implementation solves the intended formal problem. The third is whether the model's assumptions and data support its target-world interpretation. The fourth is whether the conclusion survives relevant alternative models, measurements or experimental challenges. No step makes the others redundant. This hierarchy also provides a useful publishing ethic: a reproducible result should make it straightforward to locate the point at which later criticism changes the claim. Scientific correction then becomes an accumulated record of improved inference rather than a contest between immutable images or competing authorities.

Research Method, Source Access, and Evidentiary Limits

This monograph is a philosophical and methodological research synthesis. It evaluates arguments from peer-reviewed articles, historical numerical research, official technical standards, institutional reports and documented simulation cases. Some sources were available as complete original HTML or author manuscripts; others could be checked through publisher abstracts, tables of contents or institutional bibliographic records. The archived Stanford Encyclopedia survey supplies a broad interpretive entry point, while Lax and Richtmyer's paper, the CMIP6 design article and the ODD protocol support specific technical or documentary claims. Distinctions between direct full-text access and abstract-level confirmation are recorded for each source in the accompanying evidence file.

No original simulation run, laboratory intervention, reanalysis of IPCC data, novel measurement or external peer review was performed. Constructed examples in the chapters are explicitly illustrative and carry no numerical findings. Bibliographic identity and source-to-claim scope were checked against accessible primary or institutional pages; paywalled books or standards are cited within the limits of what the publisher makes publicly inspectable. The original contribution is the sustained comparison of inferential warrants across mathematical, computational, empirical and decision stages, plus an integrated critical reading of case studies and limitations. These are independent interpretive conclusions within existing philosophical debates and established technical methods. They do not assert the discovery of a new scientific principle or a newly founded theory.

Conclusion: When Simulation Becomes Scientific Evidence

Simulation provides scientific evidence when the investigator can justify a specific passage from a computationally implemented representation to a claim about a target. The passage must be appropriate to the question and backed by relevant mathematical, computational, empirical and interpretive checks. Numerical stability and convergence address the implementation of a formal problem; validation connects the model to observed aspects of a target; uncertainty analysis locates weaknesses of the inference; reproducibility and provenance preserve the route by which a claim was produced. the National Research Council's VVUQ analysis and NASA's lifecycle approach give these distinctions a technical and institutional expression.

The philosophical conclusion extends across the case studies. Monte Carlo output often warrants a conditional numerical inference; Lorenz-style nonlinear computation reveals limits and structures of predictability; Schelling-style agent simulation demonstrates a possible generative relation; CMIP6 ensembles support carefully evaluated conditional climate projections. These contributions differ, but each becomes more meaningful when its evidentiary scope is explicitly specified. Beisbart and Norton, Schelling (1971), Lorenz (1963) and Eyring and colleagues are therefore not interchangeable illustrations of one universal sort of proof. The decisive scholarly practice is to make premises, implementations, comparisons, uncertainties and conclusions separately inspectable. Computed worlds contribute to knowledge through accountable relationships with the world they are intended to illuminate.

References

Winsberg, Eric, and Ramon Alvarado. 2026. “Computer Simulations in Science.” In The Stanford Encyclopedia of Philosophy, Spring 2026 ed., edited by Edward N. Zalta and Uri Nodelman. Stanford: Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/spr2026/entries/simulations-science/.

Winsberg, Eric. 2003. “Simulated Experiments: Methodology for a Virtual World.” Philosophy of Science 70 (1): 105–125. https://doi.org/10.1086/367872.

Frigg, Roman, and Julian Reiss. 2009. “The Philosophy of Simulation: Hot New Issues or Same Old Stew?” Synthese 169 (3): 593–613. https://doi.org/10.1007/s11229-008-9438-z.

Parker, Wendy S. 2009. “Does Matter Really Matter? Computer Simulations, Experiments, and Materiality.” Synthese 169 (3): 483–496. https://doi.org/10.1007/s11229-008-9434-3.

Humphreys, Paul. 2004. Extending Ourselves: Computational Science, Empiricism, and Scientific Method. New York: Oxford University Press. https://doi.org/10.1093/0195158709.001.0001.

Durán, Juan Manuel. 2018. Computer Simulations in Science and Engineering: Concepts, Practices, Perspectives. Cham: Springer. https://doi.org/10.1007/978-3-319-90882-3.

Beisbart, Claus, and John D. Norton. 2012. “Why Monte Carlo Simulations Are Inferences and Not Experiments.” International Studies in the Philosophy of Science 26 (4): 403–422. https://doi.org/10.1080/02698595.2012.748497.

Lax, Peter D., and Robert D. Richtmyer. 1956. “Survey of the Stability of Linear Finite Difference Equations.” Communications on Pure and Applied Mathematics 9 (2): 267–293. https://math.berkeley.edu/~wilken/228B.S07/LaxRichtmyer.pdf.

Higham, Nicholas J. 2002. Accuracy and Stability of Numerical Algorithms. 2nd ed. Philadelphia: Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9780898718027.

Metropolis, Nicholas, and Stanislaw Ulam. 1949. “The Monte Carlo Method.” Journal of the American Statistical Association 44 (247): 335–341. https://doi.org/10.1080/01621459.1949.10483310.

Lorenz, Edward N. 1963. “Deterministic Nonperiodic Flow.” Journal of the Atmospheric Sciences 20 (2): 130–141. https://journals.ametsoc.org/abstract/journals/atsc/20/2/1520-0469_1963_020_0130_dnf_2_0_co_2.xml.

Schelling, Thomas C. 1971. “Dynamic Models of Segregation.” Journal of Mathematical Sociology 1 (2): 143–186. https://doi.org/10.1080/0022250X.1971.9989794.

National Research Council. 2012. Assessing the Reliability of Complex Models: Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification. Washington, DC: National Academies Press. https://doi.org/10.17226/13395.

American Society of Mechanical Engineers. 2009, reaffirmed 2021. ASME V&V 20: Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer. New York: ASME. https://www.asme.org/codes-standards/find-codes-standards/standard-for-verification-and-validation-in-computational-fluid-dynamics-and-heat-transfer/2009/print-book.

National Aeronautics and Space Administration. 2024. NASA-STD-7009B: Standard for Models and Simulations. Office of the Chief Engineer, March 5, 2024. https://standards.nasa.gov/standard/NASA/NASA-STD-7009.

Intergovernmental Panel on Climate Change. 2021. “Future Global Climate: Scenario-Based Projections and Near-Term Information.” In Climate Change 2021: The Physical Science Basis, Working Group I Contribution to the Sixth Assessment Report, Chapter 4. Cambridge: Cambridge University Press. https://www.ipcc.ch/report/ar6/wg1/chapter/chapter-4/.

Eyring, Veronika, Sandrine Bony, Gerald A. Meehl, Catherine A. Senior, Bjorn Stevens, Ronald J. Stouffer, and Karl E. Taylor. 2016. “Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) Experimental Design and Organization.” Geoscientific Model Development 9: 1937–1958. https://doi.org/10.5194/gmd-9-1937-2016.

Peng, Roger D. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–1227. https://pmc.ncbi.nlm.nih.gov/articles/PMC3383002/.

Stodden, Victoria, Marcia McNutt, David H. Bailey, Ewa Deelman, Yolanda Gil, Brooks Hanson, Michael A. Heroux, John P. A. Ioannidis, and Michela Taufer. 2016. “Enhancing Reproducibility for Computational Methods.” Science 354 (6317): 1240–1241. https://doi.org/10.1126/science.aah6168.

Saltelli, Andrea, et al. 2019. “Why So Many Published Sensitivity Analyses Are False: A Systematic Review of Sensitivity Analysis Practices.” Environmental Modelling & Software 114: 29–39. https://doi.org/10.1016/j.envsoft.2019.01.012.

Saltelli, Andrea, et al. 2020. “Five Ways to Ensure That Models Serve Society: A Manifesto.” Nature 582: 482–484. https://www.nature.com/articles/d41586-020-01812-9.

Grimm, Volker, Steven F. Railsback, Christian E. Vincenot, et al. 2020. “The ODD Protocol for Describing Agent-Based and Other Simulation Models: A Second Update to Improve Clarity, Replication, and Structural Realism.” Journal of Artificial Societies and Social Simulation 23 (2): 7. https://www.jasss.org/23/2/7.html.

Batterman, Robert W., and Collin C. Rice. 2014. “Minimal Model Explanations.” Philosophy of Science 81 (3): 349–376. https://doi.org/10.1086/676677.

Winsberg, Eric. 2010. Science in the Age of Computer Simulation. Chicago: University of Chicago Press. https://press.uchicago.edu/ucp/books/book/chicago/S/bo9003670.html.