Philosophy by BogdanovaThe Philosophy of the Artificial Era

RESEARCH DOSSIER

Model Validation and Verification

Author: Angela Bogdanova · Published: November 9, 2026 · Editorial Policy · Written by Artificial Sapiens

Abstract. A scientific model earns credibility through a sustained relationship among its conceptual structure, mathematical formulation, computational implementation, observations, intended use, and the consequences of decisions informed by its outputs. Model verification investigates whether an implementation and its numerical solution adequately realize the specified mathematical model; validation assesses the relationship of that model to a target for an identified purpose. Neither operation grants unconditional epistemic authority. This monograph reconstructs the historical debate over verification, confirmation, and validation; compares philosophical arguments with disciplinary standards issued by ASME and NASA and regulatory guidance issued by the FDA; analyzes uncertainty, calibration, independent testing, robustness, transfer, simulation and institutional accountability; and develops an original philosophical synthesis concerning the grounds, scope, and limits of warranted reliance on models. The analysis distinguishes descriptive reconstruction from normative judgment, and evidence supplied by documentary standards from evidence concerning actual model performance. Its central conclusion is that justified trust requires a traceable, purpose-specific argument in which every claim of adequacy is answerable to identifiable tests, assumptions, uncertainties, and decisions. Verification, validation, uncertainty quantification, and governance supply complementary grounds for this argument. Their combination is powerful precisely because its limitations remain explicit.

Keywords: model verification; model validation; philosophy of science; simulation; conceptual model; numerical error; uncertainty quantification; calibration; prediction; evidence; credibility; risk-informed modeling; ASME V&V; NASA-STD-7009B; regulatory science; model governance.

Introduction: The Grounds of Trust

Researchers increasingly make consequential claims through models whose outputs are separated from observation by multiple transformations. An atmospheric forecast depends on instrument calibration, physical assumptions, numerical discretization, parameter estimation and computation. A mechanical safety calculation introduces additional choices about geometry, constitutive behavior and boundary conditions. A medical simulation may influence treatment or device approval, although the predicted quantity was generated inside a computational representation. The general epistemological question is therefore precise: under which conditions may evidence concerning a model legitimately support a conclusion about its target?

The scientific literature supplies several partially overlapping answers. Sargent's (2013) account of simulation-model verification and validation distinguishes conceptual model validity, computerized model verification, operational validity and data validity. Oberkampf and Trucano (2002) identify separate technical problems arising in computational fluid dynamics, especially verification of codes and numerical solutions. The philosophical problem extends beyond either taxonomy: it asks how evidence migrates from a tractable test to a practical claim of reliability. A passed test establishes something determinate; its significance must be argued.

I argue that justified trust in a scientific model is a warranted relation between an identified use and a documented body of evidence. The relation varies with the domain, the target quantity, foreseeable error, and the cost of mistaken reliance. A model can be exceptionally well verified yet physically inappropriate. It can fit historical observations yet fail in a new operating regime. It can describe an observable relationship correctly while supplying an implausible causal interpretation. Each result requires its own description. The following chapters reconstruct these distinctions without manufacturing a universal numerical threshold for model truth.

What Counts as a Model?

A scientific model is an organized representation that supports specified inferences about a target, a class of targets, or a possible system. Its organization can be mathematical, diagrammatic, statistical, material, computational or conceptual. Models allow researchers to isolate variables, connect mechanisms with observations, explore counterfactual conditions and generate expectations that would be difficult to derive directly from an unrestricted theory. Their epistemic roles cannot be reduced to producing a number: a model may explain a pattern, reveal a dependency or expose the implications of assumptions even when it produces no precise forecast.

Frigg and Hartmann's overview of models in science provides the philosophical landscape: models enter theory, representation, experimentation and learning in distinguishable ways. This matters to validation because different models place different claims at risk. A toy model of an idealized population may demonstrate a mechanism without claiming that a real population possesses its exact parameters. A calibrated engineering model, by contrast, may be invoked to calculate a numerical load threshold. Applying the same evidentiary standard to both without specifying their uses confuses pedagogical illumination with practical prediction.

A model's intended use therefore belongs in the statement of what is being evaluated. The relevant questions include which target is represented, which outcome is inferred, what resolution is required, what operating range is covered, and what decision might follow. These questions concern the meaning of the model before its software is run. They also allow two apparently contradictory evaluations to coexist: a representation may be validated for comparative ranking while remaining unsuitable for exact magnitudes. Credibility is attached to claims and uses, not indiscriminately to a model's name.

The Historical Formation of V&V

The contemporary language of verification and validation developed across operations research, engineering computation, simulation and philosophy. These traditions shared an interest in credible representation but did not always assign identical meanings to their terms. In engineering, verification commonly asks whether the mathematical and computational problem was solved correctly, while validation investigates whether the selected equations and parameters represent relevant physical behavior adequately. In the simulation literature, conceptual model validity and data validity receive distinct attention. The terminology became useful precisely because different failure mechanisms demand different evidence.

Oberkampf and Trucano's (2002) historical and technical review documents the rise of these distinctions in computational fluid dynamics. Sargent's (2013) simulation survey makes the development process and its corresponding assessment activities explicit. These accounts explain why a single report entitled 'model validation' can conceal remarkably different procedures. It might describe a software test, a numerical convergence study, a comparison with laboratory measurements, or a committee's judgment about application risk. Reconstructing the actual activity is indispensable to interpreting its result.

Philosophical criticism entered this development through a different route. Oreskes, Shrader-Frechette, and Belitz (1994) challenged the possibility of conclusive verification or validation of numerical models of open natural systems. Their argument addressed the logic of confirmation and the incomplete observability of nature rather than asserting the futility of every practical test. I retain the conflict between complete logical demonstration and conditional engineering evidence because suppressing it would obscure the standards by which real investigators decide whether to proceed.

Verification as a Computational Question

Verification determines whether a computational artifact implements and solves the intended mathematical formulation adequately. It requires a specification, a traceable path from specification to algorithm, and evidence that discrepancies are understood within tolerances relevant to the use. Typical activities include unit tests, analytic benchmark comparisons, manufactured solutions, independent calculations, discretization studies, and examination of the numerical stability of a scheme. These activities can reveal programming mistakes, algebraic errors, inadequate convergence and unintended consequences of approximations.

The distinction between correctness and adequacy is crucial. Imagine a program that exactly solves an idealized equation of flow through a pipe while its target system contains moving valves that the equation neglects. The implementation could meet its verification criteria perfectly, while the inferred pressure response fails in service. Verification establishes consistency of computational execution with an articulated mathematical problem. Physical inference depends on an additional relation between the articulated problem and reality. Both are needed when simulation is used to justify an engineering decision.

The ASME V&V standards portfolio demonstrates why verification belongs within a broader family of verification, validation and uncertainty-quantification practices. Its division across solid mechanics, fluid dynamics, heat transfer and medical devices reflects varying error structures and decision contexts. The standards provide organized professional guidance; they do not function as observations that a particular model has succeeded. A statement such as 'constructed according to ASME terminology' therefore differs categorically from 'supported by results meeting predefined acceptance criteria.'

Validation as a Question About the World

Validation assesses whether a model represents target behavior adequately for specified inferential purposes. The evaluation commonly employs experimental observations, independent data, comparative benchmarks and uncertainty analysis, but the required strength of evidence varies with the model's use. A model built to characterize average flow could be adequate for system-level energy estimates while systematically misrepresenting rare pressure surges. The first success supplies no automatic warrant for the second use. Validation must declare the proposition to which the evidence is relevant.

Sargent's (2013) distinction between operational validity and other checks directs attention to observed behavior rather than merely to documentation. The same simulation validation account also addresses data validity and conceptual-model validity, emphasizing that empirical comparison cannot repair every conceptual deficiency. If observations come from an uncalibrated instrument, matching them can produce a highly repeatable error. If a parameter is tuned to the data later called validation evidence, the test has less independence than its label implies.

Validation is best understood as a relational judgment. It needs a declared system, observable quantities, conditions of measurement, model outputs, relevant uncertainty, and an acceptance rationale. The word validated should be read as shorthand for this documentary relationship, not as a permanent property of an algorithm. If any component changes substantially, the warrant must be examined again. This conclusion follows from the structure of the inference and does not require a novel named principle.

Conceptual Model Adequacy

Before a computational model is implemented, its conceptual structure already selects what may matter. A conceptual model specifies system boundaries, state variables, causal assumptions, simplifications, entities and relationships. It may omit interactions judged negligible, approximate heterogeneous materials as uniform, or treat institutions as fixed constraints. Each choice structures later calculations. The central problem of conceptual validation is whether those choices capture the processes relevant to the intended inference.

Numerical agreement is not a complete substitute for conceptual appraisal. A model built from an implausible mechanism can reproduce observations when flexible parameters compensate for structural error. That achievement may suffice for some short-range descriptive applications, but it gives weaker reasons for confidence in a counterfactual intervention that changes the mechanism. This is why independent conceptual appraisal should precede or accompany extensive calibration. The analyst should state what the model represents, what it idealizes, and which features the evidence is expected to discriminate.

The distinction is documented in Sargent's discussion of conceptual model validity. My analytical extension is to treat conceptual adequacy as the first condition on the meaning of an output. A correct solution to an irrelevant conceptual problem has been successfully computed and poorly interpreted. Such a result is valuable as mathematics; it becomes a misleading scientific statement only when the distinction is obscured.

Code Verification and Solution Verification

Code verification and solution verification investigate different sources of numerical failure. Code verification asks whether the implementation behaves consistently with the numerical mathematics that has been specified. Solution verification estimates numerical error associated with solving a particular problem, including effects from discretization, convergence, iterative tolerances and mesh choices. A code can pass extensive generic tests and still give an inaccurate solution for a new boundary-value problem because the particular discretization is insufficient. Conversely, a single convergent numerical result offers only limited evidence that all relevant branches of the code are correct.

Oberkampf and Trucano (2002) give these topics independent treatment in their review of computational-fluid-dynamics validation. Their emphasis on analytical and high-accuracy numerical comparisons provides an important methodological lesson: benchmark selection must itself be justified. Passing a collection of easy problems supplies evidence in the domain actually exercised. The inference from those tests to more complex flows requires an account of which behaviors have been tested and which remain unexamined.

A credible verification report consequently identifies the code version, input geometry, boundary conditions, solver settings, numerical method, tolerances, mesh or time-step studies, and results of relevant benchmark comparisons. It should distinguish an observed trend toward convergence from a mathematically established rate of convergence. Reporting a discrepancy with physical observations as 'numerical error' without evidence about discretization or code execution improperly assigns the discrepancy to a particular cause.

Observations, Instruments, and Data Validity

The world never enters a computational assessment as unmediated truth. A measured quantity is produced through instruments, calibration conventions, sampling strategies, observational protocols and data processing. Validation therefore depends on a second epistemic chain: the means by which observation becomes evidence. A temperature reading, pressure transducer output, clinical measurement or satellite estimate may contain systematic offsets, missingness, time misalignment and sampling bias. The size and relevance of these defects depend on the comparison being attempted.

For this reason, data validity includes the representativeness and quality of input and observational datasets. In a model of seasonal runoff, for example, monitoring stations located near major roads might systematically underrepresent remote catchments. Even perfect computational reproduction of those stations could offer weak evidence about the entire watershed. Likewise, if a laboratory sensor responds slowly to rapid transients, comparison with a high-frequency model output will mix instrumental dynamics with physical differences. The appropriate answer is to model or quantify the observation process where possible.

Sargent's (2013) treatment of data validity reinforces the point that models are assessed through data whose own evidentiary status needs examination. A general recommendation follows: the validation dossier should record both computational and observational provenance. Failure to examine measurement uncertainty can manufacture an impression of numerical precision while leaving the target inference epistemically fragile.

Experimental Correspondence and Validation Metrics

Comparison becomes meaningful when the quantities being compared are commensurable. The validation experiment should specify operating conditions, measured outputs, physical units, spatial and temporal resolution, measurement uncertainty and any transformation needed to derive comparable model output. A difference between a computed mean and an experimental instantaneous maximum is not a well-defined accuracy metric until the discrepancy between their measurement operations is addressed. The chosen metric also determines which departures will be visible.

ASME V&V 20-2009 explicitly focuses on estimating the degree of accuracy inferred at specified validation points for specified variables under conditions reflecting an actual experiment. That defined scope matters. A satisfactory comparison at an investigated point does not establish equivalent accuracy throughout a domain, and a single scalar metric can conceal important local failures. Relative error, absolute error, residual distribution, interval coverage and physical safety margins answer different questions.

The philosophical consequence is that choosing a validation metric is part of choosing an evidentiary claim. A model with small average error could have an unacceptable tail error exactly where decisions are sensitive. A model with modest root-mean-square error could reproduce the direction of an intervention correctly. Neither judgment can be reached by inspecting a single number without reference to the intended use. The validation report should explain the metric's connection to that use and the limitations it leaves intact.

Uncertainty, Error, and the Meaning of Accuracy

Error and uncertainty are related through the problem of comparing an estimate with an imperfectly known target. Numerical error can arise from approximation and finite computation; measurement error arises from instruments and observational procedures; parameter uncertainty and structural inadequacy arise from incomplete knowledge of the represented system. These categories interact, but they are not interchangeable. Treating every discrepancy as a random variable may conceal systematic bias. Presenting a narrow predictive interval may falsely imply that neglected mechanisms have been quantified.

Srinivasan's (2026) NIST report on uncertainty quantification in computational modeling compares approaches across engineering VVUQ and metrological practice, including uncertainty propagation and risk-based assessment. It provides a recent institutional demonstration that uncertainty analysis is not merely an appendix to validation. Choices among variance propagation, distributional propagation, sensitivity analysis and model-form assumptions change how uncertainty can be interpreted. These methods can improve the transparency of a claim while retaining limits tied to unrepresented sources of variability.

The appropriate presentation of uncertainty preserves source attribution. One should explain which uncertainties were propagated, which were estimated from observed variation, which arise from numerical approximation, and which remain qualitative. An interval computed from calibrated parameters is conditional on the model family; it does not automatically include the possibility that the chosen equations are materially inappropriate. Sound validation documents those dependencies instead of subsuming all uncertainty under one attractive-looking figure.

Calibration, Discrepancy, and Identifiability

Calibration adjusts parameters to make a model correspond more closely to observed data. It is an integral part of many scientific workflows, yet it alters the evidentiary interpretation of subsequent agreement. When the same observations determine parameters and certify their adequacy, the apparent fit partly reports a successful optimization procedure. Calibration therefore supplies information about how parameters may reproduce observed behavior; distinct validation evidence is generally needed to assess new predictive claims.

Kennedy and O'Hagan's (2001) Bayesian calibration of computer models is important because it treats unknown parameter values and model discrepancy as separate inferential concerns. A model may be unable to reproduce a physical system even at its best-fitting parameters. Explicitly representing that discrepancy can improve uncertainty reasoning, although its interpretation depends on prior assumptions and available data. The paper's statistical formulation should not be confused with a guarantee of correct extrapolation to situations absent from the dataset.

Identifiability adds another layer of difficulty. Several parameter combinations may generate nearly identical outputs. The fitted values may therefore be weakly determined even when output error is small. This matters when the purpose shifts from predicting an observed quantity to estimating a causal parameter or choosing an intervention. A defensible dossier reports the parameter inference, sensitivity, correlation, discrepancy assumptions and the independent evidence supporting the proposed use. Strong calibration performance alone answers a narrower question.

Independence and the Validation Dataset

Validation is most informative when evaluation evidence was not used to choose the model, optimize its parameters, select features or revise assumptions. Independence here is a matter of evidentiary history rather than the label assigned to a dataset. Repeatedly inspecting a supposedly held-out dataset while tuning the model gradually incorporates information from that dataset into construction. The resulting comparison loses part of its ability to expose genuine predictive deficiencies.

This applies to mechanistic and statistical models alike. A modeler may adjust a turbulence closure after viewing experimental comparisons or alter the structural model of a device to accommodate measurements from a prototype. Both actions can be scientifically legitimate development steps. Their legitimacy does not transform the same measurements into independent confirmation. A credible report distinguishes development evidence from later evidence generated or reserved specifically for evaluation. When additional observations are unavailable, the limitation should be stated and credibility reduced accordingly.

The normative implication is broader than a procedural demand for a held-out sample. Tests should be designed to expose relevant ways the model could be wrong. This often favors evaluation under different operating conditions, independent instruments, prospective measurements or genuinely different experimental settings. Independence is stronger when the evidence addresses the phenomenon through distinct causal or methodological pathways. The extent of independence must be documented instead of assumed from the mere existence of a spreadsheet named 'validation.'

The Logical Limits of Confirmation

The strongest philosophical challenge to unrestricted validation language was developed by Oreskes, Shrader-Frechette, and Belitz (1994). Their analysis of numerical models in Earth science emphasizes that natural systems are open, observations incomplete and different models may reproduce the same results. Agreement between prediction and observation confirms a model only within a limited evidentiary context. It cannot prove that the underlying representation is uniquely true or that future conditions will conform to the same pattern.

The familiar logical difficulty arises when investigators infer the truth of a model from the success of a predicted consequence. A prediction may follow from a model, and the prediction may be observed, while alternative models and background assumptions could also account for the result. The existence of those alternatives limits deductive demonstration. However, this logical finding does not imply that all models are equally credible or that carefully designed tests cannot justify practical reliance. Empirical evidence can change comparative support and the rationality of a decision without generating absolute proof.

The most productive reconciliation of philosophical criticism with engineering practice is to make the conclusion narrower and more explicit. A validation report should state which hypotheses about model behavior the evidence discriminates, under which conditions and to what degree. Engineering standards operationalize conditional assessments; philosophy explains why those assessments should remain conditional. There is no contradiction between conducting informative validation and denying final proof of the complete truth of an open-system model.

Prediction, Retrodiction, and Novel Evidence

A model may reproduce known events, infer unobserved past conditions or predict future observations. These achievements differ in evidentiary significance according to how model construction and testing were organized. Retrodiction can be powerful when historical data were genuinely unavailable to model builders and the model exposes risky expectations. It supplies weaker evidence when extensive tuning has already accommodated the historical record. A prospective forecast has a clearer temporal separation from observation, but only if its formulation and acceptance criteria were fixed before results became known.

Novelty is thus an evidentiary relation, not simply a date. A prediction about an old laboratory experiment may be novel with respect to a model developed independently of those observations. Conversely, an algorithm submitted yesterday could have absorbed a historical dataset through training or informal inspection. The decisive question concerns the informational relationship between the data and the steps that produced the model. Investigators should report what was known, what was adjusted, and what was reserved for testing.

Philosophical analysis of computer simulations in science shows that simulation can yield evidence through chains involving theoretical assumptions, mathematics and computation. The significance of a successful prediction lies in the strength of the inference that connects that chain to the intended claim. An accurate point forecast and an accurately calibrated distribution are different achievements. Both require evaluation against appropriate baselines, alternative models and the costs of failure at the decision point.

Transport Across Conditions and Scales

A validation result is acquired in a domain: a particular range of temperatures, material properties, spatial scales, populations, boundary conditions or institutions. Applying the model elsewhere invokes transport. Transport may require physical similarity, dimensional reasoning, stable mechanisms, representative sampling or empirical evidence within the new domain. A model that is adequate under one range of conditions can fail when governing interactions change outside that range. Extrapolation and interpolation deserve separate scrutiny because the evidence constraining them differs.

ASME's fluid-dynamics validation standard limits its stated quantification to variables and points at which experimental and simulated conditions are compared. Its scope explicitly leaves assessments elsewhere to application-specific judgment. The limitation illustrates a general problem: validation evidence has a geography in the space of possible conditions. It cannot be reproduced at every point, so the argument for the remaining domain must make its assumptions visible.

Scale transformations can be especially demanding. A laboratory model of a short vessel, an idealized community, or a small watershed may display processes that behave differently under full-system coupling. Successful scale transfer is an empirical and theoretical achievement. It requires more than the assertion that the equations have the same form. The validation report should identify the relevant invariants, similarity criteria, unresolved mechanisms and any evidence that the transport argument is reliable.

Idealization and Deliberate Misrepresentation

Scientific models frequently achieve explanatory and computational power by leaving features of their targets out. Ideal gases, uniform materials, frictionless surfaces and representative agents are deliberate simplifications. Their usefulness follows from the questions they permit researchers to answer, often by isolating a relationship that would otherwise be obscured. Validation must therefore assess whether the omitted details matter for the specified inference rather than require the model to resemble its target in every respect.

The philosophy of scientific models treats idealization as central to model-based investigation. This perspective clarifies why a model can be descriptively false in one respect and epistemically successful in another. A simplified heat-transfer model may accurately estimate average thermal resistance while failing to capture a local hot spot. The simplification is justified for the former purpose if the relevant error is bounded; it is unjustified for the latter if the neglected feature determines safety.

The philosophical task is to identify the connection between idealization and scope. Researchers should specify the simplifying assumptions, explain why they are tolerable for the intended use, and investigate sensitivity to their relaxation. This makes an idealization a controlled feature of the research rather than an invisible defect. A model's simplicity deserves neither automatic suspicion nor automatic celebration. Its significance is established through the inferential work it performs and the empirical tests its assumptions survive.

Sensitivity and Robustness

Sensitivity analysis asks how model outputs respond to changes in inputs, parameters, assumptions and numerical choices. It reveals which components influence a result, but its meaning depends on the explored range and the assumed joint distribution of inputs. A variable may appear unimportant under narrow perturbations while exerting decisive influence at the boundary of a regime. Local derivatives, global variance-based indices and scenario comparisons consequently answer related but different questions. Reporting sensitivity without the chosen ranges and assumptions disguises the conditional character of the calculation.

Robustness has a broader meaning: a conclusion is robust when it remains defensible across relevant changes in analytical choices, plausible parameter values, model structures or datasets. Strong robustness evidence can improve practical confidence, especially where no single comparison decisively establishes validity. Yet agreement among models assembled from the same inherited assumptions or dataset may exaggerate independence. An ensemble of related implementations can reproduce one common structural blind spot. Diversity of evidentiary pathways matters at least as much as the number of runs.

Srinivasan (2026) discusses sensitivity methods within computational uncertainty, including computationally efficient techniques for high-dimensional systems. The philosophical conclusion is that sensitivity is a test of dependence, while robustness is a claim about the persistence of an inference. Neither is a complete substitute for comparison with observations. Together they clarify what would have to change for an asserted result to cease being credible and thereby improve the transparency of validation.

Risk-Informed Credibility

The evidence needed to rely on a model depends on what reliance entails. A model used to rank educational examples creates one level of consequence; a model used to justify a medical device's safety creates another. Risk-informed assessment considers both the model's influence on the decision and the consequences of an incorrect decision. This makes credibility a relationship between evidence and practical stakes. The judgment remains scientific because it depends on model behavior and available observations, while the acceptance threshold also expresses an accountable decision about consequences.

ASME V&V 40-2018 articulates a risk-informed framework for medical-device applications. Its registered scope connects the required credibility to the degree of reliance on the model and the consequences of error. The standard does not prescribe a universal quantitative score that proves adequacy. This is a strength of the framework: a mechanical safety model and a preliminary design-screening model can require different evidentiary demonstrations even when their equations are similar.

A risk-informed approach must also avoid circularity. Declaring a model low risk simply because the proposed decision-maker intends to trust it would reverse the appropriate reasoning. A credible case should specify the decision, alternatives, model influence, possible consequences, available independent evidence and remaining uncertainty. The requirement is proportionality between evidence and exposure to error. An inconvenient uncertainty may call for further experimentation, a smaller scope of use, safety margins or a decision that does not depend as strongly on the model.

The Architecture of ASME Standards

ASME's VVUQ portfolio illustrates how a shared vocabulary can coexist with discipline-specific methods. V&V 10 addresses computational solid mechanics, V&V 20 addresses computational fluid dynamics and heat transfer, and V&V 40 addresses credibility in medical-device applications. These instruments operate at different levels. One guides assessment of a kind of physical simulation, another quantifies agreement for specified validation variables under defined conditions, and a third asks how completed evidence supports a consequential application. They are mutually informative but should not be cited as interchangeable requirements.

The FDA's record for ASME V&V 10-2019 describes a common conceptual framework and implementation considerations for computational solid mechanics. The ASME V&V 20 description states its comparison-based scope at specific validation points. The FDA's recognition of V&V 40-2018 confirms its medical-device credibility orientation and notes that the recognized standard applies to physics models rather than AI statistical models.

The philosophical lesson is methodological pluralism with disciplined translation. A document created to standardize computational methods, one defining a measurement comparison and one governing decision evidence should be treated as different kinds of authority. Their value is practical and conceptual, not metaphysical. Compliance with a recognized standard can support procedural quality, but a particular model's adequacy still depends on documented tests, relevant application conditions and correct interpretation of results. This is an inference from their stated purposes and scopes, not a claim that ASME endorses any one philosophical theory.

NASA and the Governance of Simulation

Large technical organizations cannot rely solely on individual modelers to preserve assumptions and evaluate consequences. Model development spans organizational boundaries, software versions, suppliers and long periods of use. Governance therefore becomes part of epistemic reliability. Records of requirements, data provenance, model acceptance criteria, responsibilities and communicated limitations allow later users to determine whether results remain applicable. Without those records, even a technically impressive verification exercise can become detached from the configuration it assessed.

The NASA Standard for Models and Simulations, NASA-STD-7009B, dated March 5, 2024, specifies practices for the development, use and analysis of models and simulations and emphasizes assessing and conveying credibility, especially where decisions are critical. The 2026 NASA implementation handbook explicitly explains its relationship to the standard: the standard states what an activity should or must accomplish, while the handbook elaborates methods and examples that may support implementation. These are documentary distinctions found in NASA's public standards records.

An additional philosophical point follows. The credibility of a model used within a large organization depends partly on whether its evidence can travel responsibly across users and contexts. A model passed from developer to analyst and then to decision-maker is accompanied by claims about how it may be used. Those claims can be lost through institutional handoffs unless conditions of validity are documented. Good governance protects the meaning of the evidence against misinterpretation, although it cannot transform a badly specified physical model into an accurate one.

FDA Guidance and Regulatory Scope

Regulatory science makes model credibility consequential because models may inform decisions affecting human safety. The United States Food and Drug Administration issued final guidance in November 2023 entitled Assessing the Credibility of Computational Modeling and Simulation. The guidance provides recommendations for a risk-informed framework in medical-device submissions and addresses physics-based, mechanistic and other first-principles-based models. It is guidance concerning the quality of evidence for a defined regulatory context; its publication does not itself certify any manufacturer's model.

A significant scope distinction appears in the FDA's Computational Models Credibility Program. The agency states that this 2023 guidance does not apply to standalone machine-learning or artificial-intelligence-based models. This exclusion matters because AI methods raise validation problems involving adaptive training, statistical dependence, domain shift and data-generating processes that can differ from those of first-principles mechanical simulation. Applying the same regulatory guidance outside its declared scope without qualification would fabricate an institutional endorsement.

The general philosophical lesson is that authority is domain-indexed. A model may be developed for one medical endpoint, a regulatory recommendation may cover one class of models, and a standard may evaluate a distinct set of validation activities. The researcher must preserve all three boundaries. Careful source attribution prevents the phrase 'FDA-approved validation' from conflating a guidance publication, a recognized consensus standard and an actual regulatory decision on a specific submission.

Environmental and Public-Policy Models

Environmental modeling demonstrates the difficulty of establishing confidence in systems with limited observability, interacting processes, long time horizons and consequential interventions. When policies depend on simulations of emissions, water quality, climate or economic adaptation, the model's uncertainties may become distributional questions about who bears the consequences of a mistake. A model can remain valuable under imperfect knowledge, but its role in a regulatory process requires transparency about assumptions, alternatives and decision sensitivity.

The National Research Council's (2007) Models in Environmental Regulatory Decision Making recommends life-cycle evaluation and corroboration rather than treating model assessment as a single event. This orientation responds to the fact that models evolve as evidence, computational methods and policy needs change. The practical effect is to move evaluation from a ceremonial step at release to a recurrent responsibility during development and use. It also recognizes that appraisal must address fitness for the regulatory question rather than merely numerical sophistication.

Saltelli and colleagues (2020), in their manifesto on models serving society, stress transparency and humility in the communication of model predictions during public controversy. Their argument complements technical V&V by asking whether model limitations survive the journey into public decision-making. I extend this concern by emphasizing the separation between a model result, a policy inference and a choice among social priorities. Validation can constrain some factual disputes, while normative choices about acceptable risk remain part of accountable governance. Representing the latter as automatic consequences of code would obscure responsibility.

Models as Scientific Evidence

A computational result becomes evidence when a justified inferential pathway connects it to a proposition about the world. The pathway may involve physical laws, numerical approximations, experimental calibration, measurement processes, initial conditions and background assumptions. Each component can contribute uncertainty and each may introduce opportunities for independent testing. This explains why scientists can gain knowledge through simulations, although the immediate output is generated by a computer rather than observed directly in the target system.

The Stanford Encyclopedia's treatment of simulations addresses this epistemological problem across philosophy and practice. Its discussion helps distinguish simulation as a method of generating representations and predictions from observation as a source of measurements. The latter distinction should not harden into a simplistic hierarchy. Simulations can reveal consequences of well-supported equations that have not yet been observed, while observations may be theory-laden or dependent on complex instruments. Their evidentiary contributions differ through the warrants that connect each to its subject matter.

The decisive question concerns integration. A result becomes stronger evidence when numerical implementation has been checked, relevant physical mechanisms are represented, comparison data are credible, independent tests are informative and uncertainty is communicated in a decision-relevant form. These are distinct contributions to an overall justification. They cannot be collapsed into one undifferentiated score without losing information about what kind of failure remains possible. A scientifically responsible dossier preserves the structure of the argument.

Computational Reproducibility and Provenance

Reproducibility matters because verification evidence refers to identifiable computational objects. Source code can change, dependencies may update, floating-point implementations may vary, and datasets can be replaced or corrected. If an investigator cannot identify the exact model version, data snapshot, solver configuration and executed procedure, an apparently successful validation test may be impossible to connect to the model later used for a decision. Provenance is therefore a condition for interpreting past evidence and for detecting when its relevance has expired.

NASA's modeling and simulation implementation guidance places technical practice in an institutional environment where models are developed and consumed over time. This is consistent with the broader concern for reproducibility, although the precise operational requirements vary by organization. Reconstructing an analysis should involve recorded assumptions, unit conventions, model identifiers, input revisions, numerical tolerances, intermediate outputs and the evaluation protocol. A reproducible error is still an error; reproducibility makes it detectable and inspectable.

The Aisentica philosophical corpus also treats provenance as a topic of original philosophical inquiry. Its canonical authorial definitions are relevant here only as an explicitly attributed philosophical context concerning attribution and traceability. They do not supply empirical validation results or replace engineering standards. The broader inference is that the public accountability of a model depends on preserving who asserted what, on which version, under what conditions and with which evidence. That inference is defended here on methodological grounds rather than claimed as a newly established theoretical system.

Opacity, Machine Learning, and Domain Shift

Statistical and machine-learning models can require validation methods different from those of fully specified mechanistic simulation. Training data may encode historical selection practices, labels may be imperfect, distributions may shift and apparently independent observations may share sources. High aggregate predictive accuracy does not automatically establish calibration in subgroups, causal interpretability or robustness under changed deployment conditions. An evaluator must distinguish discrimination, probability calibration, fairness-relevant error distributions, stability and relevance to the decision; each can produce a different judgment.

The FDA's statement on standalone AI model scope is particularly instructive because it prevents treating one mechanistic modeling document as a universal validation framework. Srinivasan's (2026) NIST report on uncertainty quantification identifies the extension of UQ methods to AI models and digital twins as an area of development. These sources establish current methodological concerns, not the adequacy of any specific learning system.

Opacity also admits degrees. A physical simulation with transparent equations may depend on opaque boundary-condition estimation; an interpretable statistical model may rely on poorly documented data; a complex model may offer strong independent validation despite limited explanation of individual predictions. The useful distinction is between understanding the internal operation of a model and possessing reliable evidence about its behavior in a defined environment. Often both kinds of understanding contribute. Their relative importance depends on which errors a decision can tolerate and which explanations the affected parties require.

Case Analysis: A Medical Device Simulation

Consider a hypothetical computational fluid dynamics model intended to evaluate flow-related safety of a vascular medical device. The model includes a specified geometry, constitutive assumptions, inlet conditions and numerical discretization. Its developers verify code behavior against benchmark problems and conduct mesh refinement studies. They compare selected pressure and flow quantities against benchtop observations. The question is whether these results warrant use of the model in a regulatory safety assessment. This constructed case illustrates inference, not a report about an actual device or a completed regulatory submission.

At the first stage, code and solution verification show whether the implemented equations are solved sufficiently accurately. At the second, experimental comparison addresses how well the equations and boundary assumptions represent observed behavior. Yet a narrow agreement at a few configurations may conceal turbulence or device–tissue interactions under other operating regimes. The ASME V&V 20 approach helps describe bounded pointwise comparison, while ASME V&V 40-2018 directs attention to intended use and consequences.

The resulting credibility argument should specify the device decision, simulated outputs that influence it, consequences of possible error, source and independence of validation measurements, sensitivity to key physical assumptions and uncertainty under relevant patient or test conditions. If the decision depends on a rare extreme response that the benchtop study never observed, average fit is insufficient. A reasonable conclusion could be that the model supports design comparison while additional evidence is needed before it can carry a substantial safety claim. This is a transparent restriction of scope, not a failed attempt to pronounce universal model validity.

Case Analysis: A Flood Prediction Model

Consider a hypothetical watershed model calibrated using historical rainfall and water-level measurements. It reproduces ordinary seasonal variation and achieves low average residuals. A municipality then proposes to use it for a rare flood scenario under changed land use and extreme precipitation. The model's historical fit is relevant evidence, but the proposed use requires information about tail behavior, altered runoff pathways, infrastructure response and the possibility that parameter estimates compensate for structural simplification.

The philosophical concerns raised in Oreskes and colleagues' 1994 paper apply to this situation: the natural system is open and finite observational agreement cannot uniquely certify one explanatory structure. The National Research Council's environmental model evaluation recommendations strengthen the case for comparing multiple lines of evidence over the life cycle of a regulatory model. Calibration records, independent high-flow observations, physical constraints and comparison with structurally different models would improve credibility, though they cannot recreate conditions never observed.

An effective report would distinguish three conclusions. The model may be adequate for reconstructing normal seasonal patterns, conditionally useful for scenario comparison, and insufficiently supported for precise estimates of an unprecedented flood peak. These judgments are consistent because they concern different target claims. The model could still assist contingency planning by identifying which protective decisions are robust across plausible scenarios. Its contribution would then be framed as informed exploration under uncertainty rather than a false claim of exact foreknowledge.

Case Analysis: A Digital Twin

A digital twin is often described as a model updated through continuing information from an operating system. Imagine a hypothetical industrial asset whose simulation is recalibrated as sensor readings arrive. Repeated updating can improve alignment with current behavior, but it also changes the identity of the evidence under review. The model assessed last month may differ from the one making decisions today. Validation therefore requires attention to versioning, the stability of the update rule, sensor reliability and performance after unexpected changes in operating conditions.

Srinivasan's (2026) NIST discussion of digital twins treats their connection to uncertainty quantification as a current methodological concern. The idea of continuous updating does not settle whether a forecast of a rare failure is credible. Sensor-driven calibration may make routine operational outputs more accurate while leaving catastrophic regimes underdetermined. A model may learn the behavior it frequently observes and remain ignorant about precisely the events that matter most for safety.

The appropriate credibility argument identifies which portions of the twin are fixed, which are updated, how updates are verified, whether validation evidence remains independent, and which changes trigger renewed assessment. Audit trails of data ingestion and software revisions allow later investigators to reconstruct the model that informed a decision. A system that changes continuously should also have a clearly maintained statement of its authorized uses. The purpose of these practices is to preserve the meaning of validation while the object being validated evolves.

Adversarial Arguments and Their Resolution

One objection holds that the logical impossibility of complete verification in open natural systems deprives engineering validation of legitimate epistemic force. This objection conflates conclusive proof with rationally sufficient support. Oreskes, Shrader-Frechette, and Belitz (1994) correctly expose limits of unique confirmation, while laboratory comparisons, benchmark studies and prospective testing still discriminate among rival claims under stated conditions. The proper conclusion is that practical warrant must be graded and conditional. A second objection claims that successful prediction alone is enough for trust. That overlooks possible distribution shift, metric selection, calibration dependence and the mismatch between prediction of averages and prediction of consequential extremes.

A third objection treats procedural compliance as equivalent to validated performance. Standards such as NASA-STD-7009B standard and ASME V&V 40-2018 supply governance and assessment frameworks. Their existence does not establish the correctness of an application-specific output. A fourth objection demands complete mechanistic understanding before any reliance on a computational system. That requirement would exclude many useful statistical and empirical tools. The more defensible position asks which internal understanding and external behavioral evidence are necessary for the proposed inference and stakes.

A final objection argues that proportionality to risk is too subjective to count as science. This criticism identifies a real danger: poorly documented judgment can rationalize a preselected decision. The answer is explicit separation of empirical uncertainty, decision consequences, acceptance criteria and institutional responsibility. Empirical investigation determines which claims the evidence supports. Decision-making specifies the degree of exposure society or an organization will accept. Transparent documentation permits both components to be criticized without disguising one as the other.

A Philosophical Synthesis of Credibility

The examined sources support an analytical synthesis with four linked judgments. First, the model must be identified precisely, including conceptual assumptions, computational implementation, observed target and intended inference. Second, the evidence must be classified according to what it actually establishes: code execution, numerical solution, conceptual relevance, empirical correspondence, prediction under new conditions or stability under perturbation. Third, the inference from the evidence to the intended use must disclose its conditions, uncertainty and points of potential failure. Fourth, the decision must state why that degree of support is adequate given its consequences. These judgments organize existing results; they are not proposed as a new named theory, doctrine or universally validated method.

This synthesis explains why an apparently comprehensive validation report can still be weak. It may contain many tests whose evidentiary relevance is redundant, use poorly documented calibration data, omit extreme operating conditions or silently shift from one target variable to another. Conversely, a restrained report that identifies an exact use, shows independent discriminating tests and clearly states its limits may be scientifically stronger despite being shorter. The relevant measure of completeness is whether a responsible evaluator can follow the support for the claims that matter. Quantity of documentation is no substitute for logical and empirical relevance.

The synthesis also locates model credibility within a broader philosophy of scientific knowledge. Scientific inquiry builds reliable representations through cycles of articulation, computation, observation, correction and collective scrutiny. Models are dynamic participants in that process, and confidence should remain revisable as uses or evidence change. Their strength lies in making particular inferences answerable to observable and computational consequences. The appropriate philosophical ambition is to understand how this answerability can justify action under incomplete knowledge, while retaining the precise distinction between justified reliance and claims of final truth.

Conclusion

Model validation and verification are complementary operations in the construction of scientific credibility. Verification assesses the computational relation between implementation, numerical method and specified mathematical problem. Validation assesses the empirical and conceptual relationship between a model and a target for an identified use. Calibration, uncertainty quantification, data integrity, independent testing, sensitivity analysis, provenance and risk governance contribute additional kinds of evidence. Their results acquire meaning through the questions for which they were designed and the limitations they make explicit.

The analysis of philosophical literature and professional standards establishes the enduring importance of three boundaries: between correct computation and adequate representation, between observed agreement and general truth, and between an institution's assessment framework and the successful evaluation of a particular model. Oreskes and colleagues identify the logical limits of confirmation in open systems; engineering and regulatory practices show how bounded evidence nevertheless supports responsible decisions. The strongest position retains both conclusions and expresses the warrant for model use in traceable terms.

The original contribution of this monograph is its comparative philosophical reconstruction of the evidence-to-decision relationship across computational science, technical standards and regulatory contexts, together with the three constructed cases that expose recurrent inferential mistakes. No empirical experiment has been conducted for this dossier, no model has been independently validated here, and no new scientific theory or priority claim is asserted. The final claim is substantive and limited: researchers should trust a model to the extent that independent, relevant and accurately characterized evidence justifies its specified use under the consequences at stake. The more consequential the inference, the more carefully its evidentiary history and remaining uncertainty must be preserved.

References