AI Study | Se-Hyeon Hwang (황세현)

Concepts by type around the ring, and every edge an assertion about what holds between them. Hover an edge to read the claim. The corpus graph shows instead which writing discusses which concept.

Ontology

The knowledge graph records which of these notes discusses which concept. This is a different thing: what kind of entity each concept is, and what I hold to be true between them. Every line below is an assertion, so every line is disputable, and the ones resting on published work carry the source. 195 assertions over 101 concepts, in 13 relations.

EvidenceSource6 Finding6 ClinicalTarget7 Standard2 Method41 ModelFamily1 Metric2 Property15 Threat9 Mechanism6 Principle6

assessesProperty

Method · Metric → Property

Provides a procedure or numerical assessment of the property, without establishing it conclusively.

Auditing assessesProperty Faithfulness methodological high confidence E01 A frozen-model explanation audit compares the readout's stated importance against the behaviour it purports to explain. no source asserted
Intervention & ablation assessesProperty Faithfulness methodological high confidence E02 Controlled perturbations compare attribution-predicted effects with actual score changes, conditional on the perturbation design. arXiv:1901.09392
Evaluation metrics assessesProperty Calibration definitional high confidence E03 Calibration measures quantify the discrepancy between predicted probabilities and observed frequencies in a specified population. arXiv:1706.04599
Evaluation metrics assessesProperty Faithfulness definitional high confidence E04 Scope: explanation-specific metrics, which quantify agreement between attributed importance and responses under a defined perturbation distribution. arXiv:1901.09392
Auditing assessesProperty Plausibility methodological high confidence E05 Blinded clinical review can assess whether an explanation looks reasonable for the task, separately from computational testing. no source asserted
Subgroup & fairness assessesProperty Calibration methodological high confidence E06 Calibration can be assessed within prespecified patient or acquisition groups; this does not establish fairness under every criterion. no source asserted
Auditing assessesProperty Calibration methodological high confidence E07 An audit can compare frozen-model probabilities with independently obtained outcomes on a declared evaluation population. no source asserted
Exploration & stress testing assessesProperty Robustness methodological high confidence E61 Performance under constructed perturbation is the operational assessment of robustness, bounded by the perturbation family chosen. no source asserted
Sensitivity analysis assessesProperty Robustness methodological high confidence E66 Varying one design decision at a time measures how much the reported result depends on it, for the decisions someone chose to vary. no source asserted
Hypothesis testing assessesProperty Calibration methodological medium confidence E74 A goodness-of-fit test compares predicted frequencies against observed ones, subject to the test's own independence assumptions. no source asserted
Intervention & ablation assessesProperty Translation invariance methodological high confidence E93 Shifting the input and measuring the output is a direct test of invariance against equivariance. Classification and localization can be served by separate heads, one invariant and one equivariant, so the two requirements need not conflict. arXiv:1605.06409
Sensitivity analysis assessesProperty Stability margin methodological high confidence E107 Sweeping a parameter and locating where behaviour changes qualitatively measures how far the current operating point sits from that boundary. no source asserted
Identifiability analysis assessesProperty Causal identifiability methodological high confidence E114 An identification argument specifies the causal target and checks which assumptions connect it to observations; exchangeability, consistency and positivity are one route for treatment effects, not assumptions verified by predictive accuracy. no source asserted
Singular value analysis assessesProperty Measurement identifiability methodological high confidence E120 For the specified linear forward model y = A*x, a nonzero null-space vector v makes x and x+v observationally identical. Full column rank removes that ambiguity in the noiseless model, but does not validate the model itself. no source asserted
Singular value analysis assessesProperty Inverse-problem conditioning methodological high confidence E121 Inverting a linear forward model divides each observable singular-direction component by its singular value, so small nonzero values amplify measurement error. Regularization can suppress that amplification by changing the estimator, without recovering discarded information. no source asserted
Control stability analysis assessesProperty Stability margin methodological high confidence E130 Given an explicit plant, controller, delay model and perturbation family, pole or frequency-domain analysis can locate a stability boundary. The resulting margin belongs to those dynamics, rather than to a classifier's held-out accuracy. no source asserted
Diagnostic reasoning assessesProperty Pretest probability methodological high confidence E136 The probability before an index test depends on presentation, history and the referral pathway. Prevalence among patients selected for surgery is not automatically the relevant starting probability for a consecutive outpatient examination. no source asserted
Bayesian decision analysis assessesProperty Clinical utility methodological high confidence E139 Expected losses can compare actions under an explicit probability model and clinically justified consequences. This assesses potential value under assumptions; it does not demonstrate that deploying the recommendation improves care. no source asserted
Reference standard adjudication assessesProperty Clinical validity methodological high confidence E148 Checking how labels were established tests whether model outputs are being compared with the intended clinical condition. It includes lesion and time correspondence and whether the reference incorporated the index test, rather than assuming a label name guarantees validity. no source asserted
Clinical feature annotation assessesProperty Clinical assessability methodological high confidence E150 Calling a finding absent requires enough coverage, resolution and appropriate views to assess it. Some findings require a sequence or manoeuvre, so an apparently negative still frame may instead be unassessable. no source asserted
Clinical feature annotation assessesProperty Clinical evidence reliance methodological high confidence E151 Independent feature labels permit comparisons between model behaviour and the clinical evidence available in each case. Correlation or decodability alone remains insufficient; the assessment needs a design that distinguishes feature availability from actual diagnostic use. no source asserted
Intervention & ablation assessesProperty Clinical evidence reliance methodological high confidence E152 A controlled edit can test whether output responds to a nominated clinical finding while preserving competing evidence. The conclusion is conditional on the edit changing that finding without introducing other diagnostic or acquisition changes. no source asserted
Comparative workflow evaluation assessesProperty Clinical utility methodological high confidence E156 Comparing care with and without the intended use of the system can assess effects on decisions, outcomes, harms and workload. A causal interpretation requires a design addressing alternative explanations for differences between the care pathways. no source asserted
Internal validation assessesProperty Calibration methodological high confidence E162 Predictions on held-out patients can be compared with observed frequencies within the sampled setting, accounting for repeated observations per patient. The assessment remains conditional on that population, label process and prediction horizon. no source asserted
External validation assessesProperty Transportability methodological high confidence E164 Evaluating a frozen model and preprocessing pipeline in another specified setting tests performance under that setting's acquisition, case mix and reference process. This directly supports a claim about the evaluated conditions; broader transport requires an additional argument. no source asserted
Sensitivity analysis assessesProperty Transportability methodological high confidence E168 Varying specified acquisition conditions, case mix or prevalence can expose which assumptions a transport claim depends on. Such checks support only the perturbations actually represented; they are not a substitute for evidence from every proposed setting. no source asserted

probesThreat

Method → Threat

Can investigate a hypothesis about the threat; nominating a candidate is not confirming a mechanism.

Auditing probesThreat Shortcut learning methodological high confidence E08 Shortcut hypotheses are explicit audit targets requiring task-specific investigation, not aggregate accuracy. doi:10.1016/S2589-7500(22)00003-6
Intervention & ablation probesThreat Shortcut learning empirical high confidence E09 Controlled changes to suspected non-diagnostic cues reveal predictor sensitivity; shown on COVID-19 radiograph classifiers, not gallbladder models. doi:10.1038/s42256-021-00338-7
Counterfactual probesThreat Shortcut learning empirical medium confidence E10 Generative comparisons can expose candidate cues, but coupled edits prevent treating each comparison as an isolated intervention. doi:10.1038/s42256-021-00338-7
Causal inference probesThreat Confounding definitional high confidence E11 An explicit causal model identifies candidate confounding paths relative to an estimand; it does not discover unmeasured confounders. doi:10.1214/09-SS057
Causal inference probesThreat Spurious correlation methodological high confidence E12 Causal analysis separates an observed association from the intervention claim it might be taken to support. doi:10.1214/09-SS057
Subgroup & fairness probesThreat Shortcut learning methodological medium confidence E13 Stratifying by suspected cue availability can reveal shortcut-compatible failures; severity and case-mix remain alternative explanations. no source asserted
Attribution & saliency probesThreat Shortcut learning empirical medium confidence E14 Attributions nominate suspicious regions for controlled testing; they do not independently establish reliance. doi:10.1038/s42256-021-00338-7
Exploration & stress testing probesThreat Shortcut learning methodological high confidence E59 Deliberately constructed conditions can surface a reliance a fixed suite never sampled; absence of a failure under testing is not absence of the reliance. arXiv:2004.07780
Exploration & stress testing probesThreat Distribution shift methodological high confidence E60 Perturbation and stress conditions stand in for shifts that have not yet occurred, and only for the shifts someone thought to construct. no source asserted
Sensitivity analysis probesThreat Distribution shift methodological high confidence E67 Leave-one-site-out and cohort re-weighting rehearse a shift using variation already present in the assembled data, and reach no further than that variation. no source asserted
Out-of-distribution detection probesThreat Distribution shift methodological high confidence E69 A support or density estimate can flag inputs unlike the training set, for the notion of likeness the model was given. doi:10.1162/089976601750264965
Sensitivity analysis probesThreat Deployment feedback loop methodological medium confidence E82 Comparing a prospectively recruited stream against operational data can expose divergence the operational data alone cannot show. no source asserted
Ensembling probesThreat Distribution shift methodological medium confidence E99 Members that converged to different modes tend to agree where the data constrained the function and diverge where they did not. The signal is informative but neither guaranteed nor specific to distribution shift. arXiv:2002.08791
Feedback correction probesThreat Distribution shift methodological high confidence E111 Measuring realized outcomes and correcting against them surfaces divergence between deployment and development conditions that an open pipeline never observes. no source asserted

canIntroduce

Method · Threat · EvidenceSource → Threat

Can create or enable the object. Neither inevitability nor sufficient causation is implied.

Confounding canIntroduce Spurious correlation definitional high confidence E15 A common-cause pathway produces association that does not represent the causal effect being estimated. doi:10.1214/09-SS057
Spurious correlation canIntroduce Shortcut learning empirical high confidence E16 An association available in training can support a rule that fails when it changes; learning that rule is not inevitable. arXiv:2004.07780
Representation learning canIntroduce Shortcut learning empirical high confidence E17 Learned representations can privilege easy predictive cues over the evidence the task intends. arXiv:2004.07780
Ultrasound acquisition canIntroduce Spurious correlation empirical high confidence E18 Acquisition and documentation workflows can couple annotations with the target imaged; demonstrated on fetal-ultrasound calipers. doi:10.1007/978-3-031-72111-3_59
Intervention & ablation canIntroduce Distribution shift empirical high confidence E19 Removing or replacing regions at test time creates inputs unlike training data; ROAR answers this by retraining, which changes the question. arXiv:1806.10758
Screening programs canIntroduce Distribution shift methodological high confidence E20 Moving a model from referred symptomatic patients into screening changes eligibility, prevalence and disease spectrum. no source asserted
Deployment & drift canIntroduce Distribution shift empirical high confidence E21 Moving between institutions changes acquisition and patient distributions, as shown for pneumonia models across hospitals. doi:10.1371/journal.pmed.1002683
Similarity & graph construction canIntroduce Spurious correlation methodological high confidence E72 A kernel or affinity matrix asserts which cases count as comparable; if acquisition rather than biology drives that similarity, downstream structure inherits it. no source asserted
Preprocessing & resampling canIntroduce Spurious correlation methodological medium confidence E76 Resampling without anti-aliasing can stamp a ratio-dependent texture on an image, and the ratio may follow the acquiring scanner. Whether it appears depends on the signal content and the sampling, so it is a mechanism to rule out rather than a universal artefact. doi:10.1109/CVPR52688.2022.01112
Deployment & drift canIntroduce Deployment feedback loop methodological high confidence E79 A model whose outputs influence the care that generates its next training data appears in its own input distribution. arXiv:2002.06673
Deployment feedback loop canIntroduce Spurious correlation methodological high confidence E81 Under-flagging a subgroup yields fewer confirmed cases in it, which trains the next model to under-flag it further. arXiv:1706.09847
Objective design canIntroduce Proxy objective definitional high confidence E83 Choosing a differentiable stand-in for a non-differentiable evaluation target is what creates the gap; the gap follows from the choice, not from poor execution. no source asserted
Proxy objective canIntroduce Shortcut learning methodological high confidence E85 A shortcut is precisely a solution scoring well on the proxy and badly on the target, so proxy optimization is the mechanism that rewards it. no source asserted
Objective design canIntroduce Outlier influence definitional high confidence E94 A squared residual grants a point influence proportional to its error, so the most anomalous record outweighs many sound ones. doi:10.1214/aoms/1177703732
Pseudo-labelling canIntroduce Label noise methodological high confidence E101 A teacher's outputs become the student's training targets, so the teacher's errors are inherited by construction rather than averaged away. doi:10.1109/CVPR42600.2020.01070
Pseudo-labelling canIntroduce Spurious correlation methodological medium confidence E102 Systematic teacher error can propagate or amplify: a subgroup the teacher under-detects can train the student to under-detect it further. Random error does not generally cancel back to the true targets, and amplification is not inevitable either. doi:10.1109/CVPR46437.2021.01139
Feedback correction canIntroduce Deployment feedback loop definitional high confidence E112 Correcting on a system's own measured output is by construction a closed loop. Whether it converges or diverges depends on the gain around it, but naming that gain requires an explicit model of the update rule and its delays. no source asserted
Histopathology canIntroduce Selection & verification bias methodological high confidence E159 When histological verification is available mainly after surgery, restricting evaluation to verified cases conditions on the clinical decision to operate. The resulting case spectrum may differ from that of patients facing the original diagnostic decision. no source asserted

usesEvidence

Method → EvidenceSource · Finding

Observations from the object enter the subject's assessment or reasoning.

Diagnostic reasoning usesEvidence Sonographic features definitional high confidence E22 Observed morphology and acoustic findings feed task-specific reasoning without determining the diagnosis alone. no source asserted
Diagnostic reasoning usesEvidence Histopathology methodological high confidence E23 A matched tissue result can be incorporated while retaining specimen, sampling and timing limits. no source asserted
Diagnostic reasoning usesEvidence Cross-sectional imaging methodological high confidence E24 CT/MRI observations contribute when they address the unresolved clinical question. no source asserted
Auditing usesEvidence Sonographic features methodological high confidence E25 Independently assessed findings can anchor an ultrasound evidence audit; annotations derived from the explanation would be circular. no source asserted
Auditing usesEvidence Histopathology methodological high confidence E26 Matched tissue diagnoses support outcome verification but cannot substitute for annotations of image-visible evidence. no source asserted
Concept-based methods usesEvidence Sonographic features methodological medium confidence E27 Independently annotated sonographic findings can be operationalized as concepts, given validated measurement and separation. no source asserted
Bayesian inference usesEvidence Clinical history & examination methodological high confidence E134 Clinical context supplies information for the prior and likelihood appropriate to the target patient population. Information already included in the prior must not be counted again as independent new evidence in the diagnostic update. no source asserted
Bayesian decision analysis usesEvidence Clinical history & examination methodological high confidence E138 History and examination help specify candidate actions, contraindications and their consequences. A disease posterior becomes an action recommendation only after the clinically relevant consequences are assigned losses. no source asserted
Reference standard adjudication usesEvidence Histopathology methodological high confidence E145 Tissue findings can anchor a diagnostic label only through an explicit correspondence between specimen, lesion, patient and examination time. The same tissue diagnosis does not make its defining morphology visible in every associated image. no source asserted
Reference standard adjudication usesEvidence Longitudinal follow-up methodological high confidence E146 Follow-up can adjudicate later events or disease evolution, but loss of observation before the target horizon is not an automatic negative label. The reference must distinguish censoring, competing events and genuinely observed event-free follow-up. no source asserted
Reference standard adjudication usesEvidence Clinical history & examination methodological high confidence E147 Index-time symptoms and examination findings can form part of a clinical target definition. An image-only surrogate cannot silently replace them when the diagnosis depends on clinical context. no source asserted
Clinical feature annotation usesEvidence Sonographic features methodological high confidence E149 Independent annotation records which features are present, absent, uncertain or unassessable in the available examination. Blinding to model output and, where appropriate, the final diagnosis helps avoid writing the prediction back into its supposed evidence. no source asserted
Clinical feature annotation usesEvidence Electrophysiological features methodological high confidence E187 Annotation can distinguish a waveform event and its time interval from the patient's broader neurological diagnosis. This prevents a patient-level label from being treated as evidence that every sampled window contains a visible event. no source asserted

providesEvidenceFor

EvidenceSource · Finding → ClinicalTarget

Can supply observations bearing on the clinical question, without alone resolving it.

Ultrasound acquisition providesEvidenceFor Gallbladder disease empirical high confidence E28 Adequately acquired ultrasound supplies diagnostically relevant gallbladder observations. doi:10.1007/s13244-017-0544-7
Sonographic features providesEvidenceFor Gallbladder disease empirical high confidence E29 Findings such as intramural cystic spaces support particular differentials; no generic feature establishes every disease. doi:10.1007/s13244-017-0544-7
Sonographic features providesEvidenceFor Malignancy & staging empirical high confidence E30 Scope: malignancy-risk assessment of gallbladder polyps. Morphology and size inform risk, not a pathological stage. doi:10.1007/s00330-021-08384-w
Histopathology providesEvidenceFor Malignancy & staging empirical high confidence E31 Appropriately sampled tissue establishes malignant histology and sampled invasion, not necessarily the complete stage. no source asserted
Cross-sectional imaging providesEvidenceFor Malignancy & staging empirical high confidence E32 CT/MRI supplies evidence about tumour extent and spread, with limits for microscopic disease. doi:10.1259/bjr.20200726
Cross-sectional imaging providesEvidenceFor Biliary tract empirical high confidence E33 Scope: MRCP assessment of ductal configuration, narrowing and filling defects; these do not alone determine cause. no source asserted
Ultrasound acquisition providesEvidenceFor Liver & hepatobiliary empirical high confidence E34 Ultrasound supplies observations of parenchyma and the visible hepatobiliary system, subject to acoustic access. no source asserted
Cross-sectional imaging providesEvidenceFor Neurological disease empirical high confidence E35 Scope: CT/MRI in stroke assessment, including haemorrhage or ischaemic injury; not every neurological diagnosis. doi:10.3174/ajnr.A3690
Histopathology providesEvidenceFor Gallbladder disease empirical high confidence E36 A matched specimen provides tissue-level evidence for the sampled lesion; surgical selection limits generalization. no source asserted
Clinical history & examination providesEvidenceFor Cardiovascular risk methodological high confidence E157 Baseline history and current treatment help define the patient and care setting for a risk estimate. Without their timing, observed risk can be confused with the risk before treatment or with treatment benefit. no source asserted
Longitudinal follow-up providesEvidenceFor Cardiovascular risk methodological high confidence E158 Risk over a specified horizon concerns future events, so later observations supply evidence that a baseline cross-section cannot provide. Censoring and competing events determine which conclusions those observations support. no source asserted
Duct dilatation providesEvidenceFor Biliary tract empirical high confidence E176 The location and extent of enlargement can help localize a biliary problem. They do not by themselves identify the cause, establish active obstruction or determine the presence of infection. no source asserted
Wall thickening providesEvidenceFor Gallbladder disease empirical high confidence E177 Thickness and its pattern bear on the differential diagnosis, but inflammation, benign remodelling and malignancy can overlap. Treating this broad finding as a cancer label discards the clinical alternatives. no source asserted
Intramural cystic spaces providesEvidenceFor Gallbladder disease empirical high confidence E178 Resolved intramural spaces can support a wall-remodelling explanation in the appropriate pattern. Their absence in a limited image is weaker evidence than their confident absence after an adequate examination. no source asserted
Posterior acoustic effects providesEvidenceFor Gallbladder disease empirical high confidence E179 A shadow behind a suspected calculus or comet-tail echoes arising in a wall can change the interpretation of a lesion. An audit that removes all tissue outside the lesion outline may remove precisely this evidence. no source asserted
Electrophysiological features providesEvidenceFor Neurological disease empirical high confidence E180 Electrical discharges can bear on an epilepsy assessment, but a discharge, a recorded clinical event and a diagnosis of recurrent seizure predisposition are different targets. A normal short recording does not universally exclude the disease. no source asserted
Cross-sectional imaging providesEvidenceFor Acute stroke triage methodological high confidence E192 Brain, vascular and tissue imaging can answer different urgent questions about haemorrhage, arterial occlusion and tissue state. The acquisition protocol determines which question an image can support; success on one task does not establish suitability for the others. no source asserted
Clinical history & examination providesEvidenceFor Gallbladder disease methodological high confidence E193 Symptoms and examination findings help distinguish an incidental structural finding from an acute clinical syndrome. An image showing a stone is not by itself a complete reference for diagnosing inflammation or deciding urgency. no source asserted

constrainsProcedure

Standard → Method · Metric

Specifies decision rules or reporting requirements. Compliance is not guaranteed.

Clinical guidelines constrainsProcedure Diagnostic reasoning methodological high confidence E37 Applicable guidelines specify how defined findings and risk factors inform a decision; the example is polyp management. doi:10.1007/s00330-021-08384-w
Clinical guidelines constrainsProcedure Screening programs methodological high confidence E38 A screening guideline supplies eligibility and follow-up rules against which a workflow can be specified and audited. no source asserted
Reporting standards constrainsProcedure Auditing methodological high confidence E39 Scope: diagnostic-accuracy components. STARD specifies reporting of participants, reference procedures and analysis. doi:10.1136/bmj.h5527
Reporting standards constrainsProcedure Evaluation metrics methodological high confidence E40 Reporting requires specified estimates and their precision; compliance does not make a metric appropriate for every claim. doi:10.1136/bmj.h5527
Reporting standards constrainsProcedure Operating point selection definitional high confidence E89 Reporting a threshold-free summary alone leaves the clinically consequential choice of operating point undocumented. doi:10.1371/journal.pone.0118432

warrantsReassessmentOf

Threat → Property · Method

Gives reason to reassess; does not assert that anything has deteriorated.

Distribution shift warrantsReassessmentOf Calibration methodological high confidence E41 Calibration established on one distribution need not persist on another, so reassessment is warranted. arXiv:1906.02530
Distribution shift warrantsReassessmentOf Uncertainty & abstention methodological high confidence E42 Shift motivates checking uncertainty quality and the risk and coverage of any acceptance policy. arXiv:1906.02530
Distribution shift warrantsReassessmentOf Subgroup & fairness methodological high confidence E43 Changed populations or acquisition conditions require reassessing subgroup performance; aggregate results cannot answer it. no source asserted
Distribution shift warrantsReassessmentOf Robustness methodological high confidence E64 A realized shift may or may not fall inside the perturbation family over which robustness was assessed; it warrants checking that coverage rather than establishing that the assessment no longer applies. no source asserted
Shortcut learning warrantsReassessmentOf Out-of-distribution detection methodological medium confidence E70 If a site signature is part of what the detector learned to call normal, it can flag a new scanner while passing novel pathology from the familiar one. This is a failure mode to test for, not a guaranteed behaviour. no source asserted
Deployment feedback loop warrantsReassessmentOf Calibration methodological high confidence E80 Once the data reflect the model's own influence, agreement between predictions and recorded outcomes still establishes calibration under that policy and observation process, but no longer under an untreated or differently acted-upon population. no source asserted
Proxy objective warrantsReassessmentOf Faithfulness definitional high confidence E84 A model optimized toward a stand-in has been selected for scoring well on the stand-in; that its reliance matches the target is a question the training never posed. no source asserted
Proxy objective warrantsReassessmentOf Auditing definitional high confidence E86 When the audit's own measure is itself a stand-in for what is being audited, the audit inherits the gap it was meant to detect. no source asserted
Proxy objective warrantsReassessmentOf Uncertainty & abstention definitional medium confidence E87 Confidence calibrated against the training objective expresses certainty about the stand-in, which is not certainty about the clinical target. no source asserted
Outlier influence warrantsReassessmentOf Robustness definitional high confidence E95 A fit skewed by a few contaminated records reports an ordinary residual and an ordinary summary; nothing in the output marks the disproportion. no source asserted
Shortcut learning warrantsReassessmentOf Ensembling methodological high confidence E100 Members sharing an architecture, a training set and a preprocessing pipeline share the biases those impose, so a shortcut present in the data can leave them agreeing confidently and wrong together. Agreement is therefore weak evidence of correctness under a shared shortcut. no source asserted
Label noise warrantsReassessmentOf Auditing definitional high confidence E103 When the reference standard is itself several points from correct, disagreement between model and standard no longer separates model error from annotation error. arXiv:2006.07159
Label noise warrantsReassessmentOf Calibration definitional high confidence E104 Probabilities fitted to agree with noisy labels are calibrated to the annotation process, and the agreement is evidence about that process rather than about the condition. arXiv:2006.07159
Deployment feedback loop warrantsReassessmentOf Stability margin methodological medium confidence E110 A path from a model's outputs back into its own training data is a feedback path, so a margin measured on the open pipeline need not bound the deployed one. The control reading holds only where the dynamics, delays and update rule are made explicit; a learning loop does not automatically possess a defined gain margin. no source asserted
Selection & verification bias warrantsReassessmentOf Pretest probability definitional high confidence E160 Prevalence among verified or referred cases need not equal prevalence among consecutive intended-use patients. The selection mechanism must be examined before using that frequency as a clinical prior. no source asserted
Selection & verification bias warrantsReassessmentOf Clinical validity definitional high confidence E161 Accuracy conditional on receiving a reference examination is not automatically accuracy in the target population. Differential verification or missing outcomes can change which errors are observable. no source asserted
Distribution shift warrantsReassessmentOf Pretest probability definitional high confidence E167 A change in referral or recruitment may alter baseline disease odds. Retaining the development prior requires evidence about the new clinical population, rather than an assumption that all shifts leave prevalence unchanged. no source asserted
Deployment feedback loop warrantsReassessmentOf Transportability definitional high confidence E191 If deployment changes who is tested, treated or labelled, the future data-generating process can depend on the policy itself. Validation under the previous care policy does not by itself cover the resulting feedback regime. no source asserted

doesNotEstablish

Property · Method · Metric · EvidenceSource · Standard → Property · Finding

The subject alone is insufficient to establish the object. Non-entailment, not incompatibility.

Plausibility doesNotEstablish Faithfulness definitional high confidence E44 A visually convincing explanation can be insensitive to learned parameters, so reasonableness does not entail faithfulness. arXiv:1810.03292
Faithfulness doesNotEstablish Plausibility definitional high confidence E45 A faithful explanation of caliper reliance accurately describes an inappropriate rule without making it clinically plausible. no source asserted
Calibration doesNotEstablish Faithfulness definitional high confidence E46 Agreement between probabilities and outcome frequencies does not constrain whether an explanation describes the predictor. no source asserted
Evaluation metrics doesNotEstablish Faithfulness definitional high confidence E47 Scope: predictive-performance metrics such as AUROC. They describe predictions against labels, not the computation. no source asserted
Segmentation & detection doesNotEstablish Faithfulness definitional high confidence E48 Localization or mask overlap does not establish that an explanation identifies the evidence a diagnostic predictor used. no source asserted
Histopathology doesNotEstablish Sonographic features definitional high confidence E49 A tissue diagnosis does not establish that a finding was visible or assessable in the frame supplied to the model. no source asserted
Reporting standards doesNotEstablish Faithfulness definitional high confidence E50 A compliant report can describe a method whose faithfulness claim remains unsupported. no source asserted
Evaluation metrics doesNotEstablish Robustness definitional high confidence E63 A score on a held-out sample from one distribution quantifies performance on that distribution and asserts nothing about behaviour under shift. no source asserted
Robustness doesNotEstablish Faithfulness definitional high confidence E65 Stability under perturbation of the learned representation is compatible with that representation encoding a shortcut; a wide margin around the wrong feature is still a wide margin. no source asserted
Evaluation metrics doesNotEstablish Plausibility definitional high confidence E68 A performance figure quantifies agreement with labels and carries no claim about whether the model's reasoning would be recognized as sensible. no source asserted
Out-of-distribution detection doesNotEstablish Robustness definitional high confidence E71 Declining to answer on inputs judged unfamiliar is not evidence of correct behaviour on inputs judged familiar. no source asserted
Hypothesis testing doesNotEstablish Robustness definitional high confidence E75 A p-value is conditional on the null being true and supplies no probability that any hypothesis holds, let alone that performance transports. doi:10.1080/00031305.2016.1154108
Translation invariance doesNotEstablish Faithfulness definitional high confidence E92 A representation trained to be unchanged by position has been rewarded for discarding the spatial correspondence that an attribution map asserts. no source asserted
Attention mechanism doesNotEstablish Faithfulness definitional high confidence E97 An attention weight reports how much a value vector was routed into an update; by the second layer that vector already summarizes much of the sequence, so the weight is a routing diagram and not an attribution. arXiv:1706.03762
Ensembling doesNotEstablish Calibration definitional high confidence E98 Spread across independently initialized models supplies a predictive distribution where a point estimate supplies only a score, but the spread is a property of the ensemble, not a comparison with observed outcomes. Calibration assessment requires that comparison. arXiv:1612.01474
Evaluation metrics doesNotEstablish Stability margin definitional high confidence E108 A point estimate on a fixed test set reports current performance; it says nothing about how far inputs or thresholds can move before that performance fails. no source asserted
Operating point selection doesNotEstablish Stability margin definitional high confidence E109 Choosing a threshold fixes an operating point without quantifying how much the score distribution may shift before the choice stops holding. no source asserted
Causal identifiability doesNotEstablish Clinical utility definitional high confidence E115 Identifying a treatment effect does not establish that a learned decision rule improves care: the policy, patient population, competing consequences and implementation still need assessment. no source asserted
Simulation-based policy evaluation doesNotEstablish Clinical utility definitional high confidence E118 A favourable simulated return is conditional on the state, action, reward and transition model. It is not an observed improvement in patient outcomes, and a reward can omit harms that matter clinically. no source asserted
Constrained optimization doesNotEstablish Clinical validity definitional high confidence E119 Even a certified global optimum only solves the stated objective over the stated feasible set; it cannot repair a mismatched clinical target, an invalid reference label or a constraint that encodes the wrong requirement. no source asserted
Measurement identifiability doesNotEstablish Clinical assessability definitional high confidence E122 A unique solution in a noiseless model can remain too sensitive to noise or finite resolution to support a finding in the actual examination. Uniqueness and usable measurement precision are different requirements. no source asserted
Evaluation metrics doesNotEstablish Inverse-problem conditioning definitional high confidence E123 Error on a fixed test set does not report how much a reconstructed quantity can change under measurement perturbations; that requires a forward model, perturbation scale and sensitivity assessment. no source asserted
Inverse reconstruction doesNotEstablish Sonographic features definitional high confidence E128 A structure present in a reconstructed image may be supplied by the reconstruction prior rather than resolved by the acquired echoes. Its appearance alone does not establish that the corresponding finding was measured in this patient. no source asserted
Stability margin doesNotEstablish Clinical utility definitional high confidence E131 A loop can remain stable while regulating the wrong quantity or pursuing a clinically harmful target. Distance from an instability boundary does not establish benefit from the action being stabilized. no source asserted
Bayesian inference doesNotEstablish Calibration definitional high confidence E135 A posterior is conditional on its probability model and prior; those choices can be wrong for the deployment population. Calling an output Bayesian does not replace checking agreement between its probabilities and observed outcomes. no source asserted
Faithfulness doesNotEstablish Clinical evidence reliance definitional high confidence E153 An explanation can accurately report that the model used a site marker or another shortcut. Correctly describing the computation does not make its evidence clinically appropriate. no source asserted
Clinical validity doesNotEstablish Clinical utility definitional high confidence E154 An output can correspond accurately to the intended clinical condition without improving decisions or patient outcomes. Clinical benefit additionally depends on what users do with it and on the consequences compared with usual care. no source asserted
Hypothesis testing doesNotEstablish Clinical validity definitional high confidence E155 Rejecting a statistical null under the sampling model cannot repair a wrong target, a biased reference process or an incorrect unit of analysis. Statistical significance does not turn those observations into evidence for the intended clinical claim. no source asserted
Internal validation doesNotEstablish Transportability definitional high confidence E163 An independent split can still share acquisition, referral, treatment and labelling mechanisms with development. Independence of held-out patients does not test whether the model's behaviour persists when those mechanisms change. no source asserted
External validation doesNotEstablish Clinical evidence reliance definitional high confidence E165 A shortcut can persist across institutions, so successful external prediction does not identify which evidence the model used. Testing transport and testing clinical reliance answer different questions. no source asserted
External validation doesNotEstablish Clinical utility definitional high confidence E166 Retrospective performance on another site's cases does not show how clinicians act on the output or whether outcomes improve. Those consequences require assessment of the intended use within care. no source asserted
Clinical validity doesNotEstablish Clinical evidence reliance definitional high confidence E169 Agreement with a valid clinical target can coexist with reliance on a correlated care-process cue. A correct diagnosis and a correct account of which clinical evidence produced it are separate claims. no source asserted
Representation learning doesNotEstablish Transportability definitional high confidence E195 A descriptor fitted as a free per-case vector has no rule mapping a new case's observations to a vector, so nothing in the fitted table applies to a case absent at training time. Optimizing an appended row afterwards is additional fitting, not inference through the trained model. no source asserted

canAudit

Method → Method · ModelFamily · EvidenceSource

Instances or outputs of the object can be explicit inspection targets of the subject.

Auditing canAudit Foundation models methodological high confidence E51 An adapted instance can be frozen and audited for its evidence use; pretraining scale is not an exemption. no source asserted
Auditing canAudit Representation learning methodological high confidence E52 Representations can be inspected for encoded clinical and acquisition information, keeping decodability separate from reliance. no source asserted
Auditing canAudit Segmentation & detection methodological high confidence E53 Segmentation outputs can be audited for dependence on documentation cues. doi:10.1007/978-3-031-72111-3_59
Auditing canAudit Attribution & saliency methodological high confidence E54 Parameter and label randomization test whether attribution outputs respond to the learned predictor at all. arXiv:1810.03292
Auditing canAudit Concept-based methods methodological high confidence E55 Audit the concept examples, held-out measurement quality, correlated concepts and claimed sensitivity. no source asserted
Auditing canAudit Counterfactual methodological high confidence E56 Inspect whether an edited comparison changes its intended target while preserving other evidence. no source asserted
Exploration & stress testing canAudit Foundation models methodological medium confidence E62 Stress conditions can be applied to a fixed model's outputs without access to its training data. no source asserted
Subgroup & fairness canAudit Similarity & graph construction methodological medium confidence E73 Degree normalization decides how far the densely connected majority sets the geometry, so the graph construction is itself a legitimate target of subgroup analysis. no source asserted
Auditing canAudit Preprocessing & resampling methodological high confidence E78 The pipeline that produced the inputs is itself an inspection target, and one that attribution over image content cannot reach. no source asserted
Operating point selection canAudit Subgroup & fairness methodological high confidence E88 One threshold applied across groups of differing prevalence produces different error profiles per group; sweeping it exposes that dependence. doi:10.1145/3287560.3287598
Auditing canAudit Post-processing methodological high confidence E91 The steps between a network's output and the reported result are a separate inspection target; a fault there is invisible to any method examining only the model. no source asserted
Auditing canAudit Objective design methodological high confidence E96 Loss choice and per-sample weighting are where class priorities are actually encoded; reading them is more direct than inferring priorities from performance. no source asserted
Auditing canAudit Pseudo-labelling methodological high confidence E105 The labelling model is an inspection target in its own right, and one that evaluating the final model against those same labels cannot reach. no source asserted

changesObservabilityOf

EvidenceSource · Method → Finding

Can change the visibility or assessability of findings without a change in underlying disease.

Ultrasound acquisition changesObservabilityOf Sonographic features empirical high confidence E57 Acquisition modes alter acoustic findings; spatial compounding changed shadow appearance in a stone phantom. doi:10.1148/rg.2017160175
Intervention & ablation changesObservabilityOf Sonographic features methodological high confidence E58 Masking can erase diagnostically relevant morphology as well as the suspected nuisance cue. no source asserted
Preprocessing & resampling changesObservabilityOf Sonographic features methodological high confidence E77 Interpolation and filtering can suppress fine texture that a reader or model would otherwise assess. no source asserted
Post-processing changesObservabilityOf Sonographic features methodological high confidence E90 Overlap-based suppression deletes true detections lying closer than its threshold, after the model has already scored them. doi:10.1109/TPAMI.2016.2577031
Implicit representation changesObservabilityOf Sonographic features methodological medium confidence E106 A field queried between its training views renders interpolated structure with the same appearance as measured structure, and the output carries no marker separating them. doi:10.1007/978-3-030-58452-8_24
Ultrasound acquisition changesObservabilityOf Duct dilatation empirical high confidence E181 Duct calibre must be assessed in an adequately visualized segment and plane. Obscuration or incomplete coverage can make a normal-looking frame uninformative about the rest of the biliary tree. no source asserted
Ultrasound acquisition changesObservabilityOf Intramural cystic spaces empirical high confidence E182 Depth, focus, resolution and the chosen plane affect whether small intramural spaces can be resolved. A change in their visibility across images need not indicate a change in the wall's underlying structure. no source asserted
Ultrasound acquisition changesObservabilityOf Posterior acoustic effects empirical high confidence E183 Beam angle and compounding can change the appearance of shadows and reverberation. Their stability under an image transformation cannot be assumed merely because the biological lesion is unchanged. no source asserted
Preprocessing & resampling changesObservabilityOf Intramural cystic spaces methodological high confidence E184 Reducing spatial resolution can remove a small resolved cavity while preserving the overall thickened-wall outline. Preserving the coarse diagnostic label therefore does not guarantee preservation of the evidence used to justify it. no source asserted
Electrophysiological recording changesObservabilityOf Electrophysiological features empirical high confidence E186 Electrode placement, montage, filtering and recording duration affect which electrical phenomena can be observed. Failure to capture an intermittent event is a limit of the observation as well as a possible clinical negative. no source asserted

limitsClaim

Principle · Property → Method · Metric · Property

Constrains what can be inferred or jointly guaranteed under the assumptions named in the assertion; does not assert an empirical failure.

Identifiability criterion limitsClaim Causal inference definitional high confidence E113 If two causal models satisfy the stated assumptions and generate the same observational distribution but disagree on the target effect, that effect is not identified; more observations from that same distribution cannot resolve the disagreement. no source asserted
Identifiability criterion limitsClaim Counterfactual definitional high confidence E116 An identified interventional distribution need not identify an individual counterfactual. Changing pixels tests a computational input intervention; interpreting the result as what would have happened to the patient requires an additional causal argument. no source asserted
Identifiability criterion limitsClaim Simulation-based policy evaluation definitional high confidence E117 Dynamics can agree on every observed action yet disagree on actions absent from the records. A simulator chooses among those possibilities through assumptions; agreement with retrospective trajectories alone does not identify the value of an unsupported policy. no source asserted
Sampling theorem limitsClaim Measurement identifiability definitional high confidence E124 For a signal band-limited to B, ideal noiseless uniform sampling above 2B permits reconstruction. If the admissible signal class instead contains aliases, distinct continuous signals can produce the same samples and the original is not uniquely determined. no source asserted
Sampling theorem limitsClaim Preprocessing & resampling definitional high confidence E125 Before decimation, frequencies outside the new representable band must be removed if aliasing is to be excluded for arbitrary inputs. Upsampling after folding has occurred cannot determine which original frequency produced a sampled component. no source asserted
Data processing inequality limitsClaim Representation learning definitional high confidence E126 For a fixed encoder whose only patient-specific input is the image, processing cannot increase mutual information with disease relative to that input. It can make existing information easier for a chosen decoder to use; dependence still does not establish causal relevance. no source asserted
Data processing inequality limitsClaim Inverse reconstruction definitional high confidence E127 With a fixed reconstruction model and no additional patient measurements, a prior can select or average among compatible images but cannot add patient-specific information absent from the input. A plausible reconstruction is therefore not a new observation. no source asserted
Feedback sensitivity identity limitsClaim Feedback correction definitional high confidence E129 For the scalar negative-feedback loop S = 1/(1+L) and T = L/(1+L), S+T=1 and hence |S|+|T|>=1 wherever defined. Suppressing output-disturbance transmission through S leaves sensor-noise transmission through T near unity; this is not a theorem about arbitrary clinical learning loops. no source asserted
Bias-variance decomposition limitsClaim Evaluation metrics definitional high confidence E132 Under squared loss with conditionally zero-mean test noise independent of training, expected prediction error decomposes into squared bias, estimator variance and irreducible noise. A single test score does not identify those terms, and the identity does not impose a conserved budget preventing bias and variance from both improving. no source asserted
Bayes error bound limitsClaim Operating point selection definitional high confidence E133 With nonzero class priors and overlapping class-conditional distributions on a set of positive probability, the fixed measurements imply positive Bayes classification error. A threshold cannot make both false positives and false negatives vanish; precision and recall nevertheless have no universal conserved sum, and better measurements can change the bound. no source asserted
Pretest probability limitsClaim Predictive values definitional high confidence E137 Bayes' rule gives PPV = Se*pi/(Se*pi + (1-Sp)*(1-pi)) for sensitivity Se, specificity Sp and prevalence pi. Transferring a positive predictive value therefore requires the target prevalence and justification that the conditional test characteristics also transfer. no source asserted
Clinical assessability limitsClaim Clinical evidence reliance definitional high confidence E185 When the acquisition cannot resolve a specified finding, agreement with a patient diagnosis cannot establish reliance on that finding in this image. Prediction may instead depend on other measured evidence or on prior information; human difficulty alone is not proof of physical non-observability. no source asserted
Data processing inequality limitsClaim Similarity & graph construction definitional high confidence E194 Each message-passing layer is a function of the previous layer's states, so a node descriptor can carry no information about a relation that the edge rule did not already supply. Repeated neighbourhood averaging contracts the differences it does not preserve; adding depth cannot recover them. no source asserted

setsRequirementsFor

ClinicalTarget → Method · Metric

The clinical question determines which target, unit, timing, evidence, or consequence the procedure or metric must address.

Gallbladder disease setsRequirementsFor Intended use specification definitional high confidence E140 Detecting a stone, diagnosing acute inflammation and characterizing possible malignancy are different questions. Each requires its own target, unit of analysis, available inputs and intended clinical action before a performance claim has a determinate meaning. no source asserted
Malignancy & staging setsRequirementsFor Reference standard adjudication definitional high confidence E141 A specimen-level cancer diagnosis must be matched to the imaged lesion and relevant examination time. Patient-level staging cannot simply become a positive label for every image frame. no source asserted
Cardiovascular risk setsRequirementsFor Reference standard adjudication definitional high confidence E142 A prognostic label needs a baseline, prediction horizon and outcome definition, with explicit treatment of censoring and competing events. Outcomes observed under treatment do not directly label the patient's untreated risk. no source asserted
Acute stroke triage setsRequirementsFor Evaluation metrics definitional high confidence E143 A triage claim must account for which urgent cases are missed, which alerts consume attention and when a result changes action. Ranking accuracy or isolated inference time does not measure the full clinical delay. no source asserted
Neurological disease setsRequirementsFor Clinical feature annotation methodological high confidence E144 A visible movement, an electrical discharge and a longitudinal neurological diagnosis are different annotation targets. The protocol must state the observation interval and relevant clinical state rather than treating an unrecorded phenomenon as absent. no source asserted
Gallbladder disease setsRequirementsFor Intervention & ablation methodological high confidence E188 For an audit of stone or wall-lesion evidence, preserving the lesion outline is insufficient if the edit destroys posterior shadowing or intramural detail. Clinical interpretation determines which surrounding signal must remain controlled. no source asserted
Acute stroke triage setsRequirementsFor Comparative workflow evaluation definitional high confidence E189 A claim of faster clinically useful triage requires measuring the path from acquisition to review and action, including alert burden and downstream delays. Faster computation alone cannot establish a shorter treatment pathway. no source asserted
Cardiovascular risk setsRequirementsFor Bayesian decision analysis definitional high confidence E190 Choosing prevention requires expected benefits and harms of an action, not only the probability of an event under observed care. A risk predictor must not be silently reinterpreted as an estimate of treatment effect. no source asserted

canManifestAs

Mechanism → Finding

Can generate the finding under specified biological or measurement conditions; neither necessary presence nor diagnostic specificity is implied.

Biliary obstruction canManifestAs Duct dilatation empirical high confidence E170 Impaired bile outflow can produce upstream duct enlargement, but its visibility depends on timing and context. Dilatation can persist after obstruction is relieved, and a nondilated duct does not universally exclude obstruction. no source asserted
Inflammatory change canManifestAs Wall thickening empirical high confidence E171 Inflammation can produce gallbladder wall oedema and thickening. The morphology is not specific to inflammation, so the same measurement needs clinical context and competing explanations. no source asserted
Neoplastic invasion canManifestAs Wall thickening empirical high confidence E172 An infiltrating gallbladder malignancy can present as wall thickening rather than a discrete mass. Because benign processes can produce the same broad finding, detecting thickness alone does not identify malignant invasion. no source asserted
Benign wall remodelling canManifestAs Intramural cystic spaces empirical high confidence E173 In gallbladder adenomyomatosis, wall remodelling can include intramural sinus spaces that appear cystic when resolved. This provides a benign structural explanation to consider rather than treating every thickened wall as malignancy. no source asserted
Acoustic propagation canManifestAs Posterior acoustic effects empirical high confidence E174 Reflection, reverberation and attenuation along the beam path can produce posterior shadows or comet-tail echoes. Such appearances arise from tissue interacting with acquisition and can carry diagnostic information rather than being disposable nuisance texture. no source asserted
Neuronal network activity canManifestAs Electrophysiological features empirical high confidence E175 Electrical activity can generate recordable field patterns, but the measured waveform depends on spatial summation, source orientation and recording conditions. Disease need not produce a diagnostic event during the available recording interval. no source asserted

Absence of an edge means not asserted, not false. doesNotEstablish is itself a positive claim of non-entailment, not a claim of incompatibility. None of this determines whether a particular trained model uses a shortcut; that needs an experiment.