omicstra · TNBC-92 results

Wang et al. 2024, 92 TNBC patients · 260 matched subarrays · 208,786 niches (v3) · 35,594 test niches · 14 held-out patients · patient-stratified 85/15 split, seed=42 · v1 numbers noted where v3 DAG decomposition deferred

what H&E gives us

Morphology at scale

Standard of care in oncology - routinely generated at diagnosis and archived at scale across institutions. Encodes tissue architecture, immune geography, stromal composition. No molecular read-out. Some morphological features - TLS in particular - are directly decodable from foundation model features (B4 random-init MLP on Virchow2 features achieves z_TLS = 25.5 cross-patient, no alignment training required - the TLS signal is in Virchow2's feature space). Others require molecular context to distinguish (immune-hot vs immune-exhausted stroma looks similar morphologically).

what spatial omics gives us

Molecules at location

ST measures which genes are expressed and where. Molecularly rich but research-use-only and hard to interpret without morphological context - knowing a spot expresses CD20, CD8, and IL-10 means more when you can see it sits at the edge of an organized immune aggregate in H&E - CD20 and CD8 placing B and T cells spatially, IL-10 flagging local immunosuppression. The shared space makes this cross-reference systematic and scalable.

The gap omicstra addresses. Spatial omics is generated on research cohorts; H&E is archived at clinical scale. A shared representation space connecting the two means: (a) research cohort findings can be expressed in H&E language and applied to existing clinical archives, and (b) an H&E niche can query the space to surface molecular context for that specific tissue location. The downstream application is risk stratification - identifying which patients have the TME composition associated with checkpoint response or poor prognosis - from a modality that already exists at institutional scale.

Did we do what the proposal said we would do?

The ResearchHub proposal stated three hypotheses and three biological deliverables. Here is the direct mapping from stated goal to observed result.

Three biological deliverables from the proposal - what was delivered

1. Integrated multi-modal cohort characterization delivered

The proposal stated: "the shared embedding space combines morphological and transcriptomic signals into a joint representation more informative for TME characterization than either modality alone, enabling cross-modal retrieval that links tissue architecture to molecular state across patients." The result: R4 AUC 0.859 [0.857, 0.862] on cross-patient retrieval, H2-A 5x ARI lift over raw ST alone on 14-class niche-level tissue states. The shared space is more informative than either modality alone - confirmed.

2. TME readability profiles delivered - and richer than proposed

The proposal stated: "per-compartment signal-to-noise analysis quantifies where morphology and expression reinforce each other vs where they diverge." The result: the B.1/B.2 split makes this concrete. TLS is where morphology and expression strongly reinforce each other - Virchow2 encodes TLS without any alignment training (B4 z=25.5) and the late-fusion contrastive alignment also detects it categorically (R6/R1 pass compartment contrasts). Immune gradient stroma is where they diverge - morphologically ambiguous, transcriptomically distinct. The trade-off finding (contrastive training degrades TLS signal while gaining cross-modal coupling) is itself a readability profile: it tells you which signals transfer freely between modalities and which require a specific alignment strategy to preserve.

3. Pathway-morphology spatial maps delivered for cross-patient transferable signals

The proposal stated: "Reactome pathway embeddings projected into the shared space reveal where specific programs co-localize with tissue architecture, supporting hypothesis generation for TME biology." The result: R4 transfers 4/4 Reactome parent pathways cross-patient (z=11-25). At sub-pathway DAG resolution (v1): BTLA co-inhibition z=6.92, IL receptor SHC signaling z=6.80, Integrin cell surface interactions z=6.21, IFN-alpha/beta regulation z=5.71, Collagen biosynthesis z=5.22-5.28. These are spatially localizable from H&E via the shared space - directly supporting hypothesis generation about where immune checkpoint and stromal remodeling programs concentrate in tissue.

On the evaluation question quality

When asked: "when you say questions you basically want to make sure that the evaluation question is actually a good question to be asking in order to evaluate the objective" Yes - and this is exactly the tension the construct-validity audits resolved.

H2 original evaluation question

"Do the 9 spatial archetypes cluster more coherently in the aligned space?" - archetype is a valid annotation category, but assigned at patient level (NMI=0.89 with patient_id). This question evaluates patient identity, not niche biology. B1 CCA wins by amplifying patient identity 3.3x. The metric and the objective came apart.

H2 corrected evaluation question

"Do niches cluster by niche-level transcriptomic tissue state (mc_megacluster, 14-class NMF)?" - this varies within a patient, so it measures niche biology directly. On this question: 5x ARI lift, R5/R6/R1 win clustering, R4 encodes biology at linear probe parity. The alignment delivers what was proposed.

H3 original evaluation question

Within-cohort CCA: "do pathway embeddings correlate with shared-latent directions?" - at 512d and 35,594 niches, this is universally positive for all 10 runs. It confirms the hypothesis is trivially true but does not discriminate between methods. Not a useful evaluation question.

H3 corrected evaluation question

Cross-patient sub-split CCA: "does the correlation transfer to held-out patients?" - this discriminates. R4 4/4 pathways; late-fusion runs 0-2/4. The question quality determines what you can conclude. Better question -> sharper answer.

On risk stratification. The results don't directly predict clinical outcomes - prospective validation on a separate cohort with outcome data is the next required step. What the results do show: the pathway signals that transfer cross-patient (BTLA co-inhibition, ECM remodeling, IFN signaling) are the same programs oncologists use to reason about checkpoint response and stromal biology in TNBC. The shared space provides a systematic method for spatially localizing those signals from H&E - the modality that already exists at clinical scale. That spatial localization is the precondition for any H&E-based risk stratification approach. The routing table tells you which alignment method to use to generate those maps.

Architecture - what's actually being aligned

Each of the 10 runs maps an H&E representation and a spatial-transcriptomics representation into a shared 512-d space and trains them to bring matched (H&E, ST) niche pairs together. Runs differ in (a) which inputs they consume, (b) the fusion architecture, and (c) the loss. The H&E side is always Virchow2 (frozen foundation model, 1280-d per spot). The ST side is always Novae GNN (64-d, frozen) plus gpath2vec pathway embeddings (512-d) - except R6 which drops gpath2vec.

The four per-niche input features

A niche = center spot + 6 spatial neighbors = 7 spots (~1200 cells). Every run sees the same four per-niche features; they differ only in which they consume and how they fuse them.

feature	dim	what it is	what it captures
virchow2_niche	1280	the 7 spots' Virchow2 embeddings mean-pooled into one vector	the average H&E morphology of the niche. Used by every run except R4.
virchow2_cell_tokens (R4 only)	7 × 1280	the same 7 per-spot Virchow2 embeddings kept separate (not pooled)	within-niche morphological detail. Lets R4's cross-attention weight each spot independently instead of averaging.
novae_niche	64	Novae GNN (GAT) latent, mean-pooled over 7 spots, z-scored per subarray	spatially-aware ST expression: what the niche expresses given its spatial transcriptomic neighborhood.
gpath2vec_niche	512	metapath2vec embedding of the niche's pathway-enrichment context (v3 build, sha-locked)	which Reactome biological processes are active in the niche, mediated through TF-restricted Fisher enrichment. Niches with similar TF-driven pathway programs sit close together. Complements novae (program-level, not raw-expression-level).

How the inputs combine - two different "+" operations

When a run's input is written as Virchow2 niche + (novae + gpath2vec), the two + signs mean different things:

inner + (novae + gpath2vec) = a true concatenation into one 576-d ST feature vector (64 + 512). The ST side has one fused vector entering its MLP tower.
outer + (Virchow2 niche + (novae + gpath2vec)) = not a concatenation. The H&E vector and the ST vector are the two sides of the contrastive pair: each passes through its own MLP tower into the shared 512-d space, where InfoNCE / SupCon / Barlow pulls matched (H&E, ST) niche pairs together. The two modalities are never stacked into a single vector.

Run definitions (all 10)

H&E input column - two ways to use Virchow2. A niche = 1 center ST spot + its 6 immediate spatial neighbors = 7 spots. Virchow2 (1280-d) is run on each spot's H&E tile independently, giving 7 spot-level vectors per niche.

"Virchow2 niche" (9 of 10 runs) - the 7 spot vectors are mean-pooled into one 1280-d niche-level vector before the alignment MLP. Local within-niche heterogeneity is averaged away.
"Virchow2 7 tile tokens" (R4 only) - the 7 spot vectors are kept un-pooled as a (7, 1280) matrix and exposed as separate tokens to a cross-attention bridge, so the ST query learns which tile contributes how much. This is what "early interaction" means in this study.

run	loss + fusion	H&E input + ST input	what it tests
R4	InfoNCE · cross-attention	Virchow2 7 tile tokens + (novae + gpath2vec)	The only non-late-fusion run. ST embedding queries the 7 raw H&E tile tokens via attention, producing a weighted-sum H&E representation conditioned on ST. Captures which of the 7 spots matter for each niche instead of averaging them. ~4M params.
R1	InfoNCE · late (2 independent MLPs)	Virchow2 niche + (novae + gpath2vec)	CLIP-style baseline. H&E and ST each go through their own MLP; matched pairs pulled together, cross-patient mismatches pushed apart.
R2	SupCon · late	Virchow2 niche + (novae + gpath2vec)	Like R1 but with soft targets: niches with similar mc_weights (NMF cell-state mixture) are partial positives, not just the exact diagonal.
R3	Barlow Twins · late	Virchow2 niche + (novae + gpath2vec)	No explicit negatives. Drives H&E × ST cross-correlation toward identity (matched dimensions correlated, off-diagonals decorrelated).
R5	AnInfoNCE · late	Virchow2 niche + (novae + gpath2vec)	R1 + per-dimension learnable temperature on the bilinear similarity (diagonal Mahalanobis). Lets the loss weight some shared-space axes harder than others.
R6	InfoNCE · late	Virchow2 niche + novae only (64-d)	Ablation: drops gpath2vec from the ST input. Tests whether the pathway anchor on the ST side matters.
B1	CCA (closed-form, linear)	Virchow2 niche + (novae + gpath2vec)	Classical canonical correlation analysis. Named baseline in the proposal.
B2	Procrustes (closed-form, orthogonal rotation)	Virchow2 niche + (novae + gpath2vec)	Best orthogonal rotation aligning the two modalities' PCA spaces.
B3	Unaligned PCA (no cross-modal training)	Virchow2 niche + (novae + gpath2vec)	Independent per-modality PCAs with L2-norm. Proposal's "unaligned" sanity floor.
B4	Random-init MLP (zero training)	Virchow2 niche + (novae + gpath2vec)	R1's architecture with Xavier init and no training. Isolates contrastive training's contribution from the architecture itself - the chance anchor.

winner architecture - R4 cross-attention end-to-end

H&E side (Virchow2 foundation model, frozen, 632M params, pretrained on 3M slides at MSK Paige). For each niche (center spot + 6 spatial neighbors = 7 spots), Virchow2 produces a 1280-d CLS embedding per spot. R4 keeps these 7 spot embeddings as separate tile tokens, stacked into a (7, 1280) matrix - it does not mean-pool them like the other runs.

ST side (Novae GNN + gpath2vec). Novae (frozen, image-based-ST-pretrained 32M-param GAT, MICS-Lab) produces a 64-d latent per spot, mean-pooled across the 7 spots and z-scored per subarray. gpath2vec (built in-project - see "How gpath2vec was built" below) produces a 512-d niche-level pathway-context vector. The two are concatenated into a 576-d ST input vector.

Cross-attention bridge. The ST vector is projected to a 512-d query Q. The 7 Virchow2 tile tokens are linearly projected to keys K (7, 512) and values V (7, 512). Standard scaled-dot-product attention computes weights w = softmax(Q · Kᵀ / √512), then produces the fused H&E vector as Σ wᵢ Vᵢ - a weighted average of the 7 tiles where the weights are learned from the ST signal. The weighted H&E vector and the projected ST vector are L2-normalized and aligned via symmetric InfoNCE (temperature τ=0.07). ~4M trainable params total.

Why this wins H1 + H3. Mean-pooling the 7 tiles (every other run) discards within-niche morphological detail. R4's cross-attention lets the ST embedding select which tile spots carry the matched signal - preserving the local heterogeneity that mean-pooling erases. That preserved structure is what enables H1's 0.859 AUC (cleanest matched-pair ranking) and H3's cross-patient pathway transfer (4/4 testable pathways significant, top sub-pathway z=6.92 for BTLA at the DAG level). R4 also produces a tighter, lower-effective-rank shared space - which is why it loses H2-A KMeans (no globular clusters to fit) while still encoding mc biology at linear-probe parity (LP accuracy 0.240 vs raw H&E 0.270).

How gpath2vec was built (v3 build)

gpath2vec is what we wrote in-project to encode "which Reactome biological processes are active in each niche" as a 512-d vector. It is what the ST side aligns against on the alignment side - so understanding how it was built is load-bearing for interpreting H3 cross-patient pathway transfer results.

v3 build: fisher_madmean_low_dim512_e5_s1234 - six steps

Per-niche variable-gene selection. For each of 286,250 niches, identify the top-variable genes by MAD/mean ratio (Median Absolute Deviation divided by mean expression - captures genes that vary across spots while controlling for absolute magnitude).
TF restriction. Restrict the variable-gene universe to the 1,658 PathwayCommons-curated transcription factors. This makes gpath2vec a transcriptional-regulatory view of pathway activity, not a generic gene-expression view.
Fisher exact test per (niche, pathway). Test whether each niche's TF-restricted variable set is significantly enriched in each Reactome pathway's gene set. 189.7M niche × pathway tests across 680 Reactome pathways at the "low" hierarchy level.
BH-FDR thresholding. Benjamini-Hochberg FDR < 0.05 -> 1,954,421 significant niche-pathway pairs (~1.03% of all tests pass).
Heterogeneous graph construction. Two edge types: (a) niche↔pathway edges for each significant Fisher pair, weight = 1 - fdr_bh (so an edge with fdr=0.05 gets weight 0.95, an edge with fdr=0.001 gets weight 0.999); (b) pathway↔pathway edges from the Reactome hierarchy (parent-child relations), unweighted. The hierarchy backbone lets random walks travel between related pathways via the ontology rather than only through shared niches.
metapath2vec random walks. Train metapath2vec (word2vec variant for heterogeneous graphs) for 5 epochs, embedding dimension 512, seed 1234. Output: 512-d vector per niche (215,388 niches with at least one Fisher-significant pathway edge) and 512-d vector per pathway (1,157 R-HSA pathway nodes) - both in the same vector space, so cosine similarity is meaningful between any niche and any pathway.

What gpath2vec captures (and what it misses). Niches that share Fisher-significant TF-mediated pathway memberships land close together in the 512-d embedding. The TF restriction is a deliberate choice: gpath2vec catches pathways where TF activity drives enrichment (immune checkpoint signaling, IFN response, NF-κB cascades, hormone receptor signaling, MYC/TGF-β downstream transcription) and misses pathways driven by non-TF gene families (collagen secretion, MMP enzymes, immunoglobulin heavy/light chains in plasma cells). This is why H2-B.2 (TLS gene-signature CCA) shows late-fusion contrastive degradation - the TLS signal is partly carried by immunoglobulin gene expression, which is downstream of TF activity but not itself TF-encoded.

Metrics in plain English

Every test in the report below uses one of these metrics. Definitions in plain language plus what a "good" value looks like.

H1 - retrieval and structural agreement (5 metrics)

metric	what it measures (plain English)	how we compute it here	what a good value looks like
R@K Recall at K	For each H&E niche query, rank all 35,594 ST niches by cosine similarity. R@K = fraction of queries where the true matched ST niche appears in the top K. R@1 = exact-match retrieval; R@10 = match shows up in the top 10.	35,594 query × 35,594 candidate ranking via cosine on L2-normalized z_he and z_st	at this scale, R@1 ~10⁻⁴ for all methods - exact niche retrieval is near-floor. R@K useful as a relative ranking, not absolute
MRR Mean Reciprocal Rank	Average of 1/rank across all queries. If the true match is always at position 1, MRR = 1.0. At position 10, MRR = 0.1. Captures how high on average the true match ranks.	same ranking as R@K, take 1/rank per query and average	like R@K, magnitudes are small at 35k scale; useful as cross-method comparison
AUC area under ROC	Pick a random matched (H&E, ST) niche pair and a random mismatched pair. AUC = probability the matched pair has higher cosine similarity than the mismatched pair. 0.5 = chance ordering, 1.0 = perfect ordering.	sklearn `roc_auc_score` on labels (matched=1, sampled mismatched=0) × cosines. 35,594 positives, 355,940 negatives	0.5 = chance, >0.7 useful, >0.85 strong. R4 hits 0.859 [0.857, 0.862]
alignment gap	Mean cosine of matched (H&E, ST) niche pairs minus mean cosine of mismatched pairs. Positive = matched pairs are systematically closer than mismatched pairs.	mean(diag(z_he · z_stᵀ)) - mean(off-diag(z_he · z_stᵀ))	positive better; R4 +0.233 vs B3 -0.053 (matched anti-correlated)
CKA Centered Kernel Alignment	Measures whether the pairwise-similarity structure of the H&E manifold matches the pairwise-similarity structure of the ST manifold. If both modalities place the same niches close together, CKA is high.	linear-kernel CKA on centered z_he and z_st (Frobenius-norm form: ‖z_heᵀ z_st‖² / (‖z_heᵀ z_he‖ · ‖z_stᵀ z_st‖))	0 = unrelated structure, 1 = identical structure. Raw concat ~0.12, R4 post-alignment 0.631

H2 - biology preservation (5 metrics)

metric	what it measures (plain English)	how we compute it here	what a good value looks like
ARI Adjusted Rand Index	Run KMeans (k=14, seed=42) on z_he. This assigns each of the 35,594 test niches to one of 14 predicted clusters. Then compare those KMeans assignments against Wang's mc_megacluster ground-truth labels (also 14 classes) niche-by-niche. ARI scores how well the two partitions match - corrected for the agreement you would expect from random label assignment. ARI=1 means the KMeans clusters and mc labels are identical (up to permutation), ARI=0 means random, ARI<0 means worse than random.	KMeans(k=14, n_init=10, seed=42) on z_he -> `adjusted_rand_score(mc_megacluster, kmeans_labels)` on the test set	raw ST baseline 0.046 - 5x lift to 0.246 is the H2-A signal. Patient-level bootstrap CI > 0 confirms it's not noise
linear probe accuracy	Train a simple linear classifier on the embedding to predict a label (mc_megacluster). Output = held-out classification accuracy. Tests whether biology is linearly recoverable even when it doesn't form globular clusters.	sklearn `LogisticRegression` (lbfgs, C=1, 14-class) on z_he with patient-stratified 11/3 train/test split, StandardScaler on inputs	R4 0.240 within range of R1 0.282, raw H&E 0.270. Linear probe rescues R4 where KMeans fails
matched-pair cosine + Welch z	For each pathologist-annotated compartment (TLS, Tumor, Necrosis, hi-TIL, lo-TIL), compute the mean cosine of matched (H&E, ST) pairs within that compartment. Welch z-test compares two compartments: is the cosine higher in one than the other?	unpooled-variance z-test on per-pair cosine aggregates between distinct and ambiguous compartments	z > +2 = passes (distinct > ambiguous as predicted). z < -2 = inverts. TLS contrasts hit +2.2 to +5.9 in R6/R2/R1
univariate CCA z_TLS	Find the 1-dimensional direction in z_he that maximally correlates with the per-niche TLS gene-signature score on held-out patients. Then z-score this correlation against a 500-permutation null.	least-squares canonical direction on z_he vs Wang's `tls_scores.tsv` (continuous), cross-patient 11/3, 500-perm null, BH-FDR. z_TLS = (observed - null_mean) / null_std	z > 1.96 = sig at α=0.05. B4 random-init z=25.5 means raw Virchow2 features already carry the signal; late-fusion z < 1.6 = degraded
bio z / patient z ratio	Two separate permutation-based z-scores: how strongly does the embedding separate niches by biology (Bareche TIME label) vs by patient identity (patient_id)? The ratio tells you whether biology dominates or patient confound dominates.	matched-null permutation z on same-vs-different-label cosine deltas (10k perms) for TIME bio signature and for patient_id; ratio = bio_z / patient_z	raw H&E floor = 0.137. R6 0.547 above floor (biology wins). B1 0.071 below floor (patient identity wins, anti-helpful)

H3 - cross-patient pathway transfer (1 metric family + 2 methodological tools)

metric	what it measures (plain English)	how we compute it here	what a good value looks like
z_A per pathway univariate CCA, cross-patient	Same procedure as z_TLS, but one z-score per Reactome pathway. Asks: does the 1-d direction in z_he that correlates with this pathway's gene signature on training patients also correlate on patients the model never saw?	least-squares canonical direction on z_he vs gpath2vec-niche-cosine-to-pathway (or AUCell score - both arms tested), cross-patient 11/3 split, 500-perm null, BH-FDR over 4 testable pathways (TGF-β excluded, set_size=2)	R4 4/4 sig (z 11-25). Late-fusion 0-2/4. Cross-patient is where alignment quality reveals itself - within-cohort is universally positive and doesn't discriminate
permutation null how we get "z" values	Randomly shuffle the pathway labels 500 times (or 10k for bio/patient), recompute the canonical correlation each time. This produces a distribution of "by chance" values. The z-score = (observed value - null mean) / null std.	500 perms (H3 cross-patient) or 10k perms (bio/patient ratio). Permute labels, not embeddings - preserves within-modality structure	\|z\| > 1.96 ≈ p < 0.05. Magnitudes report effect strength relative to chance
BH-FDR Benjamini-Hochberg	When you test many things (ex. 4 pathways, 6 contrasts, 395 DAG nodes), some will appear significant by chance. BH-FDR is the standard multiple-testing correction that controls the fraction of false positives across the family of tests at a chosen rate (ex. 5%).	applied per test family per hypothesis: 4 H3 parent pathways, 6 H2-B.1 compartment contrasts, 10 H2-B.2 runs, 395 H3 DAG nodes	BH-FDR < 0.05 is the standard cutoff for reporting "significant" findings

Why so many metrics. Each metric tests a different operationalization of biology. AUC tests whether matched pairs rank above mismatched - the retrieval question. ARI tests whether the embedding produces cluster structure aligned with a label - the geometry question. Linear probe tests whether the label is linearly recoverable from the embedding - the encoding question, independent of geometry. Cosine contrasts and z_TLS test whether specific biological signals are preserved. z_A cross-patient tests whether what works on training patients transfers to new patients - the generalization question. Different alignment strategies optimize different operationalizations, which is what produces the trade-offs in the routing table.

H1 · cross-modal retrieval and structural alignment

Given an H&E niche embedding, rank all 35,594 ST niche embeddings by cosine similarity and recover the true matched ST niche. AUC = probability a matched pair ranks above a mismatched pair (0.5 = chance, 1.0 = perfect ordering). CKA = pairwise similarity structure agreement between the two modalities' manifolds.

0.859

R4 AUC (v3)

[0.857, 0.862]

0.631

R4 CKA (v3)

[0.626, 0.636]

+0.233

R4 alignment gap

matched - mismatched cosine

0.496

B4 AUC (chance)

[0.493, 0.499]

0.440

B3 AUC (below chance)

unaligned PCA

AUC by run (v3) - with Hanley-McNeil 95% CIs

R4* cross-attn

0.859[0.857, 0.862] · CKA 0.631 · wins H1 + H3

R1 InfoNCE late

0.761[0.758, 0.764] · CKA 0.312 · 3-way tie H2-A clustering

R5 AnInfoNCE

0.761[0.758, 0.764] · CKA 0.314 · top H2-A ARI point estimate

R6 novae-only

0.737[0.734, 0.740] · highest H2-C bio/patient ratio

R2 SupCon

0.725[0.722, 0.728] · CKA 0.101

B2 Procrustes

0.706[0.703, 0.710] · classical ceiling

R3 Barlow

0.674[0.671, 0.677] · only contrastive below classical

B1 CCA

0.544[0.541, 0.547] · anti-helpful on bio/patient

B4 random init

0.496[0.493, 0.499] · chance anchor

B3 unaligned PCA

0.440[0.437, 0.443] · falsifier floor

Absolute Recall@1 ~0.0002 across all runs - 1 in 5,000 queries finds its exact matched ST niche in the top position at 35,594 candidates. AUC measures reliable ordering (matched pairs rank above mismatched pairs), not exact retrieval. Practical use would aggregate multiple niches into a larger tissue zone - that step is not yet implemented and is the natural next direction.

H1 verdict: supported. 5/6 contrastive runs beat every classical baseline with non-overlapping 95% CIs. R3 Barlow is the exception (CI gap 0.026 below B2). B4 at chance confirms training - not architecture - creates alignment. R4 at 0.859 exceeds the proposal's expected range of 0.65-0.75.

H2 · structural biology preservation

H2 visual evidence - same z_he UMAP per run under three colorings

How to read. Each row is one alignment run's z_he UMAP; the layout is the same across columns within a row, only the coloring changes.

Column 1 (patient_id): 14 distinct colors, one per test patient. R4 mixes patients; R1 forms patient blobs; B1 separates patients into tight isolated clusters - this is the H2-C bio/patient story at the visual level.
Column 2 (mc_megacluster): 14 niche-biology classes from Wang's NMF + K-means consensus. R5/R6/R1 (column 2 of R1 row here) form globular mc clusters - H2-A KMeans ARI ~0.24. R4's mc clusters exist but sit on a tight non-globular manifold - linear-probe still recovers them at parity (0.240).
Column 3 (TLS gene-signature score): continuous, viridis-colored. A coherent yellow region in the UMAP = niches with high TLS scores cluster together in the aligned space. The TLS signal is most spatially coherent in R4 and B1, fragmented in R1 - matching H2-B.2 CCA z-scores (R4 z=11.4, R1 z < 1.6).

Source: scripts/_scratch/h2_visual_evidence.py (UMAP cache from runs/tnbc-92_v3/eval/figures/umap_cache/, metadata from data/embeddings/niches_v3/).

Part A - niche clustering (corrected label: mc_megacluster)

Original evaluation question used 9 spatial archetypes (patient-level pseudobulk NMF, NMI=0.89 with patient_id - equivalent to asking which country a neighborhood is in). Corrected to Wang's 14-class mc_megacluster label (per-patient NMF on ST counts -> K-means consensus on factor centroids, varies at niche resolution within a patient).

TLS shows up twice in H2 because the cohort gives two distinct TLS signals - same FTU, two operationalizations. This is the "two of the four ground truths" from the proposal framing, made concrete.

Categorical "Lymphoid nodule" (used in Part B.1 below): Wang's pathologist hand-annotation of organized lymphoid aggregates visible on H&E - 15-category per-spot label from annotBySpot.RDS (Zenodo), "Lymphoid nodule" is the TLS class. Loaded into the niche-join's compartment column on the center spot only (discrete labels can't be averaged). Sparse: 344 spots / 238 niches across 23 of 92 patients (~0.36% of annotated spots). Gold-standard morphological TLS - if a pathologist saw an aggregate, it's here.
Continuous TLS gene-signature score (used in Part B.2, middle column above): We score every niche against the Cabrita et al. 2020 30-gene TLS panel (CCL19/21, CXCL13, CD79A/B, MS4A1, BCL6, PAX5, IGHM, etc.) on Wang's raw counts via extract_biological_signals.py::extract_tls_scores(), then mean-pool the 7-spot niche to a single value (tls column in niches_v3). Dense: every niche with ST counts gets a score, NaN only if no neighbor has data.

These aren't the same thing - a niche can have high tls score (TLS-related genes elevated) without being labeled Lymphoid nodule (no organized aggregate visible). That mismatch is why both tests live in H2: same FTU, the categorical test routes to late-fusion contrastive (which optimizes within-compartment cosine), and the continuous test routes to morphology-preserving methods (B4 random-init z=25.5 - TLS gene expression is in Virchow2 features without any alignment training).

KMeans ARI on mc_megacluster (v3)

R5 AnInfoNCE

0.246

R6 novae-only

0.244

R1 InfoNCE

0.234

B1 CCA (disqualified)

0.211

R6 fused (z_joint)

0.205

raw H&E alone

0.164

R4 cross-attn

0.101

raw ST alone

0.046

Beats both modalities alone: 1.5x over raw H&E (0.164), 5.4x over raw ST (0.046). B1 CCA reaches 0.211 but is disqualified by Part C - bio/patient ratio 0.071, below the raw-H&E floor of 0.137, with patient z 113.0 against R6's 31.8. It scores by encoding patient identity, not biology. Part A: supported after correction
In plain terms: an H&E slide can be read for tissue state more accurately once the model has been taught to match it against gene expression - more accurately than either the slide or the expression data on its own.

The gain is geometric, not informational. ARI rises 50% over raw H&E (0.164 -> 0.246) while the linear probe moves 1.2 points (0.270 -> 0.282). Measured against the majority-class floor of 0.216 (mc2, probe-test fold) rather than uniform 1/14, every LP lift is small: raw H&E +5.4pt, R1 +6.6pt, R4 +2.4pt. Alignment reshaped the manifold so mc classes became globularly separable; it did not add decodable information.
In plain terms: the tissue signal was already present in the H&E features. Alignment did not uncover new biology - it rearranged the space so ordinary clustering could find what was already there.

Do not fuse. Concatenating the ST vector onto the H&E vector degrades the result: z_joint 0.205 < z_he 0.246. The mechanism is ST-supervision of the H&E tower during training, not combination at inference - only H&E enters at test time.
In plain terms: the benefit comes from training the image model against expression data, not from feeding both in together. At prediction time you only need the H&E slide.

Label caveat. NMI(mc_megacluster, patient_id) = 0.539, above the 0.5 confounder threshold this project applies elsewhere. Far below archetype's 0.89, but non-trivial: every ARI in this row partly rewards patient identity. Silhouette is non-discriminating (-0.072 to +0.008 across every run and baseline); ARI carries this row alone. Source: runs/tnbc-92_v3/eval/H2/mc_coherence.parquet.
In plain terms: the tissue-state labels partly track which patient a sample came from, so some of the apparent accuracy reflects recognising the patient rather than the biology.

TLS gene-signature CCA on z_he (v3)

B4 random-init

z=25.5

B1 CCA

z=16.6

R4 cross-attn

z=11.4

B2/B3 classical

z=11.3

R5 marginal

z=2.4

R1/R2/R3/R6

<1.6

Cross-patient sub-split, 500 perms, BH-FDR < 0.05. TLS is already in Virchow2 features - contrastive late-fusion training degrades this. Part B.2: supported for morphology-preserving methods

Bio/patient ratio (v3, delta-method 95% CI)

R6 novae-only

0.547

R5 AnInfoNCE

0.497

R1 InfoNCE

0.461

R4 cross-attn

0.376

raw H&E floor

0.137

B1 CCA

0.071

All 6 contrastive runs above raw H&E floor. B1 amplifies patient identity ~3x - below floor. R4 has lowest patient z (21.5), enabling H3 cross-patient transfer. Part C: supported; B1 fails

Part B.1 compartment cosine contrasts: TLS vs TIL-stroma passes in R6/R2/R1/R5/B2 (Welch z = +2.2 to +5.9). Tumor/Necrosis invert (z down to -15) - annotation granularity issue, not biology absence. R3 Barlow uniquely passes necrosis contrasts. Part B.1: TLS only

H2 · annotation taxonomy - what requires what

The H2 findings surface a fundamental question: what kind of annotation is needed to evaluate the alignment, and where does that annotation come from? The compartment-cosine inversion (Tumor/Necrosis failing, TLS passing) is not a failure of alignment - it is a readout of annotation vocabulary quality. This taxonomy maps each annotation type to its source, what it can test, and what the results showed.

annotation type	source	H2 result	what it tells us	what is needed to improve
mc_megacluster 14-class cross-patient consensus on per-patient NMF	systematically computed Wang's per-patient NMF on ST counts (variable K per patient) -> K-means (K=14) on the resulting factor centroids to derive cross-patient consensus classes. No FM, no expert annotation. Fully reproducible.	passes 5x ARI lift. R5/R6/R1 win clustering. R4 at linear probe parity.	The best-performing H2 label. Systematically derived from the data itself. Because it varies at niche resolution within a patient, it directly tests whether the aligned space encodes niche-level biology rather than patient identity. No human in the loop.	Higher k (more classes) or a different factorization basis could reveal finer tissue states. Biological identity per mc class derived from Wang's km14 consensus matrix top-differential genes (ex. mc12 = B-cell follicle / TLS center with IGKC/IGLC/HLA-DR/TRAC; mc05 = plasma cell zone; mc01 = CAF/desmoplastic stroma; full table below; derivation in `scripts/extract_mc14_differential_markers.R`). Validates the label biologically without per-cohort expert annotation.
TLS gene signature score continuous per-niche score from ST counts	systematically computed Wang's published per-spot TLS gene-signature score (`tls_scores.tsv`), aggregated per niche. Liu/Cabrita-style TLS signature scored via gene-signature aggregation on ST counts. No FM, no expert.	passes B4 z=25.5, R4 z=11.4 cross-patient. Already in Virchow2 features.	TLS has both a gene-expression signature (captured by gpath2vec and AUCell) and a morphological signature (captured by Virchow2). The B4 finding shows the morphological signature is strong enough to predict the gene-expression score cross-patient without any alignment training. The two modalities reinforce each other on TLS - this is the clearest example of cross-modal readability the project produced.	This works because TLS has a conserved morphology. For signals without a conserved morphological appearance, this approach would not generalize - you need the ST signal to decode the biology.
TLS categorical label "Lymphoid nodule" in Wang's 17-class annotation	expert pathologist annotation Wang et al. pathologist-annotated per spot. Not reproducible without expert re-annotation.	passes R6/R2/R1/R5/B2 pass compartment contrasts (z=+2.2 to +5.9).	TLS passes because it is a structurally coherent, well-curated annotation - every niche labeled TLS contains the same basic biological unit (organized immune aggregate). This is the gold standard for what a well-defined annotation enables. The within-compartment cosine is high because the annotation is pure.	Expert pathologist annotation at this granularity is expensive, slow, and not scalable. The ideal path is to train a TLS detector on Virchow2 features (given B4 z=25.5, this is tractable) that can produce TLS labels systematically at scale - no expert required per slide.
Tumor Wang's pathologist label	expert pathologist annotation n=4,384 niches in test set	inverts z down to -15. Fails in 4-7 runs.	The Tumor annotation pools all malignant cells regardless of subtype, growth pattern, or immune context. A highly proliferative tumor nest and an invasive tumor front with dense immune infiltrate both get the label "Tumor." Within-compartment cosine is low not because alignment fails but because the annotation class is internally heterogeneous - there is no single biology to align to.	Two paths: (a) sub-type the Tumor label using a better FM - a pathology FM trained on tumor subtype classification could subdivide into "tumor nest," "invasive front," "tumor-immune interface" categories that are each morphologically coherent. (b) Use molecular sub-typing from ST directly (TNBC subtypes BL/M/MSL/LAR/IM per spot) as the evaluation target instead of the coarse pathologist label.
Necrosis Wang's pathologist label	expert pathologist annotation n=1,178 niches in test set	mixed Inverts in most runs. R3 Barlow uniquely passes (z=+5.8, +9.1).	Necrosis has a distinctive morphological appearance (ghost cells, karyorrhexis, nuclear debris) but its transcriptomic signal is weak by definition - dead and dying cells produce degraded RNA. The cosine inversion may reflect that Novae captures very little signal from necrotic spots, so matched-pair cosine is low not because the spaces are misaligned but because one side of the pair has no information. R3 Barlow's unique success may reflect its redundancy-reduction objective being less sensitive to low-signal inputs.	A necrosis-specific evaluation would need a different ST signal - RNA integrity scores or apoptosis pathway signatures rather than expression-based encodings. Alternatively, necrosis is a case where H&E is the more informative modality and ST adds little - the alignment should not be expected to help here.
Spatial archetypes Wang's 9-class patient-level NMF	systematically computed NMF on pseudobulk per patient. No expert needed but patient-level.	misleading NMI=0.89 with patient_id. ARI on archetypes = patient classification.	Systematically computed but at the wrong resolution for a niche-level evaluation. Every niche in a patient inherits one archetype - so "archetype ARI" mechanically measures patient classification. The label is useful for patient-level risk stratification (SA4 good prognosis, SA8 poor prognosis) but cannot evaluate whether the aligned space captures niche-level biology.	Already resolved: use mc_megacluster instead. Archetypes remain useful as a patient-level downstream evaluation target - if the aligned space can predict archetype from niche embeddings without leaking patient identity, that would be a meaningful result. This is a future experiment.
High/low TIL stroma Wang's pathologist label	expert pathologist annotation n=105-354 niches (small, well-curated)	reference class Used as the "ambiguous stroma" comparison target. Small n but high within-compartment homogeneity.	High and low TIL stroma are the comparison baseline for the TLS contrast - they are the "morphologically ambiguous" compartments the proposal predicted would show lower matched-pair cosine than TLS. TLS vs TIL-stroma passes as predicted. The small, well-curated nature of the TIL annotations means high within-compartment purity.	TIL density scoring is already semi-automated in clinical practice (TILs scoring guidelines). A computational TIL scorer on Virchow2 features could produce continuous TIL scores at scale, enabling this contrast without per-slide expert annotation.

The 14 mc_megacluster classes - biological identity (click to expand)

How these were derived. Wang's km14.RDS holds k-means centroids over 418 per-patient NMF factors in 3,888-variable-gene space. Ranking a class by its absolute centroid loading returns MT-CO2, GAPDH, ACTB and ribosomal proteins for nearly every class - 54 unique genes across all 14 lists - because those genes are high everywhere and therefore identify nothing. The ranking below is differential: a gene's loading in class X minus its mean loading across the other 13. Reproduce with Rscript scripts/extract_mc14_differential_markers.R. Identities are read off the differential markers; they are an interpretation of the marker sets, not a Wang-supplied annotation.

class	biological identity	top differential markers	reading
mc01 26 prototypes	CAF / desmoplastic stroma	AEBP1, BGN, MYL9, TAGLN, COL6A2, COL1A1, COL1A2, LGALS1, MMP14, POSTN	collagens + smooth-muscle actin + periostin - classic cancer-associated fibroblasts
mc02 45 prototypes	inflamed / IFN-stimulated epithelium	GABRP, TAP1, S100A8, IFI27, HLA-DRA	antigen presentation (TAP1, HLA-DRA) with interferon-response markers
mc03 29 prototypes	transcriptionally active / lncRNA-high	MALAT1, NEAT1, MT-RNR2, PTPN14, SOX4	nuclear lncRNA dominant - proliferating or developmental state
mc04 61 prototypes	basal-like TNBC epithelium	KRT81, TM4SF1, GABRP, CRYAB, S100P, S100A6, MFGE8, KRT7, KRT17	basal keratins + S100 family; the largest class
mc05 39 prototypes	plasma cell zone	IGKC, IGHA1, IGHM, IGHG3, IGHG1, IGLC2, MZB1, CD74, C1QA, CD52	immunoglobulins + MZB1 (marginal-zone B) + complement
mc06 19 prototypes	apocrine / secretory glandular tumour	MUCL1, APOD, AZGP1, PIP, FASN, ALOX15B, SPDEF, MUC1, TFF3	AZGP1 + PIP = apocrine; FASN = lipogenic. LAR-like
mc07 8 prototypes	basal myoepithelial, keratin-high	LOXL2, KRT16, KRT14, KRT5, MGP, IGFBP2, KLK5, KRT19	KRT14 / KRT5 basal programme; LOXL2 collagen cross-linking. BL1-like
mc08 8 prototypes	proliferating stem-like tumour	CD24, GLO1, H2AFZ, STMN1, SUMO2, HMGN1, LAPTM4B	CD24 + STMN1 + H2AFZ - proliferation and stemness
mc09 39 prototypes	plasma cells + IFN-activated inflammation	IGKC, IGHM, CD74, IGHA1, IFI6, BST2, IFI27, C1QA, WARS, S100A8	immunoglobulin-high like mc05, but with an interferon module on top
mc10 26 prototypes	tumour-adjacent fibroblastic stroma, immune-mixed	MALAT1, COL3A1, COL1A1, COL1A2, DCN, CCDC80, IGFBP4, CD74, LUM	collagens + decorin + lumican with CD74 - stroma at an immune boundary
mc11 26 prototypes	mitochondrial-high / metabolically active	MT-RNR2, MT-RNR1, MALAT1, MT-ND2, MT-ND5, MT-ND1, MT-CYB, MT-ND4	almost entirely mitochondrial. hypoxic, or a low-RNA-quality QC flag
mc12 52 prototypes	B-cell follicle / germinal centre canonical TLS	IGKC, IGLC2, IGHM, IGLC3, IGHG1, IGHG3, IGHA1, CD74, HLA-DRA, C3, CD52, HLA-DRB1, TRAC	immunoglobulins + class II HLA + TRAC - B and T compartments together. 65% of all Lymphoid-nodule niches
mc13 9 prototypes	secretory tumour + follicular dendritic cells	LTF, SAA1, MSLN, FDCSP, LGALS3BP, BEST1, CLDN4, CLDN3	FDCSP = follicular dendritic cell secreted protein - TLS support stroma; MSLN = mesothelin
mc14 31 prototypes	ribosomal-high / translationally hyperactive	RPS24, RPL34, RPL7, RPS3A, KRT15, RPLP0, GAS5	ribosomal protein programme on a basal epithelial background

Why three classes are immunoglobulin-high. mc05, mc09 and mc12 all load immunoglobulins, and they are not redundant: mc12 adds class II HLA and TRAC (organised follicle, B and T together), mc05 adds MZB1 and complement (plasma-cell zone, antibody secretion), mc09 adds an interferon module (IFI6, BST2, IFI27). The TLS sub-type analysis below splits along exactly this axis - which is why the three sub-types cross-cut mc rather than reproducing it.

The general principle from H2. An annotation class works as an H2 evaluation target when it is: (1) defined at niche resolution (not patient level), (2) internally homogeneous (one biological unit, not a heterogeneous pool), and (3) has signal in both modalities (not dead cells with no RNA). When those conditions hold, the alignment is testable. When they don't, the annotation confound is the finding - it tells you where better annotations or better foundation models are needed. The evaluation surface reveals annotation quality as much as alignment quality.

TLS sub-types - molecular heterogeneity within the "Lymphoid nodule" annotation

Wang annotated TLS as a single category ("Lymphoid nodule"), but mature germinal-center follicles, plasma-cell-rich zones, and immature lymphoid aggregates are distinct biology. Clustering the 238 TLS-annotated niches in gpath2vec space (ward linkage, K=3, all 23 patients with TLS regions represented) reveals three cross-patient sub-types:

sub-type	niches / patients	dominant mc class	top differential pathways (vs other TLS clusters)
C1 - transcription / chromatin-modifying TLS germinal-center-like	138 / 18	mc12 (B-cell follicle) 78%	UP: mRNA processing, chromatin modifying enzymes, SUMOylation of DNA repair / replication / chromatin proteins, ISG15 antiviral. Consistent with active class-switch recombination / somatic hypermutation in germinal-center B-cells.
C2 - neutrophil / innate-immune TLS inflamed boundary follicle	23 / 12	mc12 + mc5 (plasma cell zone) + mc10 split	UP: Neutrophil degranulation (+1.66, strongest signal across all sub-types), JAK-STAT IL-12, mitochondrial cristae, XBP1(S) chaperone activation. Consistent with neutrophil-recruiting plasma-cell-active TLS at inflamed tumor-stroma boundaries.
C3 - translation / effector TLS antibody-producing zone	77 / 19	mc12 + mc5 split	UP: Ribosome-associated quality control, Eukaryotic Translation (initiation/elongation/termination), SRP-cotranslational protein targeting, ROBO receptor signaling, EIF2AK4 amino-acid response. Consistent with metabolically demanding effector function - antibody secretion by plasma cells, cytokine production by activated T-cells.

What this surfaces. The 3 gpath2vec sub-types cross-cut mc_megacluster - mc12 niches appear in all 3 with different TF-driven regulatory programs running in them. So gpath2vec discovers an axis of TLS variation that NMF-based mc clustering does not capture: functional metabolic state of the TLS (transcriptionally engaged vs translationally active vs neutrophil-recruited), independent of which broad NMF cell-state the niches are assigned to. All three sub-types recur in 12-19 of the 23 TLS-carrying patients - they are not patient-specific artifacts. Source: `scripts/_scratch/h3_tls_subtypes.py`; figures at `runs/tnbc-92_v3/eval/figures/h3_tls_subtypes.png` and `_summary.png`.

What can be systematically computed today

- mc_megacluster (per-patient NMF on ST counts + K-means consensus, any cohort with ST)
- TLS gene-signature score (Liu/Cabrita-style signature scored per niche on ST counts)
- gpath2vec pathway embeddings (TF-restricted Fisher EA on Reactome + metapath2vec)
- AUCell pathway scores (rank-based gene-set enrichment per niche)
- Spatial archetypes (NMF on patient-level pseudobulk - useful for patient stratification, not niche-level evaluation)

What currently requires expert annotation or better FMs

- Tumor sub-type labels (invasive front vs nest vs interface) -> needs pathology FM trained on subtype classification, or expert re-annotation with finer vocabulary
- TLS categorical label at scale -> tractable via Virchow2 detector (B4 z=25.5 shows the signal is there)
- TIL density scoring -> semi-automatable with current FM features
- Necrosis reliable scoring -> needs RNA integrity or apoptosis-specific signal, not expression-based ST
- Per-spot TNBC molecular subtype (BL/M/MSL/LAR/IM) -> Bareche subtypes are patient-level; per-spot subtyping is a future experiment

H3 · pathway interpretability - cross-patient transfer

Within-cohort CCA universally positive for all 10 runs at 512d / 35,594 niches - non-discriminating. Cross-patient sub-split (11 train / 3 held-out, 500 permutations, BH-FDR < 0.05, 4 testable pathways - TGF-beta excluded, only 2 Reactome DAG nodes) is the real test.

Cross-patient z_A by run and pathway (v3)

runImmuneECMCell CyclePCDsig/4

R4*25.416.016.511.24/4

B1/B2/B37.5-15.26.7-17.16.7-10.43-7.54/4

R5-1.93.02.13.02/4

R1 / R3<3<2<2<11/4

R2 / R6<1<1<1<10/4

H3 DAG decomposition - 395 sub-pathway nodes (v1) - per-run sig rate, z_he cross-patient BH-FDR<0.05

R4*

80%wins every parent (ECM 93%, Immune 83%, TGF-beta 81%, PCD 75%, Cell Cycle 69%)

B2 Procrustes

68%competitive at fine resolution

B3 unaligned PCA

68%rotation-equivalent to B2 on this test

B1 CCA

63%classical, lower than B2/B3

R6 novae-only

3.8%wins H2, near floor on DAG

R1 InfoNCE

3.5%wins H2 clustering, near floor on DAG

R3 Barlow

1.8%

R2 SupCon

1.5%

R4 top sub-pathway hits: BTLA co-inhibition z=6.92 (checkpoint receptor, ICB resistance mechanism), IL receptor SHC signaling z=6.80 (cytokine signaling), Integrin cell surface interactions z=6.21 (ECM adhesion), ECM proteoglycans z=5.73, IFN-alpha/beta regulation z=5.71, Collagen biosynthesis/degradation z=5.22-5.28. These span immune checkpoint biology and stromal remodeling - distinct programs, not a single axis, at sub-pathway resolution.

H3 verdict: supported for R4 and classical baselines. R4 4/4 pathways cross-patient (z=11-25). Late-fusion runs 0-2/4. At sub-pathway DAG resolution, R4 encodes distinct immune checkpoint and stromal programs - the 5-parent aliasing (all pathways to one axis) is a coarse-aggregation artifact from overlapping Reactome gene sets, not an encoding property.

Task-conditional routing table (v3)

No single method wins every objective. Different biological questions require different alignment strategies. This routing table is what the omicstra MCP agent encodes as task-conditional dispatch. As the evaluation runs on additional cohorts, the routing pattern accumulates. TNBC-92 v3 is the second row.

if the question is	route to	evidence (v3 unless noted)
Find the ST niche matching this H&E niche	R4	H1: AUC 0.859 [0.857, 0.862], CKA 0.631, alignment gap +0.233
Group niches by TME tissue state on a tissue section	no ranking	H2-A does not rank the aligned methods on this cohort - retracted from the earlier "3-way tie, R6 parsimony winner". The lift stands (5.4x over raw ST, 1.5x over raw H&E) but the 200x patient-bootstrap intervals overlap for 8 of the 9 adjudicated methods - R5 0.246 [0.213, 0.424] through B4 0.189 [0.134, 0.443]; only R4 (0.101 [0.094, 0.154]) falls outside, as the lowest. With NMI(mc_megacluster, patient_id) = 0.539 and a linear probe moving 1.2 points against a 0.216 majority floor, the metric is not strong enough to order these methods. Use any of them for this task and say why; do not cite a winner.
Amplify biology, suppress patient identity in cohort analysis	R6 / R5	H2-C: R6 ratio 0.547 [0.476, 0.617]. R4 if leakage minimization is the priority (patient z 21.5, lowest in grid).
Transfer pathway signal to new, unseen patients	R4	H3: 4/4 pathways cross-patient (z=11-25). DAG: 80% of 395 sub-pathways significant, wins every Reactome parent.
Distinguish which pathway drives a specific niche	B2	H3 DAG: B2 68% at fine resolution. Off-diagonal cosine 0.38 vs R4 0.75 - per-pathway axes preserved.
Decode TLS gene signature from H&E morphology (no ST needed)	B4 (random-init Virchow2 projection)	H2-B.2: B4 z=25.5 cross-patient. The TLS signal is in Virchow2's feature space - any non-distorting projection preserves it. Contrastive late-fusion training degrades the signal.
Avoid for any cohort-level biology interpretation	B1 (CCA)	H2-C: ratio 0.071 [0.053, 0.088] below raw H&E floor (0.137). Amplifies patient identity 3.3x.

Consolidated proposal-alignment matrix (v3)

hypothesis	proposal alignment	evaluation question	observed result (v3)	interpretation	scientific conclusion	primary metrics	strong methods	baselines
H1 Cross-Modal Retrieval	Directly matched the proposal goal of aligning H&E and ST embeddings within a shared latent space using contrastive learning.	Can H&E embeddings retrieve matched ST niches more accurately than unaligned or classical methods?	R4 AUC 0.859 [0.857, 0.862]. R1=R5 0.761 [0.758, 0.764]. R6 0.737 [0.734, 0.740]. R2 0.725 [0.722, 0.728]. R3 0.674 [0.671, 0.677]. Classical: B2 Procrustes 0.706 [0.703, 0.710], B1 CCA 0.544 [0.541, 0.547], B3 Unaligned PCA 0.440 [0.437, 0.443] (sub-chance). B4 random-init 0.496 [0.493, 0.499] - CI explicitly straddles 0.500. 5 of 6 contrastive runs beat B2 with non-overlapping 95% CIs; R3 Barlow is the lone exception (CI gap 0.026 below B2).	Training (not architecture) creates alignment - B4's CI explicitly straddles 0.500. R4's cross-attention (the only non-late-fusion run) adds substantial gain over InfoNCE late-fusion: AUC 0.859 vs R1's 0.761.	Contrastive cross-attention captured stronger cross-modal alignment than classical or late-fusion baselines. R4 exceeded the proposal's 0.65-0.75 expected range.	R@K, MRR, AUC, alignment gap	R4 (best), R1, R5, R6, R2	B1, B3, B4; R3 loses to B2
H1 Structural Agreement	Matched the proposal objective of evaluating latent geometry agreement across modalities.	Do aligned embeddings exhibit stronger cross-modal structural agreement than baselines?	R4 CKA 0.631 [0.626, 0.636] - top of grid. R5 0.314 [0.306, 0.320]. R1 0.312 [0.305, 0.318]. R3 0.306 [0.299, 0.312]. R6 0.239 [0.233, 0.245]. R2 0.101 [0.098, 0.105]. Classical: B2/B3 0.125 (rotation-equivalent), B1 0.076, B4 0.120. Alignment gap: R4 +0.233, B3 -0.053 (anti-correlated), B4 -0.000. CKA before alignment (raw) = 0.119; R4 post-alignment +0.51 absolute lift (5x).	Matched-vs-mismatched pairwise structure separation increased ~5x in R4 vs raw. Late-fusion runs gain ~2-3x. Classical at near-raw levels. B3's negative gap = matched pairs anti-correlated in unaligned PCA space.	Contrastive cross-attention improved latent geometry substantially beyond classical correlation methods and late-fusion contrastive. CKA gain reflects genuine manifold agreement, not just summary-statistic improvement.	CKA (before/after), alignment gap, matched-vs-mismatched cosine separation	R4 (CKA 0.631, gap +0.233)	B1, B3, B4 (0.076-0.125, gap +0.007 to -0.053)
H2 Part A Structural Coherence corrected label	Directly evaluated the proposal hypothesis that aligned latent spaces preserve biologically meaningful TME organization.	Do the proposal's 9 spatial archetypes cluster more coherently in the aligned space? Corrected to mc_megacluster (per-patient NMF + K-means consensus, niche-resolution).	Archetype labels are patient-level pseudobulk - NMI=0.89 with patient_id - ARI on archetype mechanically resolves to patient classification (B1 wins by amplifying patient identity 3.3x). On corrected mc_megacluster label: R5 ARI 0.246 [0.213, 0.424], R6 0.244 [0.202, 0.411], R1 0.234 [0.203, 0.396] - 3-way tie within seed noise. R4 KMeans ARI 0.101 [0.094, 0.154] but linear probe accuracy 0.240, within range of R1 (0.282) and raw H&E (0.270). Raw ST alone: 0.046 - 5x lift over raw ST.	R4 encodes biology recoverable by linear probe but its tight manifold prevents KMeans from forming globular clusters - geometry issue, not biology absence. R1/R5/R6 produce visibly separable cluster regions for tissue-section region grouping.	R1 and R6 are right for grouping niches into biologically distinct regions on a tissue section. R4 trades cluster geometry for tighter manifold structure that enables cross-patient transfer (H3).	ARI, silhouette, LP accuracy, kNN purity, rank-matched KMeans	R5, R6, R1 (KMeans+LP); R4 (LP parity)	B1, B2, B3 lose 11+ ARI points on archetype->mc shift
H2 Part B.1 Compartment Contrasts	Matched the proposal goal of compartment-level analysis across TME regions.	Do morphologically distinct compartments (Tumor, TLS, Necrosis) show higher matched-pair cosine than ambiguous stroma (High-TIL, Low-TIL)?	TLS vs TIL-stroma passes in R6 (+5.6/+5.9 Welch z), R2 (+5.4/+5.4), B2 (+3.7/+2.8), R5 (+2.4/+3.2), R1 (+2.2/+4.1). Tumor and Necrosis contrasts invert decisively (z down to -15) in 4-7 runs. R3 Barlow uniquely passes Necrosis contrasts (z=+5.8, +9.1). R4 fails every contrast (0/6 pass).	TLS is the one well-curated, internally homogeneous annotation - passes as predicted. Tumor (n=4384) pools heterogeneous subtypes; Necrosis (n=1178) has weak ST signal by definition (degraded RNA in dead cells). Annotation granularity confound, not biology absence.	Compartment-level alignment quality depended on annotation granularity and morphological purity. H2-B.1 holds for TLS only. Tumor/Necrosis arms not testable as written - needs FM sub-typing or finer expert annotation vocabulary.	Matched-pair cosine, Welch z (one-sided, distinct > ambiguous)	R6, R2, R1, R5, B2 (TLS); R3 (Necrosis, unique)	B1 (1/6 pass), B3 (0/6 - sanity floor)
H2 Part B.2 TLS Gene-Signature CCA	TLS gene signatures scored spatially - one of the four ground-truth sources named in the proposal.	Does z_he linearly predict the continuous per-niche TLS gene-signature score on held-out patients via univariate CCA + permutation null?	v3, z_he, cross-patient sub-split (11 train / 3 held-out), 500 perms, BH-FDR<0.05: B4 random-init z_TLS = 25.5 (strongest). B1 CCA 16.6. R4 cross-attn 11.4. B2/B3 (rotation-equivalent classical) 11.3. R5 marginal 2.4. R1/R2/R3/R6 fail (z < 1.6). 6 of 10 runs significant.	TLS morphology lives in Virchow2's foundation-model feature space - B4 (random Xavier projection of Virchow2 features + L2-norm) preserves the cross-patient predictive signal end-to-end. Classical methods and R4 also preserve it. Late-fusion contrastive training distorts those features gaining cross-modal coupling - measurable trade-off, not free.	Supported for morphology-preserving methods. Late-fusion contrastive fails this single-modality decoding task - informative negative finding: contrastive alignment has a cost on signals already encoded by the foundation model. A systematic TLS detector on Virchow2 features is a direct next step.	Univariate CCA z_TLS on z_he, cross-patient 11/3, 500-perm null, BH-FDR	B4 (best), B1, R4, B2/B3, R5 (marginal)	R1, R2, R3, R6 fail - late-fusion family below significance
H2 Part C Bio vs Patient extension	Extended the proposal by explicitly evaluating biological signal preservation vs patient leakage.	Does the aligned space encode biology more strongly than patient identity? bio z / patient z ratio on Bareche TIME signatures.	v3, z_he, 10k perms. R6 ratio 0.547 [0.476, 0.617], R5 0.497, R1 0.461, R4 0.376, R2 0.372, R3 0.250. All 6 contrastive runs above raw H&E floor (0.137). B1 ratio 0.071 [0.053, 0.088] - below raw floor (patient z 113 vs raw 37 = 3.3x amplification). R4 has lowest patient z (21.5 - strongest patient suppression on z_he).	Linear probes recover biology even where clustering underestimates structure. R4's patient suppression is what enables H3 cross-patient transfer. B1 CCA amplifies patient identity ~3x and should never be used for cohort-level interpretation.	Shared embeddings preserved biological organization while suppressing patient-specific signal. B1 amplifies patient identity more than biology and should be avoided. The bio/patient ratio is a practical filter for whether an alignment is usable for cohort-level interpretation.	Patient-ID decoding, bio z / patient z ratio on TIME/MC_global/MC_tumor, 10k-perm null	R6, R5, R1 (ratio); R4 (suppression)	B1 (anti-helpful, ratio 0.071 below raw 0.137 floor)
H3 Pathway Interpretability cross-patient parents	Directly matched the proposal objective of evaluating pathway-associated biological signal within shared latent geometry.	Do the 5 Reactome pathways (TGF-beta, Immune, ECM, Cell Cycle, PCD) correlate with shared-latent directions cross-patient?	Within-cohort CCA: universally positive (10/10 runs, all cells BH-FDR<0.001) - non-discriminating. Cross-patient sub-split (z_he, 4 testable - TGF-beta set_size=2 excluded): R4 4/4 (z_A 11.2 / 16.0 / 16.5 / 25.4 for Immune / ECM / Cell Cycle / PCD). Classical B1/B2/B3 4/4 at lower z (3-17). R5 2/4. R1/R3 1/4. R2/R6 0/4. R4 5-pathway off-diagonal cosine 0.75 (aliased).	Fit-on-all CCA is supported universally but doesn't discriminate at 512d / 35,594 niches. Cross-patient sub-split is the discriminating test. R4 dominant. Late-fusion runs encode pathway-correlated directions that are patient-specific - they don't transfer. R4's aliasing (off-diag 0.75) IS the mechanism of cross-patient generalization: one robust biology axis is harder to overfit than five distinct ones.	Shared latent geometry preserved transferable pathway signal. R4's tight manifold generalizes across patients; R1's wider manifold encodes patient-specific gradients that don't transfer. Routing: R4 for cross-patient magnitude; B2 for per-pathway distinguishability.	Univariate CCA per pathway, perm null (500), BH-FDR over 4 testable, cross-patient 11/3 split	R4 (4/4); B1/B2/B3 (4/4 at lower z)	R1/R3 (1/4), R5 (2/4), R2/R6 (0/4) - late-fusion fails cross-patient
H3 Cross-Patient Transfer (DAG) v1 - v3 retrain deferred	Matched the proposal goal of evaluating generalization across held-out patients at sub-pathway resolution.	Does pathway latent structure transfer at sub-pathway resolution? Extends 5-parent test to 395 Reactome DAG nodes.	v1, z_he cross-patient one-sided BH-FDR<0.05 on 395 sub-pathway nodes: R4 80% sig (317/395) - wins every parent (ECM 93%, Immune 83%, TGF-beta 81%, PCD 75%, Cell Cycle 69%). B2/B3 (rotation-equivalent) 68%. B1 63%. Late-fusion runs at floor: R6 3.8%, R1 3.5%, R3 1.8%, R2 1.5%. R4 top hits: BTLA co-inhibition z=6.92 (immune checkpoint), IL receptor SHC z=6.80, Integrin cell surface z=6.21, ECM proteoglycans z=5.73, IFN-alpha/beta regulation z=5.71, Degradation of ECM z=5.47, Collagen chain trimerization z=5.22.	R4's top hits are biologically distinct sub-programs spanning immune-checkpoint biology (BTLA, IL signaling, IFN regulation) and stromal/ECM remodeling (Integrin, collagen, proteoglycans). The 5-parent off-diagonal cosine 0.75 was a coarse-aggregation artifact from overlapping Reactome gene sets - at sub-pathway resolution R4 distinguishes mechanisms.	Cross-attention improved pathway-level transferability across biologically distinct sub-programs spanning immune-checkpoint biology (relevant to checkpoint-inhibitor response) and stromal/collagen remodeling (relevant to TME architecture). Classical baselines competitive at fine resolution - useful for per-pathway distinguishability.	Per-node univariate CCA z_A on z_he, cross-patient 11/3, perm BH-FDR over 395 nodes	R4 (overall + every parent); B2/B3/B1 (competitive at fine resolution)	R1/R2/R3/R6 at floor (1.5-3.8%)
Systems-Oriented Evaluation	Strongly aligned with the proposal framing of multimodal alignment as orchestration and evaluation, not single-model optimization.	Did one alignment strategy dominate every biological objective simultaneously?	No. R4 wins H1 retrieval + H3 cross-patient transfer + H3 sub-pathway. R1/R5/R6 win H2-A tissue-region clustering. Classical baselines competitive at H3 fine pathway detection but fail H1 retrieval (B1 0.544 vs R4 0.859). B1 amplifies patient identity (ratio 0.071, below raw H&E floor). Trade-off reproduces v1 -> v3.	Niche-level cluster geometry, cross-modal retrieval, and cross-patient pathway transfer are not co-monotonic objectives. R4's tight aliased manifold trades cluster geometry (H2-A) for retrieval + pathway transfer (H1 + H3). R5/R6/R1's wider manifold does the reverse.	The framework exposed biologically meaningful trade-offs supporting task-conditional routing rather than a single globally optimal embedding strategy. omicstra's role is to expose these trade-offs so downstream analyses route the right question to the right method.	Integrated H1 + H2-corrected + H3-cross-patient	R4, R1, R5, R6 (each wins different families)	All baselines lose at least one family decisively; B1 fails H2-C diagnostic

Status and what's next

phase 1 (science) - complete

The full v3 alignment grid (R1-R6, B1-B4) is evaluated end-to-end across H1/H2/H3 with confidence intervals on the headline metrics. Numbers are reproducible from `runs/tnbc-92_v3/eval/*` and the v3 summary notebook. The routing table is the deliverable: it tells downstream analyses which alignment strategy to use for which question, with evidence-cited rationale per route.

phase 2 (agent layer) - read path built, compute path not

The MCP server runs: an EDA admissibility gate that validates a committed summary against a cohort-free contract, an EDA subgraph in LangGraph with a human-in-the-loop interrupt, and the routing table above exposed as a tool. Routing is split into a generalizable contract (`configs/routing_contract.json` - task taxonomy and decision rules, no cohort numbers) and this cohort's measured evidence (`projects/tnbc-92/routing_evidence.json`). A cohort with no evidence file is not routable; it escalates rather than inheriting another cohort's winner. Every decision is emitted as a typed record with a content-addressed id, so two ledgers can be joined to test whether swapping the model changed any decision. Not built: the compute path - `he_agent`, `st_agent`, `alignment_agent`, `eval_agent` and `karpathy_loop` remain the existing scripts (`align.py`, `eval.py`, `build_niche_join.py`), which emit no records. Provenance for the numbers in this report is chain-of-custody via those scripts' artifacts, not replay.

Deferred experiments. Two pre-registered follow-ups are queued and not blocking the current results: (a) DAG-resolution H3 decomposition on the v3 grid (the v1 sub-pathway numbers reported here would be re-derived on v3 - the v1 result is the headline shown in this report), and (b) R5b (AnInfoNCE + cross-attention) as a rescue test for whether per-dim temperature can recover per-pathway specificity while keeping cross-patient transferable magnitude.

Modalities and their foundation models

omicstra's MCP/agent framing treats modalities as pluggable - each modality is a frozen foundation-model encoder + a niche-aggregation step + an evaluation-agent contract (defined in MODALITY_TEMPLATE.md). The current cohort uses H&E + spatial transcriptomics (original ST). Below are the modalities that would add biological signal at the niche or patient level, with the current state of foundation models for each - including the project's anchor modalities and adjacent technologies. FM column is a snapshot; this field moves in months, not years.

modality	what it adds biologically	foundation models available now (survey Jul 2026)	TNBC-relevance + integration cost
H&E (canonical anchor) already integrated; project's primary modality	Tissue architecture, immune geography, stromal composition at cellular scale. Standard of care - generated at diagnosis and archived at institutional scale across decades of clinical practice. The anchor modality everything else is aligned to.	Tile-level: Virchow2 (MSK Paige, 632M, 3M slides - current primary), UNI2-h (Mass General, 681M - swap), H-optimus-0 / H-optimus-1 (Bioptimus), Atlas / Hibou (Histai), CONCH (image + text, Mahmood lab), Phikon-v2 (Owkin), Prov-GigaPath (Microsoft + Providence), CTransPath, HIPT. Slide-level tier (new since v3): TITAN (Mahmood lab, 335,645 WSIs, vision-language aligned to pathology reports, Nat Med 2025), THREADS (slide-level, pretrained on 47,171 H&E paired with genomic + transcriptomic profiles), PRISM (Paige, on Virchow tiles). Most mature FM ecosystem in computational pathology.	Maximum (project anchor). Integration cost: zero - already integrated. Open question: encoder swap to test whether the routing-table verdict holds across pathology FMs. THREADS is the highest-priority swap candidate - it is already molecularly supervised, which is the exact mechanism H2 attributes the alignment gain to, so it directly tests whether contrastive training against ST is redundant with pretraining against bulk molecular profiles.
Spatial transcriptomics (canonical anchor) original ST, Visium; already integrated	Molecularly rich per-location expression. The ST side of every run in the current grid. Research-use-only and hard to interpret without morphological context.	Novae (MICS-Lab, GAT, niche-level, current primary), Nicheformer (Nat Methods 2025, SpatialCorpus-110M), scGPT-spatial (SpatialHuman30M, 30M profiles, MoE decoder), SToFM (ICML 2025), HEIST (graph FM, ST + proteomics), SpatialFormer (Nat Comp Sci 2026), AIDO.Tissue, OmniCell, SpaFoundation, SpatialFusion (pathway-informed niche mapping - overlaps gpath2vec's stated purpose, worth a direct comparison).	Maximum (project anchor). Integration cost: zero. Novae remains the right pick for niche-level encoding and that choice is now defensible on architecture rather than availability: Nicheformer does not ingest spatial coordinates, and scGPT-spatial outputs cell-level rather than neighborhood-level representations.
Multiplex IF / IHC Vectra Polaris, CODEX, Lunaphore	Protein-level spatial maps of 6-50 markers per slide. Directly resolves immune cell composition (CD3/CD4/CD8/CD20/CD68/FoxP3), checkpoint expression (PD-L1, BTLA, LAG3), proliferation (Ki67), and stromal markers (a-SMA, FAP). Complementary to ST: catches translation/post-translational layer the gene-expression signal can miss.	Superseded as of 2025 - a Virchow-equivalent now exists. KRONOS (Mahmood lab), panel-agnostic FM self-supervised on 47M single-marker patches, 175 markers, 16 tissue types, 8 platforms, using a channel-wise stem plus sinusoidal marker-identity embeddings. Supports segmentation-free cell and patch phenotyping and cross-cohort spatial retrieval. Segmentation still via Mesmer / Cellpose / StarDist. In-silico mIF from H&E is now its own tractable line: ROSIE (ConvNeXt, predicts biomarker expression from H&E) and MIPHEI-ViT (pathology-FM encoder, ORION restained H&E/mIF).	High TNBC yield (protein-level immune phenotype is what ICB response stratification depends on). Integration cost: medium - need paired CODEX/Vectra runs on a subset of the same TNBC patients, or use a separate TNBC mxIF cohort (ex. Keren et al. 2018, Schurch et al. 2020). ROSIE/MIPHEI are the generalized form of this report's own "train a TLS detector on Virchow2 features" next step (B4 z=25.5) and should be benchmarked against rather than reinvented.
Spatial proteomics CODEX, IMC, MIBI-TOF	High-plex (30-60 markers) protein imaging at single-cell resolution, spatially co-registered. Closer to "the truth" than ST for immune cell typing because antibodies detect protein, not mRNA. Reveals signaling-active vs signaling-quiescent cells (phospho-marker panels).	KRONOS is the dominant FM here as well - its training corpus spans CODEX, MxIF, COMET and IBEX, so one encoder covers this row, the mIF row and the CyCIF row. Open-source pipelines remain: steinbock (Bodenmiller lab), Mesmer + manual gating.	Medium TNBC yield. Integration cost: high - need paired runs (expensive, $5-15k per slide) or a separate cohort. Best for follow-up validation rather than primary alignment training.
CyCIF (cyclic IF) t-CyCIF, COMET; 40-60 markers	Cyclic immunofluorescence - 40-60 markers via repeated rounds of antibody staining on the same slide (Sorger lab's t-CyCIF, Lunaphore COMET commercial). Resolves cell type + signaling state at single-cell resolution with much higher plex than standard mIF (3-7 markers). Reveals immune cell subtypes and signaling-active vs quiescent states that ST and mIF can only approximate.	KRONOS (COMET is one of its eight pretraining platforms), plus the imaging stack: MCMICRO (Lin lab) for registration, Cellpose / StarDist / Mesmer for segmentation, scimap for downstream analysis.	Very high - published TNBC CyCIF datasets exist (Lin / Sorger lab work). Most aligned with the project's TLS-sub-types finding, since CyCIF can directly resolve germinal-center vs T-cell-zone vs plasma-cell composition in TLS. Integration cost: medium-high (paired runs or use existing TNBC CyCIF cohorts).
Single-cell RNA-seq 10x Chromium, smart-seq3	Per-cell transcriptomic profile - resolves cell types that ST can only see as mixtures (spots are ~100um, contain ~10-50 cells). Reference-mapping scRNA atlases to ST niches recovers per-cell composition. For TNBC, lets you distinguish CD8 effector vs CD8 exhausted T cells within a single ST spot.	Previous generation: scGPT, Geneformer, scFoundation, Universal Cell Embeddings, CellPLM. Current generation: CellFM (800M params, 100M human cells, Nat Commun 2025), STATE and Stack (Arc Institute; Stack does in-context learning over 149M cells with no task-specific finetuning), C2S-Scale (LLM-based, to 27B params over 50M cells + text), TranscriptFormer (CZI, 112M cells across 12 species), GeneMamba (SSM architecture, linear complexity). Many mature options; benchmark before committing.	High TNBC yield - existing TNBC scRNA atlases (Wu et al. 2021, Bassez et al. 2021 anti-PD-1 cohort, Pal et al. 2021) are public. Integration cost: low - reference-map via Tangram/cell2location, then feed cell-composition vector or pretrained-FM embedding as an additional ST-side input to the alignment grid. Still the highest yield-per-unit-cost extension in this table.
Visium HD 10x Genomics, 2 um spot pitch	Next-generation spatial transcriptomics at 2 um x 2 um spot resolution (vs 100 um diameter / 150 um spacing in the original ST platform used here). Effectively subcellular resolution: each cell is covered by 5-10 HD spots. Resolves individual immune cells within a niche where original ST only sees cell mixtures.	ST-side FMs with much better fit than v3 implied: scGPT-spatial was pretrained on a corpus explicitly including Visium HD alongside Visium, Xenium and MERFISH, with a mixture-of-experts decoder for protocol heterogeneity. Novae was trained on subcellular image-based ST (MERSCOPE/Xenium/CosMX), so HD resolution is closer to its training distribution than the original ST platform. Bin-aggregated HD spots (8 um, 16 um) can be treated as Visium-like for legacy pipelines.	High - would address the 100 um spot-mixture problem that motivates current niche aggregation (k=6 neighbors, ~1200 cells). Integration cost: medium - new Visium HD tissue runs required (existing TNBC-92 cannot be re-acquired at HD resolution). Best applied to a successor cohort.
DNA methylation EPIC/EPICv2 array, WGBS, cfDNA methylation	Genome-wide CpG methylation - the stable epigenetic layer. Gives tissue-of-origin, tumor subtype, and cell-of-origin calls that expression cannot, because methylation state is far less sensitive to sampling and preservation than RNA. Methylation classifiers are already clinical-grade in some tumor types. Deconvolutes bulk immune composition without single-cell data.	MethylGPT (transformer, 154,063 human methylomes from 5,281 datasets, 49,156 CpG sites, 7.6B tokens) and CpGPT (sequence-, position- and epigenetic-context aware, strong cross-cohort generalization, cross-platform imputation). Both mature enough for frozen-encoder use. Classical baselines remain strong (penalized linear models, PC-based clocks).	High TNBC clinical relevance - methylation subtyping, HRD-associated promoter silencing (ex. BRCA1), and cfDNA methylation for minimal residual disease. Integration cost: low at the patient level (array data is cheap and widely archived on FFPE), high if a spatially-resolved methylation read-out is wanted. Most useful at the patient-stratification layer, not niche-level alignment.
scATAC-seq / multi-omic 10x Multiome, scNMT	Chromatin accessibility per cell - the regulatory layer upstream of gene expression. Identifies which transcription factors are bound and active, which enhancers are open, which lineage decisions are being made. Complementary to gpath2vec's TF-restricted Fisher view.	scBasset (Yuan & Kelley 2022), CHROMBPNet (Pampari et al. 2023), Geneformer handles peak-counts, ENFORMER for sequence-level. Less mature than scRNA FMs but production-ready for the most common use cases.	Medium TNBC yield. Integration cost: medium - paired scATAC on TNBC cohorts is less common than scRNA. Useful for validating gpath2vec's TF interpretation directly (do the TFs gpath2vec catches have chromatin accessibility at their motifs?).
Spatial multi-omics (co-profiling) spatial CITE-seq, SPOTS, spatial ATAC-RNA-seq, MISAR-seq, DBiT-seq, Slide-tags	Two or more molecular layers measured on the same section in one assay - RNA + surface protein (spatial CITE-seq, SPOTS), RNA + chromatin accessibility (spatial ATAC-RNA-seq, MISAR-seq), or barcoded whole-cell capture (Slide-tags). Early demonstrations of RNA + protein + ATAC in a single workflow exist.	No single dominant FM. Integration frameworks are the current state of the art: MultiSP (Cell Genomics 2026, cross-modality adversarial learning + modality-specific generative modeling), spEMO (Nat Biomed Eng 2026, unifies pathology-FM embeddings with spatial multi-omic data). Per-layer encoders from the rows above (Novae/scGPT-spatial for RNA, KRONOS for protein, scBasset for ATAC) are the practical composition today.	Structurally the most important new row for omicstra, independent of biological yield. Co-profiled modalities give matched pairs on the same section with no registration step, which removes the single largest integration cost in this table and eliminates the serial-section alignment error that every "paired run" row above quietly assumes away. Integration cost: medium and falling.
Spatial epigenomics Spatial-CUT&Tag, Spatial-ATAC, Spatial-DBiT	Spatially-resolved histone marks (H3K27ac, H3K27me3) or chromatin accessibility at niche scale. Identifies which enhancers and regulatory elements are active in specific tissue regions - the regulatory layer directly upstream of ST gene expression. Published methods include Spatial-CUT&Tag (Deng et al.), DBiT-seq, and emerging commercial platforms.	scBasset and CHROMBPNet for chromatin-state encoding, ENFORMER for sequence-level. Spatial-epigenomics-specific FMs are nascent - most analyses currently use peak-calling + ChromVAR-style scoring. See also the co-profiling row: spatial ATAC-RNA-seq delivers this layer already paired with expression.	High biological yield - directly validates gpath2vec's TF-restricted Fisher predictions by measuring chromatin accessibility at the predicted TFs' motifs. Low integration maturity. Useful when assays become more standardized.
Spatial metabolomics MALDI-MSI, DESI	Spatially-resolved small-molecule abundance (lipids, amino acids, drugs, metabolites). Directly captures the metabolic state of the TME - hypoxia, fatty acid oxidation, glutaminolysis, ICB drug penetration. Complements expression-based modalities by measuring effector molecules directly.	First FM candidate now exists: MetaboFM (self-supervised ViT representations over a curated MSI corpus spanning species, ionization modes, tissues and instrument types), explicitly framed as filling the gap left by KRONOS and scFoundation in adjacent modalities. Preprint stage - treat as emerging, not settled. METASPACE ecosystem and m2aia remain the production pipelines; most workflows still use peak-fitting + per-metabolite quantification.	High novel-biology yield, low integration maturity. Co-registration with H&E is technically tractable but requires same-section runs. Most useful as orthogonal validation of metabolic signatures inferred from ST.
TCR/BCR repertoire scTCR-seq, repertoire deep-seq, Slide-TCR-seq, Spatial VDJ	Per-cell TCR/BCR sequences from infiltrating T and B cells. Directly resolves immune clonality, antigen specificity (where TCR specificity databases exist), and shared vs private clones. Highly informative for TLS biology (BCR clonal expansion in germinal centers) and ICB response. Spatial variants (Slide-TCR-seq, Spatial VDJ) now place clonality directly on the tissue grid, removing the dissociation step that previously made this modality niche-incompatible.	TCR-BERT, ESM-2 (and now ESM-3) for TCR/BCR protein sequences, TCRdist for similarity. FM space maturing fast. Most production pipelines still use per-clone summary statistics. No spatial-repertoire-specific FM yet.	High TNBC yield - tied directly to the TLS sub-types identified in this report (C1 germinal-center-like, C2 neutrophil/innate, C3 translation/effector). Spatial VDJ would test whether those three gpath2vec sub-types differ in BCR clonal expansion, which is the direct mechanistic prediction. Integration cost: high - paired spatial repertoire + ST is rare. Public TNBC TCR repertoires from Bassez et al. 2021 are usable at the dissociated level.
IHC (single-marker) DAB, chromogenic; clinical standard	Single-marker protein staining at the slide level - the clinical pathologist's everyday tool. ER / PR / HER2 / Ki67 (the breast cancer essentials, what defines "triple-negative"), PD-L1 (ICB stratification), CD3 / CD8 (TIL scoring per CAP/ASCO guidelines). Available in essentially every pathology archive worldwide.	Pathology FMs (Virchow2, UNI2, TITAN) work directly on IHC slides plus marker-specific quantifiers (positivity scoring, H-score). HistoQC for quality control, QuPath for analysis pipelines, HistoSegNet for ROI segmentation. Note UNI2-h was pretrained on H&E and IHC.	Very high - clinical-standard markers are how TNBC patients are typed in the first place. Integration cost: low - attach as patient-level or spot-level continuous markers to the alignment grid; useful as orthogonal validation for ST-derived immune phenotype.
Bulk RNA-seq + clinical TCGA, METABRIC, in-house	Patient-level transcriptomic profile + outcomes (OS, DFS, ICB response, treatment trajectory). Already partially used in this project as Bareche TIME/MC_global/MC_tumor patient-level labels. Adds a clinical-outcome layer for risk-stratification training.	FMs less load-bearing for bulk - traditional pipelines (DESeq2, GSEA, ssGSEA, decoupler) work well. scGPT and C2S-Scale can ingest pseudobulk. THREADS is trained on H&E paired with bulk molecular profiles, so it effectively encodes this modality into the H&E tower.	High TNBC yield for risk stratification. Integration cost: low - patient-level features attach as side inputs to the alignment grid or as outcome targets for downstream prediction.
WES/WGS + mutational signatures DNA-seq, mutational signatures	Patient-level DNA variants, copy-number alterations, mutational signatures (BRCA1/2 status, HRD score). Critical for TNBC stratification - BRCA1/2-mutated TNBC responds differently to PARPi than HRD-negative. Could route patients to alignment-strategy subsets.	ENFORMER for regulatory variants, AlphaMissense for missense, SigProfiler / deconstructSigs for signature decomposition. No single dominant patient-level genomic FM yet.	High TNBC clinical relevance. Integration cost: low for patient-level metadata, medium for treating mutational profile as a learned representation. Most useful at downstream patient-stratification layer, not at niche-level alignment.
LCM / DNA-seq laser capture microdissection + spatial genomics	Laser capture microdissection of specific tissue regions (tumor nests / stroma / TLS / invasive front) followed by DNA sequencing. Reveals spatially-resolved mutational architecture - clonal evolution at the tumor-immune interface, TLS-resident B-cell clonality, sub-clonal heterogeneity within tumor regions.	DNA sequencing FMs less mature for LCM use cases. Standard pipelines: variant calling (MuTect2, Strelka), clonal inference (PyClone, SciClone, dpClust), mutational signature decomposition. Emerging sequence-level FMs (ENFORMER, AlphaMissense) work at the variant level.	High - directly informs intratumoral heterogeneity and ICB resistance evolution. Integration cost: high - LCM is slow and expensive; few paired LCM + ST cohorts exist publicly. Best applied as a follow-up experiment on a specific hypothesis (ex. TLS-resident clonal expansion in responders vs non-responders).
CODA Coupled Object Detection-Annotation; serial-section 3D mapping	3D tissue reconstruction from serial H&E sections (~4 um apart) with cell-type and compartment annotations. Adds the third spatial dimension (z) currently flat at 2D - quantifies 3D tumor geometry, vascular network density, immune-aggregate volume, invasion-front topology. Developed by the Kiemen lab (Johns Hopkins) for high-throughput 3D pancreas / breast tissue mapping.	Task-specific deep-learning models (semantic segmentation + cell-type classification) trained per-tissue rather than a single generic FM. The underlying segmentation models can ingest Virchow2 / UNI2 features. CODA itself is more a production pipeline than a pretrained encoder; the FM layer is the H&E backbone.	Medium for the current cohort (would require serial sections, not in the TNBC-92 protocol). High for prospective cohorts where 3D context matters for TLS biology (germinal-center volume vs surface area), tumor-immune interface geometry, and invasive front analysis.
Intratumoral microbiome INVADEseq, SmT, SHM-seq, HiPR-FISH	Spatially-resolved bacterial localization within tumor tissue, co-detected with host transcriptome. Intratumoral bacteria localize to specific TME compartments and associate with immune exclusion and chemotherapy metabolism.	No FM. Standard pipelines: 16S/metagenomic classification (Kraken2, MetaPhlAn) plus host-microbe co-detection assays.	Low-to-medium TNBC yield at present, but a genuinely orthogonal axis - it measures something no other row in this table can see. Integration cost: high (requires the specific co-detection assay). Listed for completeness; not a near-term route.
Radiology (MRI / CT / PET-CT) patient-level imaging; clinical-decision bridge	Pre-treatment radiologic phenotype - tumor size, lymph node involvement, distant metastases, FDG-PET metabolic activity, MRI background parenchymal enhancement (BPE). Patient-level, not niche-level - bridges the tissue-scale findings to whole-patient clinical decisions (chemotherapy regimen, surgical extent, ICB candidacy).	RAD-DINO (Microsoft), BiomedCLIP, MedCLIP, LLaVA-Med, MICA. Mature for radiology - several pretrained encoders with reasonable cross-institutional generalization.	Medium for niche-level alignment, very high for patient-level risk stratification. Integration cost: low - patient-level radiologic features attach as additional patient-side metadata. Bridges H&E-derived TME phenotype to whole-body clinical context.
H&E x ST pretrained alignment not a modality - a pretrained cross-modal encoder	Not an additional data modality but a pretrained version of the alignment this project trains from scratch. Relevant because it changes what the R1-R6 / B1-B4 grid is being compared against: every arm in the current grid is trained in-project on TNBC-92, so the grid measures "which in-project objective is best", not "does in-project training beat a general pretrained alignment".	OmiCLIP (Nat Methods 2025), a transcriptomic-image dual-encoder trained on the ST-bank corpus of 2.2M paired tissue patches from 1,007 samples across 32 organs, representing each patch's transcriptome as a "sentence" of top-expressed gene symbols. Its Loki platform exposes tissue alignment, annotation, cell-type decomposition, image-transcriptomics retrieval and ST expression prediction from H&E; benchmarked against 22 methods across 5 simulations, 19 public and 4 in-house datasets. See also spEMO (Nat Biomed Eng 2026) for the multi-omic generalization.	Immediate and methodological rather than biological. OmiCLIP's image-to-transcriptomics retrieval function is the same task as H1, so it is the natural B5 baseline arm: a frozen pretrained cross-modal alignment evaluated on the identical H1/H2/H3 surface. Integration cost: low - public weights, no new tissue required, runs on existing TNBC-92 embeddings. The routing-table row it would add ("use a general pretrained alignment vs a cohort-specific one, and for which question") is arguably the most transferable row in the table.

What this means for omicstra's roadmap. The current 10-run grid evaluates two-modality (H&E + ST) alignment. The framework is designed to extend to 3+ modalities via the same MCP/agent pattern - each new modality adds an encoder agent, a niche-aggregation step, and rows to the alignment grid. Three things changed since v3 was written. First, the "no FM exists" verdict on the protein-imaging family is retired: KRONOS covers mIF, spatial proteomics and CyCIF with one panel-agnostic encoder, which collapses three medium-cost rows into one. Second, co-profiling assays (spatial CITE-seq, spatial ATAC-RNA-seq, Slide-tags) remove the serial-section registration step that every "paired run" row in this table otherwise assumes away, and are now the cheapest honest route to a third modality. Third, a pretrained H&E-ST alignment (OmiCLIP/Loki) now exists and belongs in the grid as a baseline arm, not in the roadmap. Highest near-term yield for TNBC remains scRNA-seq reference mapping (low cost, mature FMs, high biological lift) and multiplex IF (high biological lift on immune phenotype, and the FM maturity bet has now resolved in its favor). Both plug into the same H1/H2/H3 evaluation surface and the same routing-table output.

Results summary · workflow investigation

Did we do what the proposal said we would do?

Architecture - what's actually being aligned

The four per-niche input features

How the inputs combine - two different "+" operations

Run definitions (all 10)

How gpath2vec was built (v3 build)

Metrics in plain English

H1 - retrieval and structural agreement (5 metrics)

H2 - biology preservation (5 metrics)

H3 - cross-patient pathway transfer (1 metric family + 2 methodological tools)

H1 · cross-modal retrieval and structural alignment

H2 · structural biology preservation

Part A - niche clustering (corrected label: mc_megacluster)

H2 · annotation taxonomy - what requires what

TLS sub-types - molecular heterogeneity within the "Lymphoid nodule" annotation

H3 · pathway interpretability - cross-patient transfer

Task-conditional routing table (v3)

Consolidated proposal-alignment matrix (v3)

Status and what's next

Modalities and their foundation models