Overview

Each task involves predicting the expression levels of the 50 most variable genes from 112×112 μm H&E-stained image patches centered on each spatial transcriptomics spot. The tasks are formulated as multivariate regression problems.

Task ID	Oncotree	Number of Samples	Technology	Sample ID
Task 1	IDC	4	Xenium	TENX95, TENX99, NCBI783, NCBI785
Task 2	PRAD	23	Visium	MEND139~MEND162
Task 3	PAAD	3	Xenium	TENX116, TENX126, TENX140
Task 4	SKCM	2	Xenium	TENX115, TENX117
Task 5	COAD	4	Xenium	TENX111, TENX147, TENX148, TENX149
Task 6	READ	4	Visium	ZEN36, ZEN40, ZEN48, ZEN49
Task 7	ccRCC	24	Visium	INT1~INT24
Task 8	LUAD	2	Xenium	TENX118, TENX141
Task 9	IDC-LymphNode	4	Visium	NCBI681, NCBI682, NCBI683, NCBI684

Reproducing HEST-Benchmark results

Ensure that HEST has been properly installed (see README, Installation)
Automatic download preprocessed patches, h5ad and gene targets
Automatic download of publicly available patch encoders

Note: Not all public foundation models can be shared due to licensing issues. We provide model-specific instructions that users can follow to access weights:

CONCH installation (model + weights request)

Request access to the model weights from the Huggingface model page here.
Install the CONCH PyTorch model:

pip install git+https://github.com/Mahmoodlab/CONCH.git

UNI weights request

Request access to the model weights from the Huggingface model page here.

GigaPath weights request

Request access to the model weights from the Huggingface model page here.

Remedis (weights only)

Request access to the model weights from the Huggingface model page here.
Download the model weights (path-152x2-remedis-m_torch.pth) and place them in {weights_root}/fm_v1/remedis/path-152x2-remedis-m_torch.pth where weights_root is specified in the config. You can also directly modify the path of remedis in {PATH_TO_HEST/src/hest/bench/local_ckpts.json}.

Launching HEST-bench via CLI

%%bash
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libffi.so.7
python ../src/hest/bench/benchmark.py --config ../bench_config/bench_config.yaml

Benchmarking your own model with HEST-Benchmark

from hest.bench import benchmark
import torch

PATH_TO_CONFIG = .. # path to `bench_config.yaml`
model = .. # PyTorch model (torch.nn.Module)
model_transforms = .. # transforms to apply during inference (torchvision.transforms.Compose)
precision = torch.float32

benchmark(        
    model, 
    model_transforms,
    precision,
    config=PATH_TO_CONFIG, 
)

Reproducing: finding genes of interest for HEST benchmark

import scanpy as sc
from hest import get_k_genes

# TODO add full paths to samples of interest here
sample_paths = ['TENX118.h5ad', 'TENX141.h5ad']

ad_list = [ sc.read_h5ad(sample_path) for sample_path in sample_paths]
genes = get_k_genes(ad_list, k=50, criteria="var", min_cells_pct=0.1)
print(len(genes))