helicalAI · bputzeys · May 2, 2025 · Apr 17, 2025 · May 1, 2025 · May 1, 2025
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -93,6 +93,10 @@ jobs:
         run: |
           python examples/fine_tune_models/fine_tune_UCE.py ++device="cuda"
 
+      - name: Execute Transcriptformer
+        run: |
+          python examples/run_models/run_transcriptformer.py
+
       - name: Execute Hyena
         run: |
           python examples/run_models/run_hyena_dna.py ++device="cuda"

diff --git a/README.md b/README.md
@@ -31,13 +31,8 @@ Let’s build the most exciting AI-for-Bio community together!
 
 ## What's new?
 
-### Saving fine-tuned models
-
-We give users the option now to save fine-tuned models. Please have a look at the examples folder [here](examples/fine_tune_models). 
-
-### Evo2
-We have integrated [Evo2](https://github.com/ArcInstitute/evo2) into our helical package and have made a model card for it in our [Evo2 model folder](helical/models/evo_2/README.md). If you would like to test the model, take a look at our [example notebook](examples/notebooks/Evo-2.ipynb)!
-Let us know what you think and we are happy to help you with the larger model (40B parameters!) if needed!
+### TranscriptFormer
+We have integrated [TranscriptFormer](https://github.com/czi-ai/transcriptformer) into our helical package and have made a model card for it in our [Transcriptformer model folder](helical/models/transcriptformer/README.md). If you would like to test the model, take a look at our [example notebook](examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb)!
 
 ### 🧬 Introducing Helix-mRNA-v0: Unlocking new frontiers & use cases in mRNA therapy 🧬
 We’re thrilled to announce the release of our first-ever mRNA Bio Foundation Model, designed to:
@@ -102,6 +97,7 @@ apptainer shell --nv --fakeroot singularity/helical/
 - [Geneformer](https://helical.readthedocs.io/en/latest/model_cards/geneformer/)
 - [scGPT](https://helical.readthedocs.io/en/latest/model_cards/scgpt/)
 - [Universal Cell Embedding (UCE)](https://helical.readthedocs.io/en/latest/model_cards/uce/)
+- [TranscriptFormer](https://helical.readthedocs.io/en/latest/model_cards/transcriptformer/)
 
 ### DNA models:
 - [HyenaDNA](https://helical.readthedocs.io/en/latest/model_cards/hyena_dna/)
@@ -125,7 +121,7 @@ Within the `examples/notebooks` folder, open the notebook of your choice. We rec
 | ----------- | ----------- |----------- |                                                        
 |[Quick-Start-Tutorial.ipynb](./examples/notebooks/Quick-Start-Tutorial.ipynb)| A tutorial to quickly get used to the helical package and environment. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Quick-Start-Tutorial.ipynb)|
 |[Helix-mRNA.ipynb](./examples/notebooks/Helix-mRNA.ipynb)|An example of how to use the Helix-mRNA model.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Helix-mRNA.ipynb) |
-|[Geneformer-vs-UCE.ipynb](./examples/notebooks/Geneformer-vs-UCE.ipynb) | Zero-Shot Reference Mapping with Geneformer & UCE and compare the outcomes. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Geneformer-vs-UCE.ipynb) |
+|[Geneformer-vs-TranscriptFormer.ipynb](./examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb) | Zero-Shot Reference Mapping with Geneformer & UCE and compare the outcomes. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb) |
 |[Hyena-DNA-Inference.ipynb](./examples/notebooks/Hyena-DNA-Inference.ipynb)|An example how to do probing with HyenaDNA by training a neural network on 18 downstream classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Hyena-Dna-Inference.ipynb) |
 |[Cell-Type-Annotation.ipynb](./examples/notebooks/Cell-Type-Annotation.ipynb)|An example how to do probing with scGPT by training a neural network to predict cell type annotations.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Annotation.ipynb) |
 |[Cell-Type-Classification-Fine-Tuning.ipynb](./examples/notebooks/Cell-Type-Classification-Fine-Tuning.ipynb)|An example how to fine-tune different models on classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Classification-Fine-Tuning.ipynb) |
@@ -154,6 +150,7 @@ A lot of our models have been published by talend authors developing these excit
 - [scGPT](https://github.com/bowang-lab/scGPT/)
 - [Geneformer](https://huggingface.co/ctheodoris/Geneformer)
 - [UCE](https://github.com/snap-stanford/UCE)
+- [TranscriptFormer](https://github.com/czi-ai/transcriptformer)
 - [HyenaDNA](https://github.com/HazyResearch/hyena-dna)
 - [anndata](https://github.com/scverse/anndata)
 - [scanpy](https://github.com/scverse/scanpy)
@@ -172,6 +169,7 @@ You can find the Licenses for each model implementation in the model repositorie
 - [scGPT](https://github.com/helicalAI/helical/blob/release/helical/models/scgpt/LICENSE)
 - [Geneformer](https://github.com/helicalAI/helical/blob/release/helical/models/geneformer/LICENSE)
 - [UCE](https://github.com/helicalAI/helical/blob/release/helical/models/uce/LICENSE)
+- [TranscriptFormer](https://github.com/helicalAI/helical/blob/release/helical/models/transcriptformer/LICENSE)
 - [HyenaDNA](https://github.com/helicalAI/helical/blob/release/helical/models/hyena_dna/LICENSE)
 - [Evo2](https://github.com/helicalAI/helical/blob/release/helical/models/evo_2/LICENSE)
 

diff --git a/docs/configs/transcriptformer.md b/docs/configs/transcriptformer.md
@@ -0,0 +1,5 @@
+::: helical.models.transcriptformer.TranscriptFormerConfig
+    handler: python
+    options:
+      show_root_heading: True
+      show_source: True
diff --git a/docs/index.md b/docs/index.md
@@ -11,13 +11,8 @@ We will update this repo on a regular basis with new models, benchmarks, modalit
 Let’s build the most exciting AI-for-Bio community together!
 
 ## What's new?
-### Saving fine-tuned models
-
-We give users the option now to save fine-tuned models. Please have a look at the examples folder [here](examples/fine_tune_models). 
-
-### Evo 2
-We have integrated [Evo 2](https://github.com/ArcInstitute/evo2) into our helical package and have made a model card for it in our [Evo 2 model card](./model_cards/evo_2.md). If you would like to test the model, take a look at our [example notebook](./notebooks/Evo-2.ipynb)!
-Let us know what you think and we are happy to help you with the larger model (40B parameters!) if needed!
+### TranscriptFormer
+We have integrated [TranscriptFormer](https://github.com/czi-ai/transcriptformer) into our helical package and have made a model card for it in our [Transcriptformer model folder](helical/models/transcriptformer/README.md). If you would like to test the model, take a look at our [example notebook](examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb)!
 
 ### 🧬 Introducing Helix-mRNA-v0: Unlocking new frontiers & use cases in mRNA therapy 🧬
 We’re thrilled to announce the release of our first-ever mRNA Bio Foundation Model, designed to:
@@ -82,6 +77,7 @@ apptainer shell --nv --fakeroot singularity/helical/
 - [Geneformer](./model_cards/geneformer.md)
 - [scGPT](./model_cards/scgpt.md)
 - [Universal Cell Embedding (UCE)](./model_cards/uce.md)
+- [TranscriptFormer](./model_cards/transcriptformer.md)
 
 ### DNA models:
 - [HyenaDNA](./model_cards/hyenadna.md)
@@ -105,7 +101,7 @@ Within the `example/notebooks` folder, open the notebook of your choice. We reco
 | ----------- | ----------- |----------- |                                                        
 |[Quick-Start-Tutorial.ipynb](./notebooks/Quick-Start-Tutorial.ipynb)| A tutorial to quickly get used to the helical package and environment. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Quick-Start-Tutorial.ipynb)|
 |[Helix-mRNA.ipynb](./notebooks/Helix-mRNA.ipynb)|An example of how to use the Helix-mRNA model.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Helix-mRNA.ipynb) |
-|[Geneformer-vs-UCE.ipynb](./notebooks/Geneformer-vs-UCE.ipynb) | Zero-Shot Reference Mapping with Geneformer & UCE and compare the outcomes. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Geneformer-vs-UCE.ipynb) |
+|[Geneformer-vs-TranscriptFormer.ipynb](./notebooks/Geneformer-vs-TranscriptFormer.ipynb) | Zero-Shot Reference Mapping with Geneformer & UCE and compare the outcomes. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb) |
 |[Hyena-DNA-Inference.ipynb](./notebooks/Hyena-DNA-Inference.ipynb)|An example how to do probing with HyenaDNA by training a neural network on 18 downstream classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Hyena-DNA-Inference.ipynb)|
 |[Cell-Type-Annotation.ipynb](./notebooks/Cell-Type-Annotation.ipynb)|An example how to do probing with scGPT by training a neural network to predict cell type annotations.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Annotation.ipynb) |
 |[Cell-Type-Classification-Fine-Tuning.ipynb](./notebooks/Cell-Type-Classification-Fine-Tuning.ipynb)|An example how to fine-tune different models on classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Classification-Fine-Tuning.ipynb) |
@@ -134,10 +130,15 @@ A lot of our models have been published by talented authors developing these exc
 - [scGPT](https://github.com/bowang-lab/scGPT/)
 - [Geneformer](https://huggingface.co/ctheodoris/Geneformer)
 - [UCE](https://github.com/snap-stanford/UCE)
+- [TranscriptFormer](https://github.com/czi-ai/transcriptformer)
 - [HyenaDNA](https://github.com/HazyResearch/hyena-dna)
 - [anndata](https://github.com/scverse/anndata)
 - [scanpy](https://github.com/scverse/scanpy)
 - [transformers](https://github.com/huggingface/transformers)
+- [scikit-learn](https://github.com/scikit-learn/scikit-learn)
+- [GenePT](https://github.com/yiqunchen/GenePT)
+- [Caduceus](https://github.com/kuleshov-group/caduceus)
+- [Evo2](https://github.com/ArcInstitute/evo2)
 
 ### Licenses
 
@@ -148,10 +149,10 @@ You can find the Licenses for each model implementation in the model repositorie
 - [scGPT](https://github.com/helicalAI/helical/blob/release/helical/models/scgpt/LICENSE)
 - [Geneformer](https://github.com/helicalAI/helical/blob/release/helical/models/geneformer/LICENSE)
 - [UCE](https://github.com/helicalAI/helical/blob/release/helical/models/uce/LICENSE)
+- [TranscriptFormer](https://github.com/helicalAI/helical/blob/release/helical/models/transcriptformer/LICENSE)
 - [HyenaDNA](https://github.com/helicalAI/helical/blob/release/helical/models/hyena_dna/LICENSE)
 - [Evo2](https://github.com/helicalAI/helical/blob/release/helical/models/evo_2/LICENSE)
 
-
 ## Citation
 
 Please use this BibTeX to cite this repository in your publications:

diff --git a/docs/model_cards/transcriptformer.md b/docs/model_cards/transcriptformer.md
@@ -0,0 +1 @@
+../../helical/models/transcriptformer/README.md
diff --git a/docs/models/transcriptformer.md b/docs/models/transcriptformer.md
@@ -0,0 +1,9 @@
+::: helical.models.transcriptformer.TranscriptFormer
+    handler: python
+    options:
+      members:
+        - process_data
+        - get_embeddings
+        - get_output_adata
+      show_root_heading: True
+      show_source: True
diff --git a/docs/notebooks/Geneformer-vs-TranscriptFormer.ipynb b/docs/notebooks/Geneformer-vs-TranscriptFormer.ipynb
@@ -0,0 +1 @@
+../../examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb
diff --git a/docs/notebooks/Geneformer-vs-UCE.ipynb b/docs/notebooks/Geneformer-vs-UCE.ipynb
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		../../helical/models/transcriptformer/README.md
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		../../examples/notebooks/Geneformer-vs-TranscriptFormer.ipynb