LLM driven nuanced cell type anntotation

[parashardhapola]

[EDITS: 2025-07-30: Added new screenshots from updated UI and added links for updated version]

Samples to be annotated

All the samples on scPCA. 😊

As an example, I have annotated all the 23 samples in the SCPCP000001 project (high grade gliomas).

Here are the annotation reports:
CyteType Report: SCPCS000001
CyteType Report: SCPCS000002
CyteType Report: SCPCS000003
CyteType Report: SCPCS000004
CyteType Report: SCPCS000005
CyteType Report: SCPCS000006
CyteType Report: SCPCS000007
CyteType Report: SCPCS000008
CyteType Report: SCPCS000009
CyteType Report: SCPCS000010
CyteType Report: SCPCS000011
CyteType Report: SCPCS000012
CyteType Report: SCPCS000013
CyteType Report: SCPCS000014
CyteType Report: SCPCS000015
CyteType Report: SCPCS000016
CyteType Report: SCPCS000017
CyteType Report: SCPCS000018
CyteType Report: SCPCS000019
CyteType Report: SCPCS000020
CyteType Report: SCPCS000021
CyteType Report: SCPCS000022
CyteType Report: SCPCS000023

Example Annotation card:

• Overview
• Justification and Review
• Supporting Literature Evidence
• Chat with a cluster

Input data

h5ad files (AnnData objects). They should have cluster information (in obs) and the gene symbols (in var).

Detailed protocol for cell type annotation

For annotation we will use the CyteType package that uses a multi-agent LLM framework for annotation. I have chosen Deepseek-R1 0528 as the underlying LLM, due to a good trade-off between cost, speed and detailed justification. However, other LLMs can also be tested.
The LLMs are provided with a context for each cluster by summarizing the metadata information connected with that particular cluster.
In terms of preprocessing, a marker gene search needs to be performed for the clusters present in the data. The top 75 marker genes are provided to CyteType.

Fully self-contained notebook is available here:
https://colab.research.google.com/drive/1BDUQwH1mIoX1cJQEtTt_4gY9Es90X-Yg?usp=sharing

Citations of scientific literature or publicly available analyses

https://github.com/NygenAnalytics/CyteType
CyteType is the only tool that provides detailed contextual information about the clusters and is able to adapt to the disease contexts.

Potential pitfalls

LLMs can hallucinate, so detailed checks across LLMs can be performed. CyteType mitigates this to a large extent through implementation of a reviewer agent that double checks the annotation and adds a confidence score to the annotation.

Required files

All information is pulled from H5ad files.

Other details

None beyond regular laptops. For large AnnData objects (more than 100K cells), subsampling can be performed as shown in the CyteType documentation.

[Jen-OMalley]

Hi @parashardhapola. I'm Jen, the Scientific Community Manager at the Data Lab. Thank you for interest in the OpenScPCA project! Our team is currently reviewing your proposed ideas, and we look forward to discussing more with you soon. We will get back to you here with any questions and/or next steps within three business days. Please let me know if you have any questions about OpenScPCA. We look forward to hearing more about you plans!

[Jen-OMalley]

@parashardhapola In the meantime, here is the contributor form. Filling out this form ensures you have agreed to the OpenScPCA terms and conditions and other policies. Please submit this at your earliest convenience. Thank you!

[parashardhapola]

Thanks @Jen-OMalley. I have filled out the form and am excited about the possibility of contributing to OpenScPCA

[jaclyn-taroni]

Hi @parashardhapola, thanks for submitting this and your interest in OpenScPCA! This is really interesting!

First, I wanted to note that only employees of or researchers with an appointment at a non-profit are eligible for OpenScPCA-related grants. You are welcome to contribute either way, but I wanted to say that upfront just in case that is a barrier (complete eligibility information is available here: https://openscpca.readthedocs.io/en/latest/grant-opportunities/).

Regarding the proposal itself, we had some initial, general questions for you and about the CyteType method:

• Are you planning on performing these analyses or recommending that the Data Lab team perform them? This discussion type is for the latter, but they are admittedly confusingly named, so I wanted to clarify.
• Can you comment on what exactly gets shared with model providers? Do you also tend to see better results when you use different model providers for various aspects of annotation (e.g., the reviewer)?
• Relatedly, can local models be used? Do you have experience using CyteType with something like Amazon Bedrock, where nothing is shared with model providers?
• Do these example reports use the cluster assignments already contained in the objects? (We are not necessarily confident that the parameters we use for assignments across all samples produce high-quality clusters for a given individual sample.)
• Do you have recommendations about how cell type labels should be reconciled across samples within a project?

Thanks again for your proposal! We’re looking forward to continuing the discussion.

[jaclyn-taroni]

Thanks for the helpful reply, @parashardhapola!

Having someone on our team run this sounds good. We would probably pick a set of samples that no one else is working on to start, and we might want to do some independent QC on the clusters themselves in parallel.

Do you have any documentation on setting CyteType to work with Ollama? It would be great to try that first.

Regarding the cross-sample issue, we might first try integrating samples within a project, rerunning CyteType, and comparing them to the individual sample reports, but I haven’t thought this through fully yet.

[parashardhapola]

Hi @jaclyn-taroni. This sounds like a good plan.

We have added information on how to Ollama with CyteType here.

[parashardhapola]

Hi @jaclyn-taroni!

Did @CCdatalab team has had a chance to run CyteType?
Let me know if any support is needed from my end :)

[jaclyn-taroni]

Hi @parashardhapola! We haven't had a chance to look yet. Unfortunately, we expect to be fairly tied up with changes to the ScPCA pipeline and Portal related to paper revisions over the next few months, but we hope to circle back after that has been completed.

[parashardhapola]

Hi @jaclyn-taroni I understand entirely! Best wishes for the revision, and looking forward to seeing the published version :)

If I'd help, then I'd be happy to run CyteType across all of scPCA; however, we will still need to spend significant time to integrate those results in the scPCA portal (if we like the results, that is); hence, as you suggest, it's better to wait until the team has more bandwidth.