Clarify metadata documentation

[sjspielman]

While portal-wide metadata includes samples with bulk libraries, the seq_unit column in the actual portal metadata (and other similar metadata columns) only provide details about the single-cell/nuclei or spot sequencing.

However, our documentation indicates that bulk may be a possible value, which it currently is not:

scpca-docs/docs/download_files.md

Line 120 in 765c86f

| `seq_unit` | `cell` for single-cell samples, `nucleus` for single-nuclei samples, `bulk` for bulk RNA-seq samples, and `spot` for spatial transcriptomics samples. |

We should update this documentation to clarify which libraries are in this file more generally.

[jashapiro]

Are the bulk libraries listed as NA or not included? Another way to phrase this: should we fix the download file itself rather than the docs?

[sjspielman]

Are the bulk libraries listed as NA or not included?

They are not included.

Another way to phrase this: should we fix the download file itself rather than the docs?

Very possibly? Also worth noting (as I have been looking into and chatting with @allyhawkins about), we don't seem to have robust indicators in any metadata downloads for whether a library has CITE-seq. While it is visible in the interactive sample metadata tables, we have pruned this information out of the downloaded metadata - both the portal-wide and project specific. The only indicator in project-specific library metadata that there is CITE is that there TSV contains columns like adt_filtering_method etc.

Anyways, this is all a bit of a problem for using portal-downloaded metadata for scpca-paper-figures reproducibility. I am pretty sure we can still wrangle portal downloads (perhaps not the full portal download though, just the project-specific tsvs) into an expected format, but the wrangling will need some planning!

[allyhawkins]

Just noting that the line you mentioned is still correct in the docs since that metadata table refers to contents of the single_cell_metadata.tsv, the portal metadata, and the bulk_metadata.tsv. In the bulk table the seq_unit is in fact bulk.

As far as indicating which samples have bulk and which libraries have CITE-seq in the portal wide metadata (such as adding an additional_modalities column), that's a much bigger lift and something we would need to change on the Portal side. It might be best to file a feature request in scpca-portal and then we can get an idea for how much effort that change would be and make a decision to pursue it.

Anyways, this is all a bit of a problem for using portal-downloaded metadata for scpca-paper-figures reproducibility. I am pretty sure we can still wrangle portal downloads (perhaps not the full portal download though, just the project-specific tsvs) into an expected format, but the wrangling will need some planning!

If we do make that metadata change on the Portal, it probably won't be before the paper gets submitted. So I would maybe just make a CITE and bulk whitelist that gets saved alongside the sample whitelist in the repo to aid in creating the metadata files needed for making the figures.