Update README with documentation of how to use

[jackmurphy2351]

We need to explain clearly to potential future users how to use this repo on their own machine. This should include:

• A quick start script (Nishat said she can write this up)
• A description of how Docker works to ensure that the user is working in the same environment we were in when we participated in this hackathon. @cdsoria , Please add some bullet points documenting this and I (@jackmurphy2351 ) can write it up.

[fbazzi19]

If you need session infos from us, here's mine:

R version 4.4.0 (2024-04-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26100)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] broom_1.0.7       epitools_0.5-10.1 ggsurvfit_1.1.0   survminer_0.5.0  
[5] ggpubr_0.6.0      ggplot2_3.5.1     readxl_1.4.3      dplyr_1.1.4      

loaded via a namespace (and not attached):
 [1] utf8_1.2.4        generics_0.1.3    tidyr_1.3.1       rstatix_0.7.2    
 [5] lattice_0.22-6    digest_0.6.37     magrittr_2.0.3    evaluate_1.0.1   
 [9] grid_4.4.0        fastmap_1.2.0     cellranger_1.1.0  Matrix_1.7-0     
[13] backports_1.5.0   Formula_1.2-5     survival_3.7-0    gridExtra_2.3    
[17] purrr_1.0.2       fansi_1.0.6       scales_1.3.0      abind_1.4-8      
[21] cli_3.6.3         KMsurv_0.1-5      rlang_1.1.4       munsell_0.5.1    
[25] splines_4.4.0     yaml_2.3.10       withr_3.0.2       tools_4.4.0      
[29] ggsignif_0.6.4    colorspace_2.1-1  km.ci_0.5-6       vctrs_0.6.5      
[33] R6_2.5.1          zoo_1.8-12        lifecycle_1.0.4   car_3.1-3        
[37] pkgconfig_2.0.3   pillar_1.9.0      gtable_0.3.6      data.table_1.16.2
[41] glue_1.8.0        xfun_0.49         tibble_3.2.1      tidyselect_1.2.1 
[45] rstudioapi_0.17.1 knitr_1.49        xtable_1.8-4      farver_2.1.2     
[49] survMisc_0.5.6    htmltools_0.5.8.1 patchwork_1.3.0   rmarkdown_2.29   
[53] carData_3.0-5     labeling_0.4.3    compiler_4.4.0   

Most of us have been using R

[karaquaid]

Here's all of my sessionInfo from my R work, in case you need it:

R version 4.5.1 (2025-06-13)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] pheatmap_1.0.13 lubridate_1.9.4 forcats_1.0.1   stringr_1.5.2   dplyr_1.1.4    
 [6] purrr_1.1.0     readr_2.1.5     tidyr_1.3.1     tibble_3.3.0    ggplot2_4.0.0  
[11] tidyverse_2.0.0

[cdiaz45]

Quick Start

Environments

• scripts/conda_environment.yml – minimal toolset.
• scripts/conda_environment_full.yml – pinned, exact replica.

Create (pick one):

# full (recommended)
conda env create -f scripts/conda_environment_full.yml && conda activate base
# or minimal
conda env create -f scripts/conda_environment.yml && conda activate base

Verify:

bedtools --version   # ~2.26.0
bcftools --version   # ~1.20
htslib --version     # ~1.20
R --version          # 4.3.x

Data (GRCh37)

# ClinVar VCF
wget https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37/clinvar.vcf.gz
wget https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37/clinvar.vcf.gz.tbi

# MC3 PanCanAtlas MAF
wget --content-disposition https://api.gdc.cancer.gov/data/1c8cfe5f-e52d-41ba-94da-f15ea1337efc
mv 1c8cfe5f-e52d-41ba-94da-f15ea1337efc mc3.v0.2.8.PUBLIC.maf.gz

Note: VCF is 1-based; BED/MAF use 0-based start. Scripts handle this.

---

Pipeline (what I ran)

1. MAF → BED (MC3 context)

bash scripts/1.generate_mc3_bed.sh mc3.v0.2.8.PUBLIC.maf.gz mc3_for_intersect.bed

2. Position-only intersect (keep all ClinVar VCF cols)

bash scripts/2.clinvar_pos_in_mc3_allvcf.sh clinvar.vcf.gz mc3_for_intersect.bed /tmp
cp /tmp/clinvar_pos_in_mc3.with_header.tsv.gz .

3. Filter to Pathogenic/Likely_pathogenic (+ rsID/allele logic)

bash scripts/4.filter_by_rsid_pathogenic.sh   clinvar_pos_in_mc3.with_header.tsv.gz   clinvar_pos_in_mc3.with_header_filtered.tsv

4. CRC subset (MONDO:0005575 or CLNDN CRC synonyms)

bash scripts/3.filter_crc_from_clinvar_mc3.sh   clinvar_pos_in_mc3.with_header_filtered.tsv   filtered_crc_hits_by_condition.tsv

5. Annotate allele match (exact vs position-only)

bash scripts/5.annotate_allele_match.sh   clinvar_pos_in_mc3.with_header_filtered.tsv   clinvar_pos_in_mc3.with_header_filtered.annot.tsv

bash scripts/5.annotate_allele_match.sh   filtered_crc_hits_by_condition.tsv   filtered_crc_hits_by_condition.annot.tsv

---

Outputs

• mc3_for_intersect.bed
• clinvar_pos_in_mc3.with_header.tsv.gz (~404.5 MB; main join)
• clinvar_pos_in_mc3.with_header_filtered.tsv (Pathogenic/Likely subset; ~26.1 MB)
• clinvar_pos_in_mc3.with_header_filtered.annot.tsv (+ AlleleMatch; ~26.3 MB)
• filtered_crc_hits_by_condition.tsv (CRC subset)
• filtered_crc_hits_by_condition.annot.tsv (CRC subset + AlleleMatch; ~4.0 MB)
• crc_hits_by_condition_MONDO-0005575.tsv (CRC subset by MONDO; ~9.3 MB)

---

Quick checks

zcat clinvar_pos_in_mc3.with_header.tsv.gz | head
wc -l clinvar_pos_in_mc3.with_header_filtered.tsv
wc -l filtered_crc_hits_by_condition.tsv

EC2 notes

• Amazon Linux + Conda; everything (bcftools/bedtools/htslib/tabix) from the env.
• Keep ~10–15 GB free; use /tmp for large intermediates.