How to install ?

[noirot]

After hours, I finally succeed to run SolidBin ! As documentation is ultra light and it seems to have no support on this tool I share my commands.

Note for developpers: I think conda environment should contain dependancies instead of having dependancies (hmmer, prodigal ...) in git repository.

Installation

Create solid env

git clone https://github.com/sufforest/SolidBin
cd SolidBin
conda env create -f environment.yml
source activate solidbin

Add dependencies

Note for developpers : please add in documentation export PATH=$PATH:/path to dependencies or following lines

conda install -c bioconda prodigal
conda install -c bioconda hmmer 
conda install -c bioconda pplacer

Install checkm:

cd CheckM-1.0.18
python setup.py install

Install CheckM databases:

mkdir CheckM_databases
cd CheckM_databases
wget https://data.ace.uq.edu.au/public/CheckM_databases/checkm_data_2015_01_16.tar.gz
tar xzf checkm_data_2015_01_16.tar.gz 
checkm data setRoot .

Preprocessing

Create kmer file

python SolidBin/scripts/gen_kmer.py mock.annotated.fna 1000 4

-> output is created beside the input file.
Note for developpers : It would be great to have an option --output !

Create coverage file from each bam file

extracted from gen_cov.sh should probably be optimised !

bedtools genomecov -ibam mock_A.sort.bam  mock_A_cov.txt
awk -F"\t" '{l[$1]=l[$1]+($2 *$3);r[$1]=$4} END {for (i in l){print i","(l[i]/r[i])}}' mock_A_cov.txt > mock_A_cov.csv

Create one coverage matrix file:

!! This script parse all files named *_cov.csv
perl ~/work/test_solidBin/src/SolidBin/scripts/Collate.pl . > coverage.tsv

add taxonomy info

I did not try yet ... to be completed

Run SolidBin

We are almost there ...
To run solidbin you MUST be in source directory !
cd <path/to/src>/SolidBin

Required options : --output AND --log !

python3 SolidBin.py --contig_file data/mock_A.annotated.fna --coverage_profiles data/coverage.tsv --composition_profiles data/kmer_4_f1000.csv --output out/result.tsv --log out/log.txt

What you should expect to see :

2020-09-24 17:44:51,989 - Input arguments:
2020-09-24 17:44:51,990 - contig_file:	<path>/data/mock_first.annotated.fna
2020-09-24 17:44:51,990 - coverage_profiles:	<path>/data/coverage.tsv
2020-09-24 17:44:51,990 - composition_profiles:	<path>/data/first_kmer_4_f1000.csv
2020-09-24 17:44:51,990 - output path:	test_solid/result.tsv
2020-09-24 17:44:51,990 - priori_ml_list:	Not Available
2020-09-24 17:44:51,990 - priori_cl_list:	Not Available
2020-09-24 17:44:51,990 - clusters:	Auto
2020-09-24 17:44:52,752 - Start generating affinity matrix
2020-09-24 17:44:55,646 - Finish generating affinity matrix
2020-09-24 17:44:55,646 - estimate clusters:	25
2020-09-24 17:44:55,646 - SolidBin mode:	naive
    Finished processing 25 of 25 (100.00%) bins.
    Finished processing 25 of 25 (100.00%) bins.
    Finished parsing hits for 25 of 25 (100.00%) bins.
    Finished extracting 43 of 43 (100.00%) HMMs.
    Finished aligning 43 of 43 (100.00%) marker genes.
  Finished aligning 43 of 43 (100.00%) marker genes.


    Finished parsing hits for 25 of 25 (100.00%) bins.
    Finished processing 25 of 25 (100.00%) bins (current: ori_result_2320).
    Finished processing 25 of 25 (100.00%) bins.
    Finished parsing hits for 25 of 25 (100.00%) bins.
    Finished processing 25 of 25 (100.00%) bins.
    Finished processing 25 of 25 (100.00%) bins.
    Finished parsing hits for 25 of 25 (100.00%) bins.
  Bin Id            Marker lineage   # genomes   # markers   # marker sets   0    1    2    3    4   5+   Completeness   Contamination   Strain heterogeneity  

  ori_result_2713    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_2638    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_2628    root (UID1)        5656         56            24        0    51   5    0    0   0       100.00          20.83              100.00         

  ori_result_2444    root (UID1)        5656         56            24        0    55   1    0    0   0       100.00           0.46               0.00          

  ori_result_2408    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_2380    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_2291    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_2236    root (UID1)        5656         56            24        0    55   1    0    0   0       100.00           4.17               0.00          

  ori_result_1955    root (UID1)        5656         56            24        0    51   5    0    0   0       100.00          12.50               0.00          

  ori_result_1907    root (UID1)        5656         55            23        0    25   30   0    0   0       100.00          71.74               0.00          

  ori_result_1856    root (UID1)        5656         56            24        0    56   0    0    0   0       100.00           0.00               0.00          

  ori_result_0       root (UID1)        5656         56            24        0    1    55   0    0   0       100.00          95.83               1.82          

  ori_result_2733    root (UID1)        5656         56            24        1    55   0    0    0   0       98.61            0.00               0.00          

  ori_result_2526    root (UID1)        5656         56            24        1    55   0    0    0   0       95.83            0.00               0.00          

  ori_result_2364    root (UID1)        5656         56            24        1    55   0    0    0   0       95.83            0.00               0.00          

  ori_result_2168    root (UID1)        5656         56            24        1    55   0    0    0   0       95.83            0.00               0.00          

  ori_result_2653    root (UID1)        5656         55            23        1    53   1    0    0   0       95.65            4.35               0.00          

  ori_result_2135    root (UID1)        5656         55            23        1    54   0    0    0   0       95.65            0.00               0.00          

  ori_result_2594    root (UID1)        5656         56            24        3    40   13   0    0   0       93.37           12.50               0.00          

  ori_result_2678    root (UID1)        5656         56            24        13   43   0    0    0   0       87.50            0.00               0.00          

  ori_result_2320    root (UID1)        5656         56            24        3    53   0    0    0   0       87.50            0.00               0.00          

  ori_result_1443    root (UID1)        5656         56            24        8    25   23   0    0   0       75.69           16.98              39.13          

  ori_result_11      root (UID1)        5656         56            24        31   7    5    11   1   1       75.52           131.25             38.89          

  ori_result_651     root (UID1)        5656         56            24        8    13   24   6    4   1       72.92           49.13              65.79          

  ori_result_376     root (UID1)        5656         56            24        39   15   1    1    0   0       62.50           12.50              100.00         

/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:938: FutureWarning: 'n_jobs' was deprecated in version 0.23 and will be removed in 0.25.
  warnings.warn("'n_jobs' was deprecated in version 0.23 and will be"
/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:938: FutureWarning: 'n_jobs' was deprecated in version 0.23 and will be removed in 0.25.
  warnings.warn("'n_jobs' was deprecated in version 0.23 and will be"
/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:1035: RuntimeWarning: Explicit initial center position passed: performing only one init in KMeans instead of n_init=10.
  self._check_params(X)
/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:938: FutureWarning: 'n_jobs' was deprecated in version 0.23 and will be removed in 0.25.
  warnings.warn("'n_jobs' was deprecated in version 0.23 and will be"
/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:938: FutureWarning: 'n_jobs' was deprecated in version 0.23 and will be removed in 0.25.
  warnings.warn("'n_jobs' was deprecated in version 0.23 and will be"
/home/cnoirot/.conda/envs/solidbin/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:1035: RuntimeWarning: Explicit initial center position passed: performing only one init in KMeans instead of n_init=10.
  self._check_params(X)

Still have strange warnings !

[ziyewang]

Thanks for the documentation. I have uploaded some test files and updated the description for generating the input files and installation methods in Readme. And the warnings are mainly caused by the different sklearn package versions, and they will not influence the performance.

Best,
Ziye

[pbravakos]

Hi,
I am trying to replicate your analysis but I do not understand how and why you get multiple files named *_cov.csv.
I have only one bam file because I mapped the pooled reads to the contigs fasta file that was created by the assembler. Should I instead map each sample separately to the contigs fasta file and repeat the analysis?
Moreover, I used genomeCoverageBed instead of "bedtools genomecov" which I assume is the same. I get though an extra contig named "genome" which I have to filter out. I don't know if this is an issue in your analysis.
Thanks

[ziyewang]

Hi, I get multiple files named *_cov.csv because I have multiple read samples, and the contig file is produced by the assembler through co-assembly of the multiple sequencing samples. If you have multiple samples and the contig fasta file is the assembly result of multiple read samples, you should have multiple bam files.
The second question is not an issue in my analysis. If you are familar with other binners, such as MetaWRAP, CONCOCT and COCACOLA, their coverage file can be used as our input directly or through some small transformations.

Thanks,
Ziye