Upgrade to Pysam 0.14.1

.gitignore

@@ -14,6 +14,7 @@ tests/*.sam
 tests/*.fai
 tests/pysam_data
 tests/cbcf_data
+tests/tabix_data
 
 samtools/config.h
 htslib/config.status
@@ -26,20 +27,10 @@ htslib/autom4te.cache
 htslib/config.cache
 htslib/deps.mk
 
+# linking tests
+BuildRead.c
 # cython files
-pysam/TabProxies.c
-pysam/csamtools.c
-pysam/ctabix.c
-pysam/cvcf.c
-pysam/chtslib.c  
-pysam/cutils.c
-pysam/calignedsegment.c
-pysam/calignmentfile.c
-pysam/cbcf.c
-pysam/cfaidx.c
-pysam/chtslib.c
-pysam/csamfile.c
-pysam/ctabixproxies.c
+pysam/libc*.c
 
 ###### Generic python ignores below ######
 
@@ -66,6 +57,7 @@ var/
 *.egg-info/
 .installed.cfg
 *.egg
+doc/_build
 
 # Installer logs
 pip-log.txt

.travis.yml

@@ -11,6 +11,8 @@ env:
     - CONDA_PY=3.4
     - CONDA_PY=3.5
     - CONDA_PY=3.6
+  global:
+    - PYSAM_LINKING_TEST=1
 
 addons:
   apt:

INSTALL

@@ -1,57 +1,102 @@
-System Requirements
-===================
+An online version of the installation instructions can be found here:
+http://pysam.readthedocs.io/en/latest/installation.html
 
-SAMtools depends on the zlib library <http://www.zlib.net>. The latest
-version 1.2.3 is preferred and with the latest version you can compile
-razip and use it to compress a FASTA file. SAMtools' faidx is able to
-index a razip-compressed FASTA file to save diskspace. Older zlib also
-works with SAMtools, but razip cannot be compiled.
+================
+Installing pysam
+================
 
-The text-based viewer (tview) requires the GNU ncurses library
-<http://www.gnu.org/software/ncurses/>, which comes with Mac OS X and
-most of the modern Linux/Unix distributions. If you do not have this
-library installed, you can still compile the rest of SAMtools by
-manually modifying one line in Makefile.
+Pysam can be installed through conda_, pypi_ and from the repository.
+The recommended way to install pysam is through conda/bioconda.
 
-curl
+Conda installation
+==================
 
-Pysam requires Python (2.7 or greater) and Cython (0.22 or greater).
-It has not been tested on many other platforms.
+To install pysam in your current conda_ environment, type::
 
-Windows support does not work yet.
+   conda config --add channels r
+   conda config --add channels bioconda
+   conda install pysam
 
-Compilation
-===========
+This will install pysam from the bioconda_ channel and automatically
+makes sure that dependencies are installed. Also, compilation flags
+will be set automatically, which will potentially save a lot of
+trouble on OS X.
 
-Unpack the distribution and enter the pysam directory. Type 
+Pypi installation
+=================
 
-python setup.py build
+Pysam provides a python interface to the functionality contained
+within the htslib_ C library. There are two ways that these two
+can be combined, ``builtin`` and ``external``.
 
-to compile.
+Builtin
+-------
 
-Installation
-============
+The typical installation will be through pypi_::
+
+   pip install pysam
+
+This will compile the ``builtin`` htslib source code within pysam.
+
+htslib_ can be configured at compilation to turn on additional
+features such support using encrypted configurations, enable plugins,
+and more. See the htslib_ project for more information on these.
+
+Pysam will attempt to configure htslib_ to turn on some advanced
+features. If these fail, for example due to missing library
+dependencies (`libcurl`, `libcrypto`), it will fall back to
+conservative defaults.
+
+Options can be passed to the configure script explicitely by
+setting the environment variable `HTSLIB_CONFIGURE_OPTIONS`.
+For example::
 
-Type 
+  export HTSLIB_CONFIGURE_OPTIONS=--enable-plugins
+  pip install pysam
 
-   python setup.py install
+External
+--------
 
-to install it within the site-packages directory of your python
-distribution. Type
+pysam can be combined with an externally installed htslib_
+library. This is a good way to avoid duplication of libraries. To link
+against an externally installed library, set the environment variables
+`HTSLIB_LIBRARY_DIR` and `HTSLIB_INCLUDE_DIR` before installing::
 
-   python setup.py install --help 
+   export HTSLIB_LIBRARY_DIR=/usr/local/lib
+   export HTSLIB_INCLUDE_DIR=/usr/local/include
+   pip install pysam
 
-for more options.
+Note that the location of the file :file:`libhts.so` needs to be known
+to the linker once you run pysam, for example by setting the
+environment-varirable `LD_LIBRARY_PATH`.
 
-Build the documentation
-=======================
+Note that generally the pysam and htslib version need to be compatible. See
+the release notes for more information.
 
-Install a version of Sphinx that matches your Python version (2 or 3) and run
+Installation from repository
+============================
 
-    python setup.py build_sphinx
+pysam depends on cython_ to provide the connectivity to the htslib_ C
+library. The installation of the source tarball (:file:`.tar.gz`)
+contains pre-built C-files and cython needs not be present
+during installation. However, when installing from the repository,
+cython needs to be installed beforehand.
+
+To install from repository, type::
+
+    python setup.py install
+
+For compilation options, see the section on Pypi installation above.
+
+Requirements
+============
 
-or
+Depending on the installation method, requirements for building pysam differ.
 
-    python3 setup.py build_sphinx
+When installing through conda_, dependencies will be resolved by the
+package manager. The pip_ installation and installation from source
+require a C compiler and its standard libraries as well as all
+requirements for building htslib. Htslib requirements are listed in
+the htslib/INSTALL file.
 
-The documentation will be put into build/sphinx.
+Installing from the repository will require cython_ to be installed.

MANIFEST.in

@@ -5,70 +5,56 @@
 #
 include MANIFEST.in
 include COPYING
+include NEWS
 include INSTALL
 include KNOWN_BUGS
 include THANKS
 include cy_build.py
 include requirements.txt
-include pysam/c*.pxd
-include pysam/c*.pyx
-include pysam/c*.c
+include pysam/libc*.pxd
+include pysam/libc*.pyx
+include pysam/libc*.c
 include pysam/*.c
 include pysam/*.h
+
+# exclude tests from pypi tar-ball - they
+# require additional data
+prune tests/
+
+# samtools
 include samtools/configure
 include samtools/config.mk.in
 include samtools/config.h.in
 include samtools/*.h
+include samtools/*.c
 exclude samtools/config.h
 include samtools/*/*.h
+
+# bcftools
+include bcftools/*.h
+include bcftools/*.c
+exclude bcftools/config.h
+
+# htslib
 include htslib/*.c
 include htslib/*.h
+include htslib/INSTALL
+include htslib/NEWS
 exclude htslib/config.h
 include htslib/Makefile
 include htslib/htslib_vars.mk
 include htslib/configure
 include htslib/config.mk.in
 include htslib/config.h.in
+include htslib/htslib.pc.in
 include htslib/htslib/*.h
 include htslib/cram/*.c
 include htslib/cram/*.h
+include htslib/win/*.c
+include htslib/win/*.h
 include cy_build.py
 include pysam.py
 include requirements.txt
 
-# pysam tests
-include tests/00README.txt
-include tests/pysam_data
-include tests/tabix_data
-include tests/*.py
-#ex1.fa
-#include tests/ex1.sam.gz
-#include tests/ex3.sam
-#include tests/ex4.sam
-#include tests/ex5.sam
-#include tests/ex6.sam
-#include tests/ex7.sam
-#include tests/ex8.sam
-#include tests/ex9_fail.bam
-#include tests/ex9_nofail.bam
-#include tests/ex10.sam
-#include tests/example.py
-#include tests/pysam_test.py
-#include tests/segfault_tests.py
-#include tests/example_*.sam
-#include tests/example_btag.bam
-#include tests/tag_bug.bam
-#include tests/example.vcf40
-#include tests/example_empty_header.bam
-#include tests/test_unaligned.bam
-#include tests/issue100.bam
-
-# tabix tests
-#include tests/example.gtf.gz
-#include tests/example.gtf.gz.tbi
-#include tests/example.bed.gz
-#include tests/example.bed.gz.tbi
-#include tests/vcf-examples/*.vcf
-
 # documentation
 include doc/*