Implement automated (regression) testing

[boulund]

As a stretch-goal for the next main version of the workflow, I'd like us to implement automated regression testing. It's not very difficult, and will improve our ability to catch small bugs, typos, etc. in pull requests and commits as early as possible.

In order to implement this in a useful way, we need at least one test case for each major path through the workflow. E.g. one gram positive, one gram negative, one with reference proteins, one which is contaminated, etc., so we can validate that all common paths through the workflow works as they should.

I'm labeling this issue as low priority and assigning it to the BACTpipe v3.0 milestone for now. There's no rush to implement this at this time.

[boulund]

Having recently containerized the BACTpipe workflow with Docker (#47), this will be much easier to implement. Any continuous integration suite (e.g. Travis CI, commonly used for Github projects) can then run the tests inside the container, making it very easy to configure and set up. But still, no rush. Better to hold off on this until all other tasks have been completed.

[thorellk]

This is something that @abhi18av also mentioned. Maybe this is the time to implement it?

[boulund]

I absolutely think so! Not sure where the nextflow field is at now, how do people normally perform automated testing of their workflows, do you know @abhi18av ?

I guess we need at least a minimal test data set that assembles and can be annotated, but using a minimum amount of resources to do so, so the test can be run relatively fast on a small vm with TravisCI/CircleCI/Github actions or whatever people use nowadays.

[abhi18av]

I do agree on this as well :)

With the new modular design of the code base, it should be easier to test at a module level as well as (sub)-workflow level. Github actions seems to be the way forward, but only after the community has figured out an optimal automated testing workflow.

This week itself, there has been major progress, with nf-core/modules#80 and nf-core/modules#85 and in a week or so it would be stabilized. Then we can see whether we can re-use that pattern or do we have to create our own :)