Y26-090 - Extend 'under-represented wells' feature to other pipelines

[SujitDey2022]

User story

As a member of the Data QC team, I would like the 'under-represented wells' feature, which was first implemented for the WGS pipeline, to be extended to more pipelines, so that I can see this information in a consistent and structured way in the MLWH, to inform my analysis.

Who are the primary contacts for this story

Katy

Who is the nominated tester for UAT

For each relevant pipeline, we would need:

• Team lead or delegated team member
• Member of Data QC team (Paul / Elliott)

Acceptance criteria

Consider if these features can be featured flagged to decouple testing and deployment.

The 'under-represented wells' feature is enabled for the following plates (in the following pipelines):

Pipeline	Plate purpose
RNA (all versions)	LB Lib PCR XP
LCMB	LCMB Lib PCR XP
Triomics	LCMT DNA PCR XP and LCMT EM PCR XP
ISC	LB Lib PCR XP
PCR Free	PF Lib XP-2

• The under-represented wells UI feature shows on the above plate purposes (regardless of pipeline)
• The under-represented information reaches the comments table in the mlwarehouse - test this carefully for each pipeline that plate purpose is used in

Dependencies

See the completed sibling issues under the parent story to see how it was done for the first pipeline.

Additional context

Turning the UI on for more plate purposes should be a simple Limber config change.

However, making sure the data gets to the MLWH might be a bit trickier. When writing the code that collects the data from Sequencescape to send to the MLWH, we took a shortcut (for the proof of concept / first pipeline), which makes some assumptions. I think this code will have to be extended / made more robust to cope with slightly different data structures in other pipelines. See https://github.com/sanger/sequencescape/pull/5229/changes#r2469045876

Deployment notes

Add any details that should be noted when deploying this feature. For example, is a database migration needed, a specifc rake task or other script that needs to be run, a dependency on another application or service that needs to be released first etc?
These notes are intended to ensure the developers releasing the feature know what needs to be done.

Add any other context or screenshots about the feature request here.

[SujitDey2022]

@KatyTaylor I had created a placeholder for you to be able to add more relevant details and acceptance criteria.

[dasunpubudumal]

Noting down the flow here:

1. When a batch is released, sequencing_pipeline.rb -> post_release_batch function is invoked.
2. post_release_batch invokes Messenger.create! with batch as the input and CommentIo as the template. CommentIo uses UnderRepWellComment class' comments function to retrieve the comments for the batch.
3. comments function invokes under_represented_well_comments that figures out the comments.

---

• When in Limber, a user clicks the under representation button, Limber invokes:

Sequencescape::Api::V2::PolyMetadatum.create!(
      key: LimberConstants::UNDER_REPRESENTED_KEY,
      value: 'true',
      relationships: { metadatable: request }
)

• Currently, Sequencescape finds poly_metadata records for each request with under_represented as the key.
• To find the requests for each batch, it uses the submissions relationship in the batch model.
• batch.submissions.flat_map(&:requests) pulls back the requests attached to the batch's submissions: the sequencing requests, but ALSO the library prep and multiplexing requests.
  • This works because the submissions in the WGS pipeline include all the requests upfront - library prep, multiplexing, and sequencing - they all share a submission.
  • For more recent, more complicated pipelines, this would not work, because there can be break points in the pipeline where new submissions must be made to continue - e.g. they just make the library prep submission up front and then subsequent requests are made by a new manual submission, or an automated one.

Notes

• A "batch" in Sequencescape is a group of requests to enable processing of a pipeline.
• batch.requests return sequencing requests for a given batch.
• batch.submissions return all the submissions related to the requests for a given batch.

Please see this discussion for more information.

But the problem here is that complex pipelines can create multiple batches.