High throughput Human Variant Calling Workflow on BlueWaters. If whole genome sequencing and analysis become part of the standard of care in many hospitals within the next few years, then human genetic variant calling will need to be performed on hundreds of incoming patients on any given day. At this scale, the standard workflow widely accepted in the research and medical community, will use thousands of nodes at a time and have i/o bottlenecks that could affect performance even on a major cluster like BlueWaters. In this presentation, we will discuss the kinds of computational bottlenecks that can be expected, as well as the tools and methods to overcome them. Specifically, we will cover the bottlenecks associated with the large number of small files created by the workflow, saturated i/o bandwidth for parts of the workflow, and unbalanced data load on the file system.
