Skip to content

Bottle Mod: Modeling Data Flows and Tasks for Fast Bottleneck Analysis

Author: Lösser, A., Witzke, J., Schintke, F., & Scheuermann, B.
Published in: 2022 IEEE International Conference on Big Data, Poster Track.
Year: 2022
Type: Academic articles
DOI: 10.48550/arXiv.2209.05358

In the recent years, scientific workflows gained more and more popularity. In scientific workflows, tasks are typically treated as black boxes. Dealing with their complex interrelations to identify optimization potentials and bottlenecks is therefore inherently hard. The progress of a scientific workflow depends on several factors, including the available input data, the available computational power, and the I/O and network bandwidth. Here, we tackle the problem of predicting the workflow progress with very low overhead. To this end, we look at suitable formalizations for the key parameters and their interactions which are sufficiently flexible to describe the input data consumption, the computational effort and the output production of the workflow's tasks. At the same time they allow for computationally simple and fast performance predictions, including a bottleneck analysis over the workflow runtime. A piecewise-defined bottleneck function is derived from the discrete intersections of the task models' limiting functions. This allows to estimate potential performance gains from overcoming the bottlenecks and can be used as a basis for optimized resource allocation and workflow execution.

Visit publication

Publication

Connected HIIG researchers

Björn Scheuermann, Prof. Dr.

Associated Research Director


  • Open Access

Explore current HIIG Activities

Research issues in focus

HIIG is currently working on exciting topics. Learn more about our interdisciplinary pioneering work in public discourse.