Skip to content

Topology-Aware Task and Data Placement for Distributed Scientific Workflows

Author: Kharma, S., Wies, T., Scheuermann, B., & Schintke, F.
Published in: Proceedings of the 26th IEEE International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW ‘26), 1-4
Year: 2026
Type: Academic articles
DOI: 10.1109/CCGridW69005.2026.00012

Modern scientific data analysis workflows frequently operate across diverse, distributed infrastructure. In such environments, task placement and data transfers between them can impact overall performance significantly. While typical scheduling approaches often focus on computational resource allocation, the impact of network constrained data transfers is often ignored or simplified. In this work-in-progress paper, we propose an approach that explicitly models data transfers while considering the network topology. Integrating topology information with data size estimates and node capability insights, we develop a cost function that estimates the makespan. It considers all previously mentioned factors, for a specified task and data placement. We formulate the task of finding a good assignment as a minimization problem and outline a genetic-algorithm-based search strategy to approximate the best placement. The proposed framework aims to enable topology-aware scheduling decisions, or to refine schedules generated elsewhere, for distributed scientific workflows.

Visit publication

Publication

HIIG Forscher*in

Björn Scheuermann, Prof. Dr.

Assoziierter Forschungsdirektor


  • Peer Reviewed

Die HIIG DNA

Forschung und Dialog

Wir stellen an unsere Forschungspraxis den Anspruch, wissenschaftliche Erkenntnis und gesellschaftlichen Dialog zu verschränken.