Topology-Aware Task and Data Placement for Distributed Scientific Workflows
| Author: | Kharma, S., Wies, T., Scheuermann, B., & Schintke, F. |
| Published in: | Proceedings of the 26th IEEE International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW ‘26), 1-4 |
| Year: | 2026 |
| Type: | Academic articles |
| DOI: | 10.1109/CCGridW69005.2026.00012 |
Modern scientific data analysis workflows frequently operate across diverse, distributed infrastructure. In such environments, task placement and data transfers between them can impact overall performance significantly. While typical scheduling approaches often focus on computational resource allocation, the impact of network constrained data transfers is often ignored or simplified. In this work-in-progress paper, we propose an approach that explicitly models data transfers while considering the network topology. Integrating topology information with data size estimates and node capability insights, we develop a cost function that estimates the makespan. It considers all previously mentioned factors, for a specified task and data placement. We formulate the task of finding a good assignment as a minimization problem and outline a genetic-algorithm-based search strategy to approximate the best placement. The proposed framework aims to enable topology-aware scheduling decisions, or to refine schedules generated elsewhere, for distributed scientific workflows.
| Visit publication |

Connected HIIG researchers
Björn Scheuermann, Prof. Dr.
- Peer Reviewed
The HIIG DNA
Research and dialogue
In our research practice, scientific insight and societal dialogue are deliberately intertwined.