The problem
Shared GPU servers are usually managed first-come-first-served or by hand: short jobs wait behind long ones, some users monopolise the cards, and GPUs stay idle at night. Cluster schedulers for deep learning (Gandiva, Tiresias) showed large gains in big companies; small university or company servers have few GPUs, no job-length information and users who want fairness.
Who needs it
Universities, AI start-ups and company R&D teams that own a few GPU servers instead of renting cloud GPUs.
Idea of the approach
- Collect (anonymised) job logs of a shared GPU server, or generate realistic ones from public traces.
- Implement 3 policies in a simulator: first-come-first-served, least-attained-service (as in Tiresias), and a fairness-aware policy with GPU sharing.
- Measure average completion time, waiting time per user, GPU utilisation and fairness.
- Optionally deploy the best policy with Slurm or a small web queue for a real lab server.
What you will build and deliver
- Simulator, policies and results.
- A small job-queue web tool for a real GPU server (optional).
- A paper.
Skills needed
Python, operating-system scheduling (CPS 3250), basic Linux administration.
Research contribution & where to publish
Contribution: scheduling for small shared GPU servers without job-length knowledge, with fairness. Venues: ICPP, Euro-Par, IEEE CLUSTER, Journal of Parallel and Distributed Computing.
Key references — start reading here
- W. Xiao, R. Bhardwaj, R. Ramjee, M. Sivathanu, N. Kwatra, Z. Han, et al.. Gandiva: Introspective Cluster Scheduling for Deep Learning. USENIX OSDI, 2018.
- J. Gu, M. Chowdhury, K. G. Shin, Y. Zhu, M. Jeon, J. Qian, H. Liu, C. Guo. Tiresias: A GPU Cluster Manager for Distributed Deep Learning. USENIX NSDI, 2019.
- H. Topcuoglu, S. Hariri, M.-Y. Wu. Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing. IEEE Transactions on Parallel and Distributed Systems, 2002.
Interested?
Send the topic title and the courses you have taken.
Contact Dr. Hamza DjigalOther topics in Edge and cloud scheduling
Budget- and deadline-aware autoscaling of Kubernetes microservices with reinforcement learning
Carbon-aware scheduling of data-pipeline workflows across cloud regions
Spot-instance-aware workflow scheduling to cut cloud costs for data pipelines
Predicting serverless cold starts and pre-warming functions with machine learning