← All capstone topics
Industry + research Edge and cloud scheduling

Scheduling deep-learning training jobs on a shared university GPU server

Many students and researchers share a few GPUs: decide who runs when, so that jobs finish sooner, nobody waits forever and the GPUs never sit idle.

Level: Undergraduate Format: Individual project Duration: 1–2 semesters Supervisor: Dr. Hamza Djigal

The problem

Shared GPU servers are usually managed first-come-first-served or by hand: short jobs wait behind long ones, some users monopolise the cards, and GPUs stay idle at night. Cluster schedulers for deep learning (Gandiva, Tiresias) showed large gains in big companies; small university or company servers have few GPUs, no job-length information and users who want fairness.

Who needs it

Universities, AI start-ups and company R&D teams that own a few GPU servers instead of renting cloud GPUs.

Idea of the approach

  • Collect (anonymised) job logs of a shared GPU server, or generate realistic ones from public traces.
  • Implement 3 policies in a simulator: first-come-first-served, least-attained-service (as in Tiresias), and a fairness-aware policy with GPU sharing.
  • Measure average completion time, waiting time per user, GPU utilisation and fairness.
  • Optionally deploy the best policy with Slurm or a small web queue for a real lab server.

What you will build and deliver

  • Simulator, policies and results.
  • A small job-queue web tool for a real GPU server (optional).
  • A paper.

Skills needed

Python, operating-system scheduling (CPS 3250), basic Linux administration.

Research contribution & where to publish

Contribution: scheduling for small shared GPU servers without job-length knowledge, with fairness. Venues: ICPP, Euro-Par, IEEE CLUSTER, Journal of Parallel and Distributed Computing.

Key references — start reading here

  1. W. Xiao, R. Bhardwaj, R. Ramjee, M. Sivathanu, N. Kwatra, Z. Han, et al.. Gandiva: Introspective Cluster Scheduling for Deep Learning. USENIX OSDI, 2018.
  2. J. Gu, M. Chowdhury, K. G. Shin, Y. Zhu, M. Jeon, J. Qian, H. Liu, C. Guo. Tiresias: A GPU Cluster Manager for Distributed Deep Learning. USENIX NSDI, 2019.
  3. H. Topcuoglu, S. Hariri, M.-Y. Wu. Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing. IEEE Transactions on Parallel and Distributed Systems, 2002.

Interested?

Send the topic title and the courses you have taken.

Contact Dr. Hamza Djigal

Other topics in Edge and cloud scheduling