← All capstone topics
Industry + research LLMs at the edge

Splitting a small LLM between a smartphone and an edge server: latency- and energy-aware partitioning

Run part of a small language model on the phone and the rest on a nearby edge server, and choose the split point that gives the fastest, most energy-efficient answer for the current network.

Level: Undergraduate Format: Individual project Duration: 2 semesters Supervisor: Dr. Hamza Djigal

The problem

Phones are too weak to run a language model fully and fast, while sending every request to the cloud is slow, costly and exposes user data. A model is a stack of layers: the first layers can run on the phone and the intermediate result is sent to an edge server that finishes the work. The best split point changes with the phone's battery, the network speed and the server load.

Who needs it

Phone makers and app companies (assistants, translation, keyboard prediction) want on-device AI that stays fast and private; telecom operators sell edge servers in their 5G networks.

Idea of the approach

  • Choose a small open model (e.g. Qwen2.5-0.5B or TinyLlama) and measure the time and output size of every layer on a phone (or a Raspberry Pi as a stand-in) and on a laptop acting as the edge server.
  • Build a cost model: phone compute time + transfer time (size / bandwidth) + edge compute time, and energy on the phone.
  • Write a decision policy that picks the split point from the measured bandwidth (start with an exhaustive search, then a light learned predictor).
  • Compare with three baselines: all on the phone, all on the edge, fixed split.

What you will build and deliver

  • A working phone + edge prototype that answers prompts through the split model.
  • A measurement dataset (layer times, output sizes, bandwidth traces).
  • The split policy and an evaluation under 3G / 4G / Wi-Fi conditions.
  • A short paper and a demo video.

Skills needed

Python, PyTorch, basic networking (sockets or gRPC); Android or Raspberry Pi is a plus.

Research contribution & where to publish

Contribution: an adaptive split policy for decoder-only LLMs, where the intermediate state grows with every generated token (unlike classic CNN splitting). Target venues: IEEE INFOCOM / IEEE ICDCS workshops, IEEE Internet of Things Journal, Computer Networks.

Key references — start reading here

  1. Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, L. Tang. Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge. ASPLOS, 2017.
  2. S. Teerapittayanon, B. McDanel, H. T. Kung. Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices. IEEE ICDCS, 2017.
  3. H. Djigal, J. Xu, L. Liu, Y. Zhang. Machine and Deep Learning for Resource Allocation in Multi-Access Edge Computing: A Survey. IEEE Communications Surveys & Tutorials, 2022.
  4. J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, S. Han. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. MLSys, 2024.

Interested?

Send the topic title and the courses you have taken.

Contact Dr. Hamza Djigal

Other topics in LLMs at the edge