← All capstone topics
Research project LLMs at the edge

Prompt compression for LLM offloading over low-bandwidth networks

Shorten long prompts on the device before sending them to the edge or the cloud, so that answers come back faster and cheaper with the same quality.

Level: Undergraduate Format: Individual project Duration: 1–2 semesters Supervisor: Dr. Hamza Djigal

The problem

Long prompts (documents, chat history) are slow to upload on weak networks and expensive to process. Prompt-compression methods drop unimportant tokens, but they are judged on quality alone; on a mobile link the real question is the end-to-end time and cost.

Who needs it

Mobile apps that summarize or answer about documents, field workers on weak networks, any company paying per token.

Idea of the approach

  • Use two compression methods (e.g. LLMLingua-2 and a simple keyword/TF-IDF baseline).
  • Measure compression time on the device, upload time under several bandwidths, and answer quality on public QA/summarization sets.
  • Propose a rule that chooses the compression rate from the measured bandwidth.
  • Relate to soft-prompt compression for relation extraction (GAPrompt).

What you will build and deliver

  • Benchmark scripts and results.
  • The adaptive compression rule.
  • A paper.

Skills needed

Python, NLP basics, Hugging Face Transformers.

Research contribution & where to publish

Contribution: bandwidth-adaptive compression with an end-to-end latency model. Venues: Findings of EMNLP/ACL, IEEE ICC, Neurocomputing, Knowledge-Based Systems.

Key references — start reading here

  1. H. Jiang, Q. Wu, C.-Y. Lin, Y. Yang, L. Qiu. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. EMNLP, 2023.
  2. Z. Pan, Q. Wu, H. Jiang, M. Xia, X. Luo, J. Zhang, et al.. LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression. Findings of ACL, 2024.
  3. GAPrompt authors (H. Djigal et al.). GAPrompt: A soft prompt compression model for few-shot relation extraction. Knowledge-Based Systems, 2025.
  4. H. Djigal, J. Xu, L. Liu, Y. Zhang. Machine and Deep Learning for Resource Allocation in Multi-Access Edge Computing: A Survey. IEEE Communications Surveys & Tutorials, 2022.

Interested?

Send the topic title and the courses you have taken.

Contact Dr. Hamza Djigal

Other topics in LLMs at the edge