The problem
Small companies produce millions of log lines but have no security team. Classic detectors give alerts without explanations; large language models can explain but are slow and may hallucinate.
Who needs it
Small and medium enterprises, managed IT service providers, university IT departments.
Idea of the approach
- Use Loghub datasets (HDFS, BGL, OpenSSH) with labelled anomalies.
- Parse logs (Drain), train a fast detector (DeepLog-style LSTM or LogBERT).
- Send only flagged windows to a small local LLM that explains the event and suggests actions; check explanations against the labels.
- Web dashboard for alerts.
What you will build and deliver
- Detection + explanation pipeline and dashboard.
- Evaluation of detection and explanation quality.
- Paper.
Skills needed
Python, Linux logs, machine learning basics.
Research contribution & where to publish
Contribution: a two-stage detector + LLM explainer evaluated for accuracy and cost. Venues: IEEE ISSRE, ACSAC (industry), Computers & Security, Journal of Systems and Software.
Key references — start reading here
- M. Du, F. Li, G. Zheng, V. Srikumar. DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning. ACM CCS, 2017.
- P. He, J. Zhu, Z. Zheng, M. R. Lyu. Drain: An Online Log Parsing Approach with Fixed Depth Tree. IEEE ICWS, 2017.
- J. Zhu, S. He, P. He, J. Liu, M. R. Lyu. Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics. IEEE ISSRE, 2023.
- H. Guo, S. Yuan, X. Wu. LogBERT: Log Anomaly Detection via BERT. IJCNN, 2021.
Interested?
Send the topic title and the courses you have taken.
Contact Dr. Hamza Djigal