← index
hackathon / security automation · nov 2022 – ongoing · active

Cyberattack Detection & Prevention Automation (HAVELSAN MAIN)

Real-time ML/LLM-based cyber attack detection and prevention system. 3rd place at HAVELSAN Trende Hackathon MAIN in idea & application category.

★ HAVELSAN Hackathon MAIN — 3rd Place
MLLLMSecurity AutomationReverse Engineering

Overview

An end-to-end security automation system combining machine learning and large language models for real-time cyber attack detection and prevention. Covers the full pipeline from data collection through preprocessing, model training, system integration, and testing.

System Architecture

The system implements a multi-stage data pipeline that ingests heterogeneous security telemetry and processes it through four sequential layers:

Data Ingestion Layer — Collects network flow logs via Zeek (Bro) and Suricata IDS alerts, Windows Event Log (Event ID 4625, 4688, 4648 for authentication and process creation), and syslog streams from Linux hosts and network appliances. All sources are normalised into a common schema using Apache Kafka as the transport backbone, ensuring fault-tolerant, ordered message delivery at scale.

Preprocessing & Feature Engineering Layer — Raw logs are parsed into structured time-series vectors. Statistical features (packet entropy, session duration, byte ratios), temporal features (hour-of-day, connection rate), and derived network features (TCP flag distributions, ICMP type frequencies) are extracted. A sliding-window aggregator computes 10-second and 60-second rolling statistics. Dimensionality reduction via PCA and mutual-information-based feature selection retains the top 32 most discriminative features for model consumption.

Detection Engine — An ensemble of three complementary ML models runs in parallel:

  • Random Forest Classifier — 200 estimators with max depth 16, trained on a labelled corpus of CICIDS2017 and internal SOC data (85,000 samples). Handles high-dimensional sparse feature spaces and provides built-in feature importance ranking.
  • XGBoost Classifier — Gradient-boosted trees (learning rate 0.05, max depth 8, subsample 0.8) optimised for imbalanced attack classes via scale_pos_weight tuning. Achieves strong discrimination on minority-class attacks (DDoS, brute-force, SQLi).
  • Autoencoder Anomaly Detector — A 5-layer deep autoencoder (64→32→16→32→64 units, ReLU activations) trained exclusively on benign traffic patterns. Reconstruction Mean Squared Error (MSE) serves as the anomaly score; thresholds are calibrated per-asset via 3-sigma on a validation set. Caters zero-day and polymorphic threats unseen by the supervised models.

A weighted voting mechanism fuses the three outputs: detection fires when at least two models agree or the autoencoder score exceeds its dynamic threshold. Ensemble predictions are published to a Redis stream for downstream consumption.

LLM Analysis & Response Orchestration Layer — A fine-tuned LLM (based on a 7B-parameter open-source model, instruction-tuned on security incident report corpora and MITRE ATT&CK mappings) receives each detection event as a structured JSON payload. The LLM produces:

  • A natural-language incident summary in plain English (e.g., “Possible SMB brute-force originating from 10.0.1.45 against domain controller — 1,247 failed logins in 60 seconds”)
  • A MITRE ATT&CK technique classification (T1110 — Brute Force)
  • An automated playbook recommendation drawn from a curated library of 40+ response templates
  • A suggested priority score (1–5) for SOC triage

The orchestration layer executes recommended playbooks via Ansible Tower webhooks and iptables/nftables rule insertion for immediate containment actions (IP blacklisting, port blocking). All actions are logged to an immutable audit trail.

Real-Time Dashboard

A WebSocket-based streaming pipeline pushes enriched detection events from Kafka to a Grafana dashboard frontend. Panel types include:

  • Live event timeline with severity heatmap
  • Per-asset anomaly score gauge (real-time autoencoder MSE)
  • Model agreement breakdown (ensemble consensus vs. split decisions)
  • Top-5 offending source IPs and destination ports
  • Automated action log with undo capability

The Grafana instance is embedded in a Flask-based web application that also exposes a RESTful API for manual playbook invocation and historical query.

Hackathon Results & Metrics

Evaluated during the HAVELSAN Trende Hackathon MAIN on a live-emulated environment simulating a mid-size enterprise network with mixed attack traffic:

| Metric | Value | | :--- | :--- | | Overall Detection Rate (TPR) | 96.2% | | False Positive Rate | 2.1% | | Average Detection Latency | 1.8 seconds (pipeline ingress → dashboard alert) | | Average Response Time (automated) | 4.3 seconds (detection → containment action) | | Zero-Day Detection (autoencoder) | 89% TPR at 3% FPR (on previously unseen payloads) | | LLM Incident Summary Accuracy | 92% (human-evaluated on 200 events) | | Playbook Recommendation Precision | 88% (top-1 match against SOC analyst decisions) |

The hybrid ML + LLM architecture proved particularly effective for alert fatigue reduction: the LLM summarisation layer cut per-incident analyst review time by an estimated 60% compared to raw alert triage.

Key Contributions

  • Designed data collection and preprocessing pipeline for network and system log data
  • Developed ML-based feature extraction and classification models
  • Integrated LLM-assisted threat interpretation and response suggestion engine
  • Built real-time detection dashboard with automated prevention triggers
  • Secured 3rd place in HAVELSAN Trende Hackathon MAIN (idea & application category)

Impact

Demonstrated feasibility of combining traditional ML with LLMs for security operations — bridging the gap between raw detection signals and actionable incident response. The system’s architecture has since informed subsequent research on LLM-augmented SOC automation pipelines.