MQTT Attack Log Generator for IoT-SIEM Research

A comprehensive system for generating realistic, labeled MQTT attack logs from the client perspective for IoT-SIEM and log anomaly detection research. This tool addresses the gap between existing network-level attack datasets (PCAPs) and application-level syslogs needed for SIEM analysis.

Overview

This system implements the Hybrid Approach for MQTT log generation, combining:

1. Instrumented Victim Clients - MQTT clients with built-in attack detection and syslog generation
2. Attack Simulators - Multiple attack scenario generators (DoS, DDoS, Malformed Packets, etc.)
3. Log Collector - Centralized log aggregation in multiple formats (JSON, CSV, syslog)
4. Orchestrator - Automated experiment coordination with timeline labeling

What This Generates

Unlike existing tools that produce PCAPs or broker-side logs, this system generates:

• ✅ Application-level syslogs from client perspective
• ✅ Attack detection indicators (connection failures, resource exhaustion, malformed packets)
• ✅ Labeled datasets with precise attack timelines
• ✅ Multiple formats (syslog, JSON, CSV) for SIEM ingestion
• ✅ Normal baseline traffic for comparison

Architecture

┌─────────────────────────────────────────┐
│  Attack Simulator                       │
│  - DoS/DDoS attacks                     │
│  - Malformed packets                    │
│  - Topic enumeration                    │
│  - Authentication attacks               │
└──────────────┬──────────────────────────┘
               │
               ↓ (attacks)
┌─────────────────────────────────────────┐
│  Victim MQTT Clients (Instrumented)     │
│  - Detect attack patterns               │
│  - Generate syslogs                     │
│  - Publish logs to MQTT topics          │
└──────────────┬──────────────────────────┘
               │
               ↓
┌─────────────────────────────────────────┐
│  MQTT Broker (Mosquitto)                │
│  Topics: syslog/#, logs/#               │
└──────────────┬──────────────────────────┘
               │
               ↓
┌─────────────────────────────────────────┐
│  Log Collector                          │
│  - Aggregates all logs                  │
│  - Outputs: JSON, CSV, syslog           │
│  - Generates statistics                 │
└─────────────────────────────────────────┘

Features

Attack Detection Capabilities

The instrumented clients detect and log:

• DoS Attacks

  • Rapid connection failures
  • CPU/Memory exhaustion
  • Message flooding (>50 msg/s)

• Malformed Packets

  • Oversized payloads (>1MB)
  • Non-ASCII topics
  • Null bytes in payloads
  • Malformed JSON
  • Deep topic hierarchies (>10 levels)

• Topic Enumeration

  • Excessive subscriptions (>50 topics)
  • Wildcard abuse

• Resource Exhaustion

  • CPU usage monitoring
  • Memory usage monitoring
  • Baseline deviation detection

Attack Scenarios

1. DoS Connection Flood - Rapid connect/disconnect to exhaust broker resources
2. DoS Message Flood - High-rate message publishing to overwhelm subscribers
3. DoS Large Payload - Sending multi-MB messages to exhaust bandwidth/memory
4. DDoS Distributed Flood - Multiple attackers simultaneously flooding
5. Malformed Packet Injection - Crafted packets with unusual characteristics
6. Topic Enumeration - Reconnaissance via wildcard subscriptions
7. Slowloris Attack - Slow connections to exhaust connection pool
8. Authentication Brute Force - Password guessing attacks

Installation

Prerequisites

• Python 3.7 or higher
• Linux system (tested on Ubuntu 20.04+)
• MQTT broker (Mosquitto recommended)

Quick Setup

# Clone or navigate to the directory
cd /home/kevin/LogBERTVADE/Mqtt_generator

# Run setup script (installs dependencies and Mosquitto if needed)
./setup.sh

# Activate virtual environment
source venv/bin/activate

Manual Setup

# Install Mosquitto broker
sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients
sudo systemctl enable mosquitto
sudo systemctl start mosquitto

# Create Python virtual environment
python3 -m venv venv
source venv/bin/activate

# Install Python dependencies
pip install -r requirements.txt

Usage

Option 1: Full Automated Experiment (Recommended)

Run the orchestrator to execute all attack scenarios with automatic labeling:

source venv/bin/activate
python orchestrator.py

This will:

1. Start the log collector
2. Deploy victim MQTT clients
3. Generate normal baseline traffic
4. Execute all enabled attack scenarios in sequence
5. Generate labeled dataset with attack timelines
6. Save logs in multiple formats

Output location: /tmp/mqtt_logs/

Files generated:

• mqtt_logs_TIMESTAMP.json - All logs in JSON format
• mqtt_logs_TIMESTAMP.csv - Tabular format for analysis
• mqtt_logs_TIMESTAMP.syslog - Standard syslog format
• mqtt_logs_{severity}_TIMESTAMP.json - Severity-specific logs
• attack_labels.json - Attack timeline and labels

Option 2: Individual Components

Run Victim Client Only

python mqtt_client_logger.py

Run Specific Attack

# DoS message flood
python attack_simulator.py --attack dos_message --topic test/target --duration 60

# DDoS distributed attack
python attack_simulator.py --attack ddos --topic test/target --duration 120

# All attacks in sequence
python attack_simulator.py --attack all

Run Log Collector Only

python log_collector.py --broker localhost --port 1883 --output /tmp/mqtt_logs

Option 3: Custom Configuration

Edit config.yaml to customize:

# Enable/disable specific attacks
attack_scenarios:
  dos_connection_flood:
    enabled: true
    duration: 60
    connection_rate: 10

# Adjust victim client count
victim_clients:
  count: 5

# Change experiment parameters
experiment:
  duration: 600
  include_normal_traffic: true

Then run:

python orchestrator.py --config config.yaml

Log Format Examples

Syslog Format

Jan 15 10:23:45 iot_device_001 mqtt_client: ALERT - DoS attack detected - rapid connection failures | {"type": "DoS", "indicator": "rapid_connection_failures", "failures": 7, "time_window": 8.3}
Jan 15 10:23:46 iot_device_002 mqtt_client: ALERT - Malformed or suspicious MQTT packet received | {"type": "MalformedPacket", "indicators": ["oversized_payload"], "topic": "test/target", "payload_size": 1048577}

JSON Format

{
  "timestamp": "2026-01-15T10:23:45.123456",
  "client_id": "iot_device_001",
  "severity": "ALERT",
  "message": "Message flooding attack detected",
  "broker": "localhost:1883",
  "attack_details": {
    "type": "Flooding",
    "indicator": "high_message_rate",
    "rate_per_second": 127.5,
    "threshold": 50
  }
}

CSV Format

timestamp,client_id,severity,message,broker,attack_type,attack_details
2026-01-15T10:23:45.123456,iot_device_001,ALERT,DoS attack detected,localhost:1883,DoS,"{""type"": ""DoS"", ""indicator"": ""cpu_exhaustion""}"

Log Analysis

Built-in Analysis Tool

# Analyze collected logs
python log_collector.py --analyze /tmp/mqtt_logs/mqtt_logs_20260115_102345.json

Output includes:

• Total log count
• Severity distribution
• Attack type distribution
• Client distribution
• Time-based statistics
• Average log rate

Using with SIEM Tools

The generated logs can be ingested into:

• ELK Stack (Elasticsearch, Logstash, Kibana)
• Splunk
• Graylog
• Wazuh
• AlienVault OSSIM

Example Logstash configuration:

input {
  file {
    path => "/tmp/mqtt_logs/*.syslog"
    type => "mqtt_syslog"
  }
}

filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:client_id} %{SYSLOGPROG:program}: %{LOGLEVEL:severity} - %{GREEDYDATA:log_message}" }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "mqtt-logs-%{+YYYY.MM.dd}"
  }
}

Dataset Statistics

A typical full experiment generates:

• Duration: ~10-15 minutes
• Log count: 1,000 - 10,000+ logs
• Attack logs: 30-50% of total
• Normal logs: 50-70% of total
• Unique attack types: 8 scenarios
• File size:
  • JSON: 1-10 MB
  • CSV: 500 KB - 5 MB
  • Syslog: 500 KB - 5 MB

Attack Label Format

The attack_labels.json file provides precise attack timelines:

{
  "experiment_name": "mqtt_attack_dataset_generation",
  "start_time": "2026-01-15T10:00:00.000000",
  "total_duration": 645.3,
  "attack_labels": [
    {
      "attack_name": "normal_traffic",
      "start_time": "2026-01-15T10:00:00.000000",
      "start_timestamp": 0.0,
      "duration": 60,
      "details": {
        "type": "normal",
        "description": "Baseline normal MQTT traffic"
      }
    },
    {
      "attack_name": "dos_connection_flood",
      "start_time": "2026-01-15T10:01:00.123456",
      "start_timestamp": 60.123,
      "duration": 60,
      "details": {
        "enabled": true,
        "duration": 60,
        "connection_rate": 10
      }
    }
  ]
}

Research Applications

This tool is designed for:

1. IoT-SIEM Development - Training and testing SIEM systems on IoT-specific attacks
2. Log Anomaly Detection - Training ML models (LogBERT, DeepLog, etc.) on labeled attack logs
3. Attack Pattern Analysis - Studying client-side manifestations of network attacks
4. Intrusion Detection Systems - Developing signature-based or anomaly-based IDS
5. Dataset Generation - Creating reproducible, labeled datasets for research

Comparison with Existing Datasets

Dataset/Tool	Output Type	Perspective	Labels	Client Logs
IoT-Flock	PCAP	Network	No	❌
MQTTset	PCAP + CSV	Network	Yes	❌
MQTT-IoT-IDS2020	PCAP	Network	Yes	❌
Broker Logs	JSON audit	Broker	Partial	❌
This Tool	Syslog/JSON/CSV	Client	Yes	✅

Troubleshooting

Broker Connection Issues

# Check if broker is running
systemctl status mosquitto

# Test connectivity
mosquitto_sub -h localhost -t test &
mosquitto_pub -h localhost -t test -m "test"

# Check broker with orchestrator
python orchestrator.py --check-broker

Permission Errors

# Ensure output directory is writable
mkdir -p /tmp/mqtt_logs
chmod 777 /tmp/mqtt_logs

# Or change output directory in config.yaml

High CPU/Memory Usage

This is expected during attack scenarios (especially DDoS). To reduce:

1. Decrease num_attackers in DDoS scenarios
2. Reduce messages_per_second in flood attacks
3. Lower victim_clients.count

No Logs Collected

1. Verify broker is running
2. Check victim clients connected successfully
3. Ensure log collector subscribed to correct topics
4. Check /tmp/mqtt_client_*.log for client-side logs

Advanced Configuration

Custom Attack Scenarios

Create custom attack scripts:

from attack_simulator import MQTTAttackSimulator

simulator = MQTTAttackSimulator("localhost", 1883)

# Custom attack logic
def custom_attack():
    client = simulator._create_client("custom_attacker")
    client.connect("localhost", 1883)
    # Your attack logic here

custom_attack()

Integration with IoT-Flock

You can run IoT-Flock attacks alongside this system:

1. Run victim clients with this tool
2. Run IoT-Flock attacks against the broker
3. Victim clients will detect and log the attacks
4. Collect logs with log collector

Extending Attack Detection

Add custom detection logic to mqtt_client_logger.py:

def detect_custom_attack(self) -> Optional[Dict[str, Any]]:
    # Your detection logic
    if suspicious_condition:
        return {
            "type": "CustomAttack",
            "indicator": "custom_indicator",
            "details": {...}
        }
    return None

Performance Tuning

For Large-Scale Experiments

# config.yaml
victim_clients:
  count: 100  # More victims

attack_scenarios:
  ddos_distributed:
    num_attackers: 200  # More attackers

For Resource-Constrained Systems

victim_clients:
  count: 2

attack_scenarios:
  dos_message_flood:
    messages_per_second: 20  # Lower rate
  
  ddos_distributed:
    num_attackers: 10  # Fewer attackers

Citation

If you use this tool in your research, please cite:

@software{mqtt_log_generator,
  title = {MQTT Attack Log Generator for IoT-SIEM Research},
  author = {Your Name},
  year = {2026},
  description = {Application-level MQTT attack log generation from client perspective}
}

Related Work

• IoT-Flock: Network-level IoT attack traffic generation
• MQTTset: MQTT attack dataset (PCAP format)
• MQTT-IoT-IDS2020: Labeled MQTT intrusion dataset
• CICFlowmeter: Network flow feature extraction
• LogBERT: Log anomaly detection using BERT

Contributing

Contributions welcome! Areas for enhancement:

• Additional attack scenarios (replay attacks, certificate attacks)
• More sophisticated detection algorithms
• Integration with additional SIEM platforms
• Performance optimizations
• Additional output formats

License

MIT License - Free for research and educational use

Support

For issues, questions, or collaboration:

• Open an issue on GitHub
• Contact: [your email]

Acknowledgments

This tool was developed to address the gap in IoT-SIEM research datasets, specifically the lack of application-level client logs for MQTT attack scenarios. It complements existing network-level datasets by providing the client perspective necessary for comprehensive SIEM analysis.