Real-Time Data Streaming

How telemetry flows from your car to the dashboard in near real-time.

The Data Pipeline

Car ECU → ESP32 → Phone → Internet → HiveMQ → Bridge → TimescaleDB → Grafana
         (poll)  (WiFi)            (MQTT)   (insert)  (store)      (display)

Message Flow (Every ~2 Seconds)

Step 1: ESP32 Polls the Car

The ESP32 sends OBD-II PID requests to the ELM327 adapter:

ESP32 sends: "010C\r"  (request RPM)
ELM327 returns: "41 0C 1A 2B"  (raw hex response)
ESP32 parses: RPM = (0x1A * 256 + 0x2B) / 4 = 1674

This repeats for all 10 PIDs (~100ms each = ~1 second total).

Step 2: ESP32 Builds JSON

After polling all PIDs, the ESP32 constructs a JSON payload:

{
  "rpm": 2500,
  "speed": 80,
  "coolant": 90,
  "intake_temp": 35,
  "throttle": 45,
  "engine_load": 60,
  "map_kpa": 101,
  "fuel_level": 75,
  "timing_adv": 12,
  "battery_voltage": 14.2
}

Size: ~150-200 bytes per message.

Step 3: MQTT Publish

ESP32 publishes to HiveMQ Cloud:

mqttClient.publish("car/telemetry", jsonPayload);

• Topic: car/telemetry
• QoS: 0 (fire and forget)
• Retain: false (don't store for late subscribers)

Step 4: Broker Routes Message

HiveMQ Cloud receives the message and delivers it to all subscribers on car/telemetry:

Publisher (ESP32) ──► HiveMQ ──► Subscriber (Bridge)
                         │
                         └──► Future: Mobile app
                         └──► Future: Alert service

Step 5: Bridge Inserts to Database

The Python bridge receives the message:

def on_message(client, userdata, msg):
    data = json.loads(msg.payload)
    cursor.execute("""
        INSERT INTO car_metrics (time, rpm, speed, ...)
        VALUES (NOW(), %s, %s, ...)
    """, (data['rpm'], data['speed'], ...))

Step 6: Grafana Displays

Grafana queries TimescaleDB every 5 seconds:

SELECT time, rpm, speed, coolant
FROM car_metrics
ORDER BY time DESC
LIMIT 1

And updates the dashboard gauges.

Latency Breakdown

This is "near real-time" - you see data within a few seconds of it happening.

Data Volume

At 1 message every 2 seconds:

• 200 bytes × 30 messages/minute = 6 KB/minute
• 6 KB × 60 minutes = 360 KB/hour
• 360 KB × 8 hours = 2.9 MB/day

HiveMQ free tier gives 10 GB/month - plenty of headroom!

Message Patterns

Current: Request-Response Polling

ESP32 ──request──► ELM327
ESP32 ◄──response── ELM327
(repeat for each PID)

Slow but reliable. Each PID takes ~100ms.

Alternative: Event-Driven (Not Used)

Some advanced OBD adapters support streaming mode where the ECU pushes changes. Our ELM327 doesn't support this.

QoS Levels Explained

We use QoS 0 because:

• Lowest latency
• If one message is lost, we get another in 2 seconds
• Telemetry is "perishable" - old readings don't matter

Scaling Considerations

Current system handles:

• 1 car
• ~0.5 messages/second
• ~10 data points/message

To scale to multiple cars:

• Use topic hierarchy: car/{car_id}/telemetry
• Bridge subscribes to car/+/telemetry (wildcard)
• Add car_id column to database

Failure Modes

The system is resilient - most failures result in temporary data gaps, not data loss.