← Back to Garden
seedling ·
telemetry esp32 devlog iot

Setup Log

Chronological log of setting up the TSI-Telemetry system.

Day 1: Infrastructure Setup (COMPLETED)

What We Built

HiveMQ Cloud ──► Python Bridge ──► TimescaleDB (Docker) ──► Grafana

Prerequisites

  • Mac Mini (always on, home network)
  • Homebrew installed
  • Docker installed
  • UV (Python package manager)

Step 1: Tailscale (Attempted)

Installed Tailscale for remote access:

brew install tailscale
sudo tailscaled &
tailscale up

Issue: Tailscale IPs weren't routing between devices (100% packet loss).

Resolution: Decided to use cloud MQTT broker instead, which doesn't require VPN.

Step 2: Mosquitto (Local MQTT)

Installed for local testing:

brew install mosquitto
brew services start mosquitto

Configured for remote access in /opt/homebrew/etc/mosquitto/mosquitto.conf:

listener 1883 0.0.0.0
allow_anonymous true

Note: Local Mosquitto used only for testing. Production uses HiveMQ Cloud.

Step 3: TimescaleDB (Docker)

Chose TimescaleDB over InfluxDB for:

  • PostgreSQL compatibility (familiar SQL)
  • Better for complex analytics
  • Easier Grafana integration
docker run -d --name timescaledb \
  -p 5433:5432 \
  -e POSTGRES_PASSWORD=telemetry123 \
  -v timescaledb_data:/var/lib/postgresql/data \
  --restart unless-stopped \
  timescale/timescaledb:latest-pg15

Created database and table:

psql -h localhost -p 5433 -U postgres

CREATE DATABASE telemetry;
\c telemetry
CREATE EXTENSION IF NOT EXISTS timescaledb;

CREATE TABLE car_metrics (
  time           TIMESTAMPTZ NOT NULL,
  rpm            INTEGER,
  speed          INTEGER,
  coolant        INTEGER,
  intake_temp    INTEGER,
  throttle       INTEGER,
  engine_load    INTEGER,
  map_kpa        INTEGER,
  fuel_level     INTEGER,
  timing_adv     INTEGER,
  battery_voltage DECIMAL(4,2)
);

SELECT create_hypertable('car_metrics', 'time');

Step 4: HiveMQ Cloud (MQTT Broker)

Created free account at https://www.hivemq.com/cloud/

Why cloud broker?

  • ESP32 on phone hotspot can't reach home network (NAT)
  • Cloud broker is reachable from anywhere
  • Free tier: 10GB/month (plenty for telemetry)

Cluster details:

  • URL: xxxxx.s1.eu.hivemq.cloud
  • Port: 8883 (TLS)
  • Credentials created in Access Management

Step 5: Python Bridge (UV)

Created bridge script to connect HiveMQ → TimescaleDB:

mkdir -p ~/telemetry-bridge
cd ~/telemetry-bridge
uv init
uv add paho-mqtt psycopg2-binary

Created mqtt_to_timescale.py (see bridge-script.md)

Run:

uv run python mqtt_to_timescale.py

Step 6: Grafana

brew install grafana
brew services start grafana

Configured for remote access in /opt/homebrew/etc/grafana/grafana.ini:

[server]
http_addr = 0.0.0.0
http_port = 3000

Access: http://<mac-mini-ip>:3000

Added PostgreSQL data source:

  • Host: localhost:5433
  • Database: telemetry
  • User: postgres
  • Password: telemetry123
  • SSL: disabled

Step 7: Dashboard

Created gauges for all metrics:

  • RPM (0-8000)
  • Speed (0-200 km/h)
  • Coolant Temp (0-130°C)
  • Intake Temp (-20 to 80°C)
  • Throttle (0-100%)
  • Engine Load (0-100%)
  • MAP (0-250 kPa)
  • Fuel Level (0-100%)
  • Timing Advance (-10 to 50°)
  • Battery Voltage (10-16V)

Query pattern:

SELECT time, <column> as "<Label>"
FROM car_metrics
ORDER BY time DESC
LIMIT 1

Pipeline Test

Tested full pipeline:

mosquitto_pub \
  -h xxxxx.s1.eu.hivemq.cloud \
  -p 8883 \
  -u esp32 \
  -P 'password' \
  --capath /etc/ssl/certs/ \
  -t "car/telemetry" \
  -m '{"rpm":3000,"speed":100,"coolant":92,"intake_temp":38,"throttle":50,"engine_load":65,"map_kpa":105,"fuel_level":70,"timing_adv":15}'

Result: Data appeared in Grafana dashboard!

Day 1 Summary

Completed

  • [x] TimescaleDB running in Docker
  • [x] HiveMQ Cloud account and cluster
  • [x] Python bridge (MQTT → TimescaleDB)
  • [x] Grafana with PostgreSQL data source
  • [x] Dashboard with 10 gauge panels
  • [x] End-to-end pipeline tested

Services Running on Mac Mini

Service Port Command
TimescaleDB 5433 docker start timescaledb
Grafana 3000 brew services start grafana
Bridge - uv run python mqtt_to_timescale.py
Mosquitto 1883 brew services start mosquitto (optional)

Day 2: ESP32 Firmware Updates (COMPLETED)

What We Built

Added WiFi and MQTT capabilities to ESP32 firmware.

Issues Encountered & Solutions

Issue Solution
Sketch too large (132%) Use PartitionScheme=huge_app in Makefile
WiFi status code 1 Phone hotspot was on 5GHz - ESP32 only supports 2.4GHz
MQTT rc=-2 Needed to set proper timeout and verify HiveMQ credentials
"sta is connecting" error Added WiFi.disconnect(true) before reconnect

Key Learnings

  • ESP32 only supports 2.4GHz WiFi (not 5GHz!)
  • iPhone: Enable "Maximize Compatibility" for hotspot
  • Android: Manually set hotspot band to 2.4GHz
  • PubSubClient needs to be installed via arduino-cli lib install
  • Use huge_app partition when combining WiFi + BLE
ESP32 → WiFi (Phone Hotspot) → Internet → HiveMQ Cloud

Files Modified/Created

firmware/tsi/
├── Config.h          # Added WiFi + MQTT configuration
├── CarData.h         # Added batteryVoltage field (float)
├── PIDCommands.h     # Added PID_BATTERY_VOLTAGE (0x42)
├── OBDParser.cpp     # Added battery voltage parsing formula
├── DisplayManager.cpp# Added battery voltage to serial output
├── WiFiManager.h/cpp # NEW: WiFi connection handler
├── MQTTManager.h/cpp # NEW: MQTT client for HiveMQ Cloud
└── tsi.ino           # Integrated WiFi + MQTT into main loop

New Dependencies

Install PubSubClient library:

arduino-cli lib install "PubSubClient"

Configuration

Update Config.h with your credentials:

#define WIFI_SSID "YourHotspotName"
#define WIFI_PASSWORD "YourHotspotPassword"
#define MQTT_SERVER "xxxxx.s1.eu.hivemq.cloud"
#define MQTT_USER "esp32"
#define MQTT_PASSWORD "YourPassword"

Compile and Upload

cd /Users/ashishrao/Personal/TSI-Telmetry
make compile
make upload

Boot Sequence

Starting TSI Telemetry...
Initializing WiFi...
Connected! IP: 192.168.x.x
Initializing MQTT...
Connecting to MQTT broker... connected!
Initializing BLE...
Connecting to OBD2 adapter...
Ready! Reading telemetry...

Architecture After Day 2

Car ECU ←BLE→ ELM327 ←BLE→ ESP32 ←WiFi→ Phone Hotspot
                                              ↓
                                          Internet
                                              ↓
                                       HiveMQ Cloud
                                              ↓
                                       Python Bridge
                                              ↓
                                        TimescaleDB
                                              ↓
                                          Grafana

Day 3: Test Drive (PENDING)

  • [ ] Ensure bridge is running on Mac Mini
  • [ ] Start phone hotspot
  • [ ] Power on ESP32 in car
  • [ ] Connect OBD2 adapter
  • [ ] Start driving!
  • [ ] Monitor Grafana dashboard in real-time
  • [ ] Post-drive: Run analytics queries