guest/guest username/password will get you read-only access to stats.
Telemetry is provided to the stats system via a tool in the Lemongrass toolkit. The ECUSim dashboard can be found here. The metadata for the systems running the software side of the team also require a good bit of logging and monitoring to assure any semblance of reliability; as such logging, metrics, and monitoring are documented along with the engine monitoring data here. Both hosts comms and car252 run telegraf for system metrics piggy backed in the influx instance that laps and telemetry is stored in.
¶ Usage
Telem.py is a very simple script. It opens a connection with python-OBD and queries the ECU for every subscriptable value.
ELM327 clones can function, but they'll introduce errors or flake out over hours and hours of racing. Ensure your ELM327 adapter is a high quality unit. I got mine from totalcardiagnostics.com.
Testing with a desktop ECU/OBD simulator has helped build out error and value handling, but there are still hangs or crashes.
There's a big if block that throws out any value that:
- Does not have "DTC" in the name. (This means DTC Freeze Frame functionality is not implemented)
- Does not have "MIDS" in the name
- Does not have "PIDS" in the name
- Does not have "O2_SENSORS" in the name
- Does not have "ELM" in the name
- Does not have "OBD" in the name
- Is not named "STATUS"
DTC values are stored as a snapshot in time as a regular value, but prefixed with the prefix "DTC." I just get rid of them but they could be useful. MIDS and PIDS values are just taxonomic. O2_SENSORS is a pair of comma separated tuplets that let you know which O2 sensors are present in which banks. ELM, OBD, and STATUS, refer to meta information and specification compliance data.
==TODO==
Add logic for continuous retrying of initial OBD connection until successfully opened.I think these ELM adapters can detect whether or not the car is on, they're connected, etc, would be nice to have the app running in the background and seamlessly handle ignition changes.Done. (Proud of this one.)
StabilityFix crashes.Done.Catch exceptions.Done.
Correctly poll the OBD objects created by python OBD to return all supported commands, and dynamically assign watchers for each command.Done.- Build somewhere I can change a value for driver and have the metrics sent change the tag. DNS, key value store, who knows
- When another race is running, set up the lap logic to allow for collection of lap times and association with telemetry.
- Annotations for new lap?
Systemd service.Done.
¶ Successful Connection Setup
[obd.obd] ======================= python-OBD (v0.7.1) =======================
[obd.obd] Using scan_serial to select port
[obd.obd] Available ports: ['/dev/ttyUSB0']
[obd.obd] Attempting to use port: /dev/ttyUSB0
[obd.elm327] Initializing ELM327: PORT=/dev/ttyUSB0 BAUD=auto PROTOCOL=auto
[obd.elm327] Connected Successfully: PORT=/dev/ttyUSB0 BAUD=38400 PROTOCOL=6
[obd.obd] querying for supported commands
[obd.obd] Sending command: 0100: Supported PIDs [01-20]
[obd.obd] Sending command: 0120: Supported PIDs [21-40]
[obd.obd] Sending command: 0140: Supported PIDs [41-60]
[obd.obd] Sending command: 0600: Supported MIDs [01-20]
[obd.obd] Sending command: 0620: Supported MIDs [21-40]
[obd.obd] Sending command: 0640: Supported MIDs [41-60]
[obd.obd] Sending command: 0660: Supported MIDs [61-80]
[obd.obd] Sending command: 0680: Supported MIDs [81-A0]
[obd.obd] Sending command: 06A0: Supported MIDs [A1-C0]
[obd.obd] finished querying with 109 commands supported
[obd.obd] ===================================================================
[obd.asynchronous] Watching command: 0103: Fuel System Status
[obd.asynchronous] subscribing callback for command: 0103: Fuel System Status
[obd.asynchronous] Watching command: 0104: Calculated Engine Load
[obd.asynchronous] subscribing callback for command: 0104: Calculated Engine Load
[obd.asynchronous] Watching command: 0105: Engine Coolant Temperature
[obd.asynchronous] subscribing callback for command: 0105: Engine Coolant Temperature
[obd.asynchronous] Watching command: 0106: Short Term Fuel Trim - Bank 1
[obd.asynchronous] subscribing callback for command: 0106: Short Term Fuel Trim - Bank 1
[obd.asynchronous] Watching command: 0107: Long Term Fuel Trim - Bank 1
[obd.asynchronous] subscribing callback for command: 0107: Long Term Fuel Trim - Bank 1
[obd.obd] '010A: Fuel Pressure' is not supported
[obd.obd] '010B: Intake Manifold Pressure' is not supported
[obd.asynchronous] Watching command: 010C: Engine RPM
obd.logger.setLevel(obd.logging.DEBUG)
connection = obd.Async()
connection.watch(obd.commands.FUEL_STATUS, callback=new_fuel_status )
connection.watch(obd.commands.ENGINE_LOAD, callback=new_value )
connection.watch(obd.commands.COOLANT_TEMP, callback=new_value )
connection.watch(obd.commands.SHORT_FUEL_TRIM_1, callback=new_value )
connection.watch(obd.commands.LONG_FUEL_TRIM_1, callback=new_value )
connection.watch(obd.commands.FUEL_PRESSURE, callback=new_value )
connection.watch(obd.commands.INTAKE_PRESSURE, callback=new_value )
connection.watch(obd.commands.RPM, callback=new_value )
connection.watch(obd.commands.SPEED, callback=new_value )
connection.watch(obd.commands.TIMING_ADVANCE, callback=new_value )
connection.watch(obd.commands.INTAKE_TEMP, callback=new_value )
connection.watch(obd.commands.MAF, callback=new_value )
connection.watch(obd.commands.THROTTLE_POS, callback=new_value )
connection.watch(obd.commands.O2_B1S1, callback=new_value )
connection.watch(obd.commands.O2_B1S2, callback=new_value )
connection.start()
=== VoIP telemetry, process monitoring, traffic, logs ===
{{:r_d:pucker-dashboard.png?400|}}
comms telegraf.conf
[global_tags]
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
hostname = "comms"
omit_hostname = false
[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "system_metrics"
skip_database_creation = true
retention_policy = ""
username = "REDACTED"
password = "REDACTED"
[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
[[inputs.kernel]]
[[inputs.mem]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]
[[inputs.net]]
interfaces = ["eth0", "lo"]
[[inputs.net_response]]
protocol = "tcp"
address = "localhost:2222"
[[inputs.procstat]]
exe = "murmurd"
[[inputs.syslog]]
server = "tcp://0.0.0.0:6514"
car252's telegraf.conf looks similar with a few endpoint specific inputs
[[inputs.procstat]]
exe = "pucker"
[[inputs.procstat]]
exe = "pulseaudio"
[[inputs.filestat]]
files = ["/tmp/device"]
md5=true
Both client and server rsyslogs are configured with a TCP syslog endpoint on the host, telegraf. This is just the default telegraf rsyslog.conf.
pi@car252-a:~ $ cat /etc/rsyslog.d/50-telegraf.conf
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
¶ forward over tcp with octet framing according to RFC 5425
. @@(o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format