tools: Add a TML trace decoder for temperature model debugging

Add `tml_decode` to decode a TML trace from a serial log file into a
parsable tab-separated table.

`tml_decode` also doubles a simple no-frills plotting utility.

Restructure the README for better readability.

tools/README.md

@@ -1,22 +1,31 @@
 # Host debugging tools for Prusa MK3 firmware
 
-## Tools
+Most of the tools require python 3 and assume an Unix environment.
+
+
+## EEPROM analysis
 
 ### ``dump_eeprom``
 
 Dump the content of the entire EEPROM using the D3 command.
 Requires ``printcore`` from [Pronterface].
 
+### ``update_eeprom``
+
+Given one EEPROM dump, convert the dump to update instructions that can be sent to a printer.
+
+Given two EEPROM dumps, produces only the required instructions needed to update the contents from the first to the second. This is currently quite crude and assumes dumps are aligned (starting from the same address or same stride).
+
+Optionally writes the instructions to the specified port (requires ``printcore`` from [Pronterface]).
+
+
+## Memory analysis
+
 ### ``dump_sram``
 
 Dump the content of the entire SRAM using the D2 command.
 Requires ``printcore`` from [Pronterface].
 
-### ``dump_crash``
-
-Dump the content of the last crash dump on MK3+ printers using D21.
-Requires ``printcore`` from [Pronterface].
-
 ### ``elf_mem_map``
 
 Generate a symbol table map with decoded information starting directly from an ELF firmware with DWARF debugging information (which is the default using the stock board definition).
@@ -28,7 +37,15 @@ When used with ``--map`` and a single elf file, generate a map consisting of mem
 With ``--qdirstat`` and a single elf file, generate a [qdirstat](https://github.com/shundhammer/qdirstat) compatible cache file which can be loaded to inspect memory utilization interactively in a treemap.
 
 This assumes the running firmware generating the dump and the elf file are the same.
-Requires Python3 and the [pyelftools](https://github.com/eliben/pyelftools) module.
+Requires the [pyelftools](https://github.com/eliben/pyelftools) module.
+
+
+## Crash dump handling
+
+### ``dump_crash``
+
+Dump the content of the last crash dump on MK3+ printers using D21.
+Requires ``printcore`` from [Pronterface].
 
 ### ``dump2bin``
 
@@ -36,19 +53,31 @@ Parse and decode a memory dump obtained from the D2/D21/D23 g-code into readable
 
 ### ``xfimg2dump``
 
-Extract a crash dump from an external flash image and output the same format produced by the D21 g-code.
+Extract a crash dump from an external flash image and output the same format produced by the D21 g-code. Requires python 3.
 
-### ``update_eeprom``
 
-Given one EEPROM dump, convert the dump to update instructions that can be sent to a printer.
+## Serial handling
-
-Given two EEPROM dumps, produces only the required instructions needed to update the contents from the first to the second. This is currently quite crude and assumes dumps are aligned (starting from the same address or same stride).
-
-Optionally writes the instructions to the specified port (requires ``printcore`` from [Pronterface]).
 
 ### ``noreset``
 
 Set the required TTY flags on the specified port to avoid reset-on-connect for *subsequent* requests (issuing this command might still cause the printer to reset).
 
 
+## Temperature analysis
+
+### ``tml_decode``
+
+Decode (or plot) the temperature model trace from a serial log file.
+
+The TML trace needs to be enabled by issuing "M155 S1 C3" and "D70 S1" to the printer, generally followed by a temperature model calibration request "M310 A F0".
+
+The parser is not strict, and will consume most serial logs with/without timestamps.
+
+By default the decoded trace is written to the standard output as a tab-separated table. If `--plot` is used, produce a graph into the requested output file instead:
+
+    ./tml_decode -o graph.png serial.log
+
+When plotting the [Matplotlib](https://matplotlib.org/) module is required.
+
+
 [Pronterface]: https://github.com/kliment/Printrun

tools/tml_decode

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+import argparse
+import re
+import struct
+
+TM_INTV = 270      # internal temperature regulation interval (ms)
+FAN_MAX_LAG = 2500 # maximum fan lag for reporting (ms)
+
+
+def parse_therm_dump(path):
+    # header
+    yield ['sample', 'ms', 'int', 'pwm', 't_nozzle', 't_ambient', 'fan']
+
+    cnt = 0
+    fan = float('NAN')
+    fan_lag = 0
+
+    for line in open(path):
+        # opportunistically parse M155 fan values
+        m = re.search(r'\bE0:\d+ RPM PRN1:\d+ RPM E0@:\d+ \bPRN1@:(\d+)$', line)
+        if m is not None:
+            fan = int(m.group(1))
+            fan_lag = 0
+        elif fan_lag > int(FAN_MAX_LAG/TM_INTV):
+            fan = float('NAN')
+
+        # search for the D70 TML output signature
+        m = re.search(r'\bTML (\d+) (\d+) ([0-9a-f]+) ([0-9a-f]+) ([0-9a-f]+)$', line)
+        if m is None:
+            continue
+
+        # decode fields
+        skip = int(m.group(1))
+        intv = int(m.group(2)) + TM_INTV
+        pwm = int(m.group(3), 16)
+        t = struct.unpack('f', int(m.group(4), 16).to_bytes(4, 'little'))[0]
+        a = struct.unpack('f', int(m.group(5), 16).to_bytes(4, 'little'))[0]
+
+        # output values
+        ms = cnt * TM_INTV
+        yield [cnt, ms, intv, pwm, t, a, fan]
+        smp = skip + 1
+        cnt += smp
+        fan_lag += smp
+
+
+def plot_therm_dump(data, output, title):
+    import matplotlib.pyplot as plt
+    import numpy as np
+
+    plt.gcf().set_size_inches(20, 5)
+    ts = np.array(data['ms']) / 1000
+    colors = iter([x['color'] for x in plt.rcParams['axes.prop_cycle']])
+
+    ax1 = plt.gca()
+    ax1.plot(ts, data['t_nozzle'], c=next(colors), label='Nozzle (C)')
+    ax1.axhline(data['t_ambient'][0], ls='-.', c='k', label='Ambient baseline (C)')
+    ax1.plot(ts, data['t_ambient'], c=next(colors), label='Ambient (C)')
+    ax1.set_ylabel('Temperature (C)')
+    ax1.set_xlabel('Time (s)')
+    ax1.legend(loc='upper left')
+
+    ax2 = plt.twinx()
+    ax2.plot(ts, np.array(data['fan']) / 255 * 100, c=next(colors), label='Fan (%)')
+    ax2.plot(ts, np.array(data['pwm']) / 127 * 100, c=next(colors), label='Heater (%)')
+    ax2.set_ylim(-1, 101)
+    ax2.set_ylabel('Power (%)')
+    ax2.legend(loc='upper right')
+
+    plt.title(title)
+    plt.tight_layout()
+    plt.savefig(output, dpi=300)
+
+
+def main():
+    ap = argparse.ArgumentParser(description='Decode (or plot) the TML trace contained in the serial LOG',
+                                 epilog="""
+        The TML trace needs to be enabled by issuing "M155 S1 C3" and "D70 S1" to the printer. By
+        Output the decoded trace to standard output by default. If --plot is provided, produce a
+        graph into IMG directly instead.
+    """)
+    ap.add_argument('log', metavar='LOG', help='Serial log file containing D70 debugging traces')
+    ap.add_argument('--plot', '-p', metavar='IMG', help='Plot trace into IMG (eg: output.png)')
+    args = ap.parse_args()
+
+    if args.plot is None:
+        # just output the streaming trace
+        for line in parse_therm_dump(args.log):
+            print('\t'.join(map(str, line)))
+    else:
+        # convert to dict and plot
+        it = parse_therm_dump(args.log)
+        data = dict(zip(next(it), zip(*it)))
+        plot_therm_dump(data, args.plot, args.log)
+
+
+if __name__ == '__main__':
+    exit(main())