Welcome to iotips

Auto direction control of 485 modules using not-gate (Part3/3)

In part 2, we learned that:

Only a small portion of RSS485 to TTL chips support automatic direction control. The majority of the chips need to be controlled by R̅E̅ and DE pin.
The SP3485 uses much less power than the MAX13487 in a 3.3v setup.
Most of the non-auto direction control modules you can buy, use a not gate as flow control.

Let’s take a closer look at how a not gate managed to control the flow direction in the XY-017 module. And then I modified the schematic a bit for easier understanding.

The TX signal goes through two not-gate and to DI pin. RO data also goes through two not-gate and to RX. The two not-gate could reform the signal and sharpen the edge, especially when we are driving an LED in this module.

The following oscilloscope graph recorded the signal of 74HC04 Pin 10 (which is the also same as pin DI) versus RS485+. The left part is the MCU asking for the holding resistor value and the right part is the reply from the sensor. There is a short voltage vibration during transitioning. The transition has to be located before the reply of the sensor. The sensor takes around 1ms before replying.

For the R̅E̅ and DI, the signal is latched by C4 (20nF ceramic cap). C4 is charged via R7 and discharged via D3 (there is no current flow into Pin 13), so the discharge rate is much faster than the charging rate. When Pin 10 is high, C4 gets charged. When Pin 10 is low, C4 discharges.

Take a closer look at pin 13 and the inverted output pin 12, we see that the 74ch04 inverts the signal when the input is around 1.3V.

The charging and discharge time has to be set up within the range for a correct latching time of R̅E̅ and DE. Or otherwise, the signal will be disturbed. For a too small capacitor, I changed the capacitor from 32nF to 1nF. R̅E̅ and DE changed state when the requesting message is not yet finished.

As a result, the 485+ 485- are pulled to neutral, and the difference between 485+ and 485- becomes zero (bus idling state). Although the sensor can still understand the message and make a reply.

And here is a zoomed-in capture.

If a too large capacitor is chosen, the transition will occur during the sensor reply signal and disturb the message. As a result, the MCU cannot understand the reply. ***In later time, I will switch to a Schmitt trigger inverter to check any improvement***

Auto direction control by transistor

On the other hand, we could use a PNP transistor as a not gate to control the flow.

MCU sending data

When TX is high, R̅E̅ and DE are low, SP3485 enters receiving mode. When TX is 1, the transistor activates, R̅E̅ and DE pull low, SP3485 output enters high-Z mode (high impedance mode), which means, the output is floating. When the output is floating, the bus line B- will be pulled down to ground while A+ pulled up to Vcc. So when TX is 1, A+ line is 1 and B- line is 0.

When TX is 0, the transistor is off, R̅E̅ and DE pull high. When DE is 1 and DI is 0, B- outputs 1, and A+ outputs 0. So when TX is 0, the 485 line is 0.

MCU receiving data

When the MCU is receiving data, the TX line is pulled 1, R̅E̅ and DE are 0, SP3485 enters receiving mode and transceives the 485 data to MCU.

	TXD	R̅E̅	DE	DI(Pull GND)	B	A	Ro
MCU sending	1	0	0	0	0(Floating)	1(Floating)	1
MCU sending	0	1	1	0	1	0	0
MCU receiving	1	0	0	0	0(Floating)	1(Floating)	1
MCU receiving	1	0	0	0	1(Floating)	0(Floating)	0

Appendix: power consumption of Sipex SP3485 and Texas Instruments SN74HC04N

The following is using a Sipex SP3485 chip with Texas Instruments SN74HC04N hex inverter. The Sipex SP3485 does not feature a low power shutdown mode. So this module cannot drop to μA level consumption, please consider Sipex SP3481 or MaxLinear SP3485 if you need a low power mode. In this circuit board, all the pull-down and pull-up resistors are changed to 100k to minimize power loss.

Average transmission current: 4.47mA (requesting: 10.8mA, replying:653.7μA)
Average idling current: 182.86μA

However, the drawback of using such a weak pull-up is that the signal will be much noisier. The good news is that RS485 calculate the difference between AB line, as a result, they compensate each other.

Power consumption of different RS485 to TTL modules (Part2/3)

In this article, I will check the power consumption of different common RS485 to TTL modules. Please note that this is the total consumption of the module including LED indicators for some of the modules.

I will be using the same setup as part 1 with the temperature humidity sensor. Baud-rate:9600, 80 bit (include starting and ending bit) request and then a 90 bit reply data.

The six RS485 to TTL modules I will be using.

1. MAX485 module

Typical operating voltage: 5V
Sleep mode: No
Slew-rate limiting: No
Fail-safe circuitry: Output short-circuit protection, thermal shutdown
AutoDirection control: No
Data Rate: 2.5 (Mbps)
ESD-Protection: –
Module LED indicator: Power LED (always ON)
Maxim Integrated Datasheet: here

Average transmission current: 112.6mA
Average idling current: 4.7mA
Consumption time: 8.5ms (only consume power during requesting)

***The MAX485 module requires 5V power input as mentioned by the manual, please take your own risk when using 3.3v. But the communication looks normal with my own test, I test ran it for 60 mins***

Average transmission current: 35.86mA
Average idling current: 2.32mA
Consumption time: 8.5ms (only consume power during requesting)

2. SP3485E module

Typical operating voltage: 3.3V, 5V logic tolerant (module power input 3V-30V)
Sleep mode: R̅E̅ high and DE low for 600ns
Slew-rate limiting: No
Fail-safe circuitry: Driver output short-circuit protection
AutoDirection control: No (Yes, controlled by 74HC04 provided by the module)
Data Rate: 10 (Mbps)
ESD-Protection: 2kV
Module LED indicator: TXD indicator, RXD indicator
MaxLinear Datasheet: here

Let’s zoom in on the reply region and have a closer look. The current chart matches the RS485 signal.

Average transmission current: 1.67mA (requesting: 2.46mA, replying 1.11mA)
Average idling current: 321uA

3. MAX13487EESA

Typical operating voltage: 5V
Sleep mode: S̅H̅D̅N̅ pin
Slew-rate limiting: Yes
Fail-safe circuitry: TTL side hot swapping protection
AutoDirection control: Yes
Data Rate: 0.5 (Mbps)
ESD-Protection: 15kV
Module LED indicator: Power LED (always ON), TXD indicator, RXD indicator
Maxim Integrated Datasheet: here

***The MAX13487EESA module requires 5V power input as mentioned by the manual, please take your own risk when using 3.3v. But the communication looks normal with my own test, I test ran it for 60 mins***

Average transmission current: 6.26mA (requesting: 6.67mA, replying:5.93mA)
Average idling current: 5.5mA

4. SCM3721ASA signal isolation YD3082EESA module

Typical operating voltage: 3V-5.5V
Sleep mode: R̅E̅ high and DE low
Slew-rate limiting: Yes
Fail-safe circuitry: Receiver pulls high when receiver’s differential inputs are either shorted, open circuit, or connected to a terminal resistor
AutoDirection control: No (Yes, controlled by HC14 provided by the module)
Data Rate: 1 (Mbps)
ESD-Protection: 15kV
Module LED indicator: None
Datasheet: here

Average transmission current: 4.5mA (requesting: 5.27mA, replying:3.8mA)
Average idling current: 3.12mA

5. SCM3725ASA signal isolation SCM3406ASA module

Typical operating voltage: 3V-5.5V
Sleep mode: R̅E̅ high and DE low
Slew-rate limiting: No
Fail-safe circuitry: Receiver pulls high when receiver’s differential inputs are either shorted, open circuit, or connected to a terminal resistor
AutoDirection control: No (Yes, controlled by HC14 provided by the module)
Data Rate: 10 (Mbps)
ESD-Protection: 15kV
Module LED indicator: None
Datasheet: here

Average transmission current: 6.84mA (requesting: 7.66mA, replying:6.09mA)
Average idling current: 5.19mA

***2 transmissions failed out of 1358 trials ***

6. ADUM5401 DC-DC isolated SP485EE module

Typical operating voltage:5V (module power input 3.3V-5V)
Sleep mode: No
Slew-rate limiting: No
Fail-safe circuitry: Driver output short-circuit protection
AutoDirection control: No (Yes, controlled by HC14 provided by the module)
Data Rate: 10 (Mbps)
ESD-Protection: 15kV
Module LED indicator: Power indicator, TXD indicator, RXD indicator
Datasheet: here

Average transmission current: 109.71mA (requesting: 120.72mA, replying:102.06mA)
Average idling current: 72.27mA

Summary

Most of the modules work stable and reliable under my test environment and the code mentioned in the previous article, only the SCM3406ASA module has transmission failure, but only a small portion.

The Max485 is a no go in industrial applications, because it has no ESD protection but the same time high power consumption (But the MAX485 is the only few chips that offer industry qualifications such as MIL-STD-883B). Besides, the MAX485 chip is not any cheaper than the other chip such as MAX481. However, this MAX485 module is extremely cheap compared to the other modules. This could be a good starting point to test out your first RS485 circuit.

The ADUM54, SP485EE module is DC-DC power and signal isolated. If you are working in a harsh environment and do not have power contain, this module may be your choice.

The MAX13487EESA chip is the only chip that has auto direction control and hot-swaps protection. If you want to make your own module and do not want to add any hex inverter, this may be your choice.

The SP3485 module consumes relatively low power, if you are working with a power limited environment, this is your first go. And in part 3, we will dive deeper into SP3485 chip and make our own PCB.

Using Modbus RTU and RS485 with Arduino and ESP32 (Part 1/3)

This article will demonstrate how to use an Arduino Mega and ESP32 to read Modbus485 sensors data using a MAX485 and MAX13487E module. The first example is an Arduino Mega with MAX485 to read a ten-in-one sensor. And the second example is using a ESP32 and MAX13487E module to read a temperature and humidity sensor.

Arduino Mega and a ten-in-one environmental sensor

This MAX 485 module is cheap and has a simple circuit diagram. However the MAX485 does not offer an automatic direction control and hot swap protection. The need of controlling the flow direction pin makes the module a bit unstable and less easy to use. This module operates at 5V.

Let’s take a look at the sensor’s manual. And we will use function 0x03-Read Holding Registers, to read all the 11 type of data.

The command in hexadecimal is as follow:

Device address	Function Code	Starting Address High	Starting Address Low	Quantity High	Quantity low	CRC High	CRC low
0x01	0x03	0x00	0x00	0x00	0x0B	0x04	0x0D

Master Read Command (from arduino)

Device address	Function Code	Number of Bytes	eCO2_H	eCO2_L	TVOC_H	TVOC_L
0x01	0x03	0x16	0x02	0x4A	0x00	0xC4

Slave Reply (from sensor) Part1

……	……	MCU_TEMP_H	MCU_TEMP_L	dB_H	dB_L	CRC_H	CRC_L
……	……	0x00	0xC8	0x00	0x42	0x53	0x25

Slave Reply (from sensor) Part2

There are one logic zero start bit and one logic one stop bit, the 8-bit data are in between. This communication has a baud rate of 9600.

/*

  RS485_HalfDuplex.pde - example using ModbusMaster library to communicate
  with EPSolar LS2024B controller using a half-duplex RS485 transceiver.

  This example is tested against an EPSolar LS2024B solar charge controller.
  See here for protocol specs:
  http://www.solar-elektro.cz/data/dokumenty/1733_modbus_protocol.pdf

  Library:: ModbusMaster
  Author:: Marius Kintel <marius at kintel dot net>

  Copyright:: 2009-2016 Doc Walker

  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.

*/

#include <ModbusMaster.h>

/*!
  We're using a MAX485-compatible RS485 Transceiver.
  Rx/Tx is hooked up to the hardware serial port at 'Serial'.
  The Data Enable and Receiver Enable pins are hooked up as follows:
*/
#define MAX485_DE      3 //Driver output Enable pin DE Active HIGH
#define MAX485_RE_NEG  2 //receiver output Enable pin RE Active LOW
/////////////////////////In many cases you may also shorting both pin together
// instantiate ModbusMaster object
ModbusMaster node;

void preTransmission()  //set up call back function
{
  digitalWrite(MAX485_RE_NEG, 1);
  digitalWrite(MAX485_DE, 1);
}

void postTransmission()   //set up call back function
{
  digitalWrite(MAX485_RE_NEG, 0);
  digitalWrite(MAX485_DE, 0);
}

void setup()
{
  pinMode(MAX485_RE_NEG, OUTPUT);
  pinMode(MAX485_DE, OUTPUT);
  // Init in receive mode
  digitalWrite(MAX485_RE_NEG, 0);
  digitalWrite(MAX485_DE, 0);

  // Modbus communication runs at 115200 baud
  Serial.begin(9600);
  Serial2.begin(9600); //serial 2: RX2 and TX2 in Arduino Mega
  // Modbus slave ID 1, numbers are in decimal format
  node.begin(1, Serial2);  //data from max 485 are communicating with serial2
  // Callbacks allow us to configure the RS485 transceiver correctly
  node.preTransmission(preTransmission);
  node.postTransmission(postTransmission);
}


void loop()
{
  uint8_t result;  

  // Read 16 registers starting at 0x00, read 11 register. Meaning that read 0x00, then read 0x01, so on and so forth. Until the eleventh resister 0x0A
  result = node.readHoldingRegisters(0x0000, 11);
  if (result == node.ku8MBSuccess)
  {
    Serial.println("------------");
    Serial.print("eCO2: ");
    Serial.println(node.getResponseBuffer(0x00));
    Serial.print("TVOC: ");
    Serial.println(node.getResponseBuffer(0x01));
    Serial.print("CH2O: ");
    Serial.println(node.getResponseBuffer(0x02));
    Serial.print("PM2.5: ");
    Serial.println(node.getResponseBuffer(0x03));
    Serial.print("HUMI: ");
    Serial.println(node.getResponseBuffer(0x04)/100.0f);
    Serial.print("TEMP: ");
    Serial.println(node.getResponseBuffer(0x05)/100.0f);
    Serial.print("PM10: ");
    Serial.println(node.getResponseBuffer(0x06));
    Serial.print("PM1.0: ");
    Serial.println(node.getResponseBuffer(0x07));
    Serial.print("Lux: ");
    Serial.println(node.getResponseBuffer(0x08));
    Serial.print("MCU TEMP: ");
    Serial.println(node.getResponseBuffer(0x09)/100.0f);
    Serial.print("NOISE (dB）: ");
    Serial.println(node.getResponseBuffer(0x0A));
  }
  delay(5000);


}

Schematic diagram for Arduino Mega and ten-in-one sensor

For ESP32, I switched to another 485 to TTL module, MAX13487E. The MAX13487E supports TTL-side hot swapping and auto direction control, which make this chip much easier to use then MAX485. From the datasheet, the MAX13487E also requires a 5v power . But this module works in my set up, so please take your own risk when powering with 3.3v.

The hexadecimal code ESP32 is sending to the sensor is:

Device address	Function Code	Starting Address High	Starting Address Low	Quantity High	Quantity low	CRC High	CRC low
0x49	0x03	0x00	0x20	0x00	0x02	0xCA	0x49

Master Read Command (from ESP32)

Reading data and print on serial monitor

#include <ModbusMaster.h>
int errorcnt =0;
int cycle =0;

#define RXD2 16
#define TXD2 17

ModbusMaster node;

void setup() {
  Serial.begin(9600);
  Serial2.begin(9600, SERIAL_8N1, RXD2, TXD2); //using serial 2 to read the signal from MAX13487E
  node.begin(73, Serial2);
}

void loop()
{
  uint8_t result;  

  // Read 16 registers starting at 0x3100)
  result = node.readHoldingRegisters(0x20, 2);
  if (result == node.ku8MBSuccess)
  {
    Serial.println("------------");
    Serial.print("Temp: ");
    Serial.println(node.getResponseBuffer(0x00)/10.0f);
    Serial.print("Humi: ");
    Serial.println(node.getResponseBuffer(0x01)/10.0f);

    Serial.print("ERROR count: ");
    Serial.println(errorcnt);    
    Serial.print("cycle: ");
    Serial.println(cycle);  
    cycle++;
  }
  else {
    errorcnt++;
    cycle++;
    Serial.print("ERROR count: ");
    Serial.println(errorcnt);
  }
  delay(5000);

}

There is another common 485 to TTL module using SP3485E. This chip operates with 3.3V and is 5V logic tolerant. The SP3485 does not support auto direction control but uses hardware to latch the enable pin. Please note that the TXD pin is connecting to the TX pin of the MCU, and RXD to RX.

In part 2, we will discuss a few common RS485 to TTL modules and check out which one is more power efficient.

Chinese Text generation for LILYGO T5-4.7 inch E-Paper ESP32

There is a python script on the LilyGo-EPD47 GitHub for text generation. But the program is not optimized for Chinese words, one big reason is that the Struct GFXfont uses Unicode interval to provide a range of characters. We usually won’t import the whole Chinese dictionary into ESP32 because there are way too many words. Instead, we will search the Unicode for the corresponding Chinese characters and only import the words which will be displayed.

And that comes to a problem when using the fontconvert.py. Usually, those Chinese words we are using will not have a close Unicode number. Take 像素(pixel)as an example, these 2 Chinese words have a Unicode of U+50CF and U+7D20. And there are already 11346 words in between, a full set of fonts could easily cover more than twenty thousand characters which the esp32 just doesn’t have enough memory to handle.

I am using these two very useful links to inspect the font file and check the corresponding Unicode of any characters. For example, I want to import a sentence “最大像素 and esp32”, found out their Unicode: 0x6700, 0x5927, 0x50CF, 0x7D20 and sort them in ascending order. Finally, load them into a modified version of fontconvert.py.

#!python3
from code import interact
import freetype
import zlib
import sys
import re
import math
import argparse
from collections import namedtuple

parser = argparse.ArgumentParser(description="Generate a header file from a font to be used with epdiy.")
parser.add_argument("name", action="store", help="name of the font.")
parser.add_argument("size", type=int, help="font size to use.")
parser.add_argument("fontstack", action="store", nargs='+', help="list of font files, ordered by descending priority.")
parser.add_argument("--compress", dest="compress", action="store_true", help="compress glyph bitmaps.")
args = parser.parse_args()

GlyphProps = namedtuple("GlyphProps", ["width", "height", "advance_x", "left", "top", "compressed_size", "data_offset", "code_point"])

font_stack = [freetype.Face(f) for f in args.fontstack]
compress = args.compress
size = args.size
font_name = args.name

# inclusive unicode code point intervals
# must not overlap and be in ascending order
intervals = [
    [32, 126],
    [160, 255],
    [0x50CF],
    [0x5927],
    [0x6700],
    [0x7D20],

    # (0x2500, 0x259F),
    # (0x2700, 0x27BF),
    # # powerline symbols
    # (0xE0A0, 0xE0A2),
    # (0xE0B0, 0xE0B3),
    # (0x1F600, 0x1F680),
]
newintervals = []

def norm_floor(val):
    return int(math.floor(val / (1 << 6)))

def norm_ceil(val):
    return int(math.ceil(val / (1 << 6)))

for face in font_stack:
    # shift by 6 bytes, because sizes are given as 6-bit fractions
    # the display has about 150 dpi.
    face.set_char_size(size << 6, size << 6, 150, 150)

def chunks(l, n):
    for i in range(0, len(l), n):
        yield l[i:i + n]

total_size = 0
total_packed = 0
all_glyphs = []

def load_glyph(code_point):
    face_index = 0
    while face_index < len(font_stack):
        face = font_stack[face_index]
        glyph_index = face.get_char_index(code_point)
        if glyph_index > 0:
            face.load_glyph(glyph_index, freetype.FT_LOAD_RENDER)
            return face
            break
        face_index += 1
        print (f"falling back to font {face_index} for {chr(code_point)}.", file=sys.stderr)
    raise ValueError(f"code point {code_point} not found in font stack!")

for interval in intervals:
    if len(interval) <= 1:
        interval.append(interval[0]+1)
    newintervals.append(interval)


for i_start, i_end in newintervals:
    for code_point in range(i_start, i_end + 1):
        face = load_glyph(code_point)
        bitmap = face.glyph.bitmap
        pixels = []
        px = 0
        for i, v in enumerate(bitmap.buffer):
            y = i / bitmap.width
            x = i % bitmap.width
            if x % 2 == 0:
                px = (v >> 4)
            else:
                px = px | (v & 0xF0)
                pixels.append(px);
                px = 0
            # eol
            if x == bitmap.width - 1 and bitmap.width % 2 > 0:
                pixels.append(px)
                px = 0

        packed = bytes(pixels);
        total_packed += len(packed)
        compressed = packed
        if compress:
            compressed = zlib.compress(packed)

        glyph = GlyphProps(
            width = bitmap.width,
            height = bitmap.rows,
            advance_x = norm_floor(face.glyph.advance.x),
            left = face.glyph.bitmap_left,
            top = face.glyph.bitmap_top,
            compressed_size = len(compressed),
            data_offset = total_size,
            code_point = code_point,
        )
        total_size += len(compressed)
        all_glyphs.append((glyph, compressed))

# pipe seems to be a good heuristic for the "real" descender
face = load_glyph(ord('|'))

glyph_data = []
glyph_props = []
for index, glyph in enumerate(all_glyphs):
    props, compressed = glyph
    glyph_data.extend([b for b in compressed])
    glyph_props.append(props)

print("total", total_packed, file=sys.stderr)
print("compressed", total_size, file=sys.stderr)

print("#pragma once")
print("#include \"epd_driver.h\"")
print(f"const uint8_t {font_name}Bitmaps[{len(glyph_data)}] = {{")
for c in chunks(glyph_data, 16):
    print ("    " + " ".join(f"0x{b:02X}," for b in c))
print ("};");

print(f"const GFXglyph {font_name}Glyphs[] = {{")
for i, g in enumerate(glyph_props):
    print ("    { " + ", ".join([f"{a}" for a in list(g[:-1])]),"},", f"// {chr(g.code_point) if g.code_point != 92 else '<backslash>'}")
print ("};");

print(f"const UnicodeInterval {font_name}Intervals[] = {{")
offset = 0
for i_start, i_end in newintervals:
    print (f"    {{ 0x{i_start:X}, 0x{i_end:X}, 0x{offset:X} }},")
    offset += i_end - i_start + 1
print ("};");

print(f"const GFXfont {font_name} = {{")
print(f"    (uint8_t*){font_name}Bitmaps,")
print(f"    (GFXglyph*){font_name}Glyphs,")
print(f"    (UnicodeInterval*){font_name}Intervals,")
print(f"    {len(newintervals)},")
print(f"    {1 if compress else 0},")
print(f"    {norm_ceil(face.size.height)},")
print(f"    {norm_ceil(face.size.ascender)},")
print(f"    {norm_floor(face.size.descender)},")
print("};")

LDO linear regulator HT7333 performance in a TO-92 package

HT7333 is a low power low drop out (LDO) linear regulator and you may consider using it in many energy-constrained projects. However, there is no in-depth performance chart shown in the datasheet, so I decided to perform a simple test to investigate the drop-out voltage in different loads. Mainly under 50ma, 100ma, 200ma and 300 ma.

I soldered a 0.75mm² wire to the leg directly to reduce the heat lost in the wire (compared to Dupont) as well as added two 10uF capacitors to the input and output of HT7333 to comply with the datasheet.

***I am using a low-end virtual load and power supply, the readings are not very accurate. I tried a few trials and the result varies quite a bit.

Under 50ma load, HT7333 proform the best. The output voltage starts to drop at around 3.5v input and the drop out voltage stays around 0.3v.

Larger the loading, the larger the drop-out voltage. The voltage starts to drop below 3.3v at 4.2v input under 300ma loading. The drop out voltage lays around 0.9v to 1.3v.

Be aware that if you are using a battery-powered microcontroller, a sudden demand in power (such as wifi connection) might surge the current over 300ma which may trigger a brownout detection.

ESP32 analog input linearity

ESP32(s) are fast and have a lot more analog input pins than the esp8266. However, the ESPs do not provide an external voltage reference pin, so you need to manually compensate for the non-linearity of the analog input readings. But first, let’s take a look at the test result.

Note that I added a 35nF ceramic capacitor parallel to the analog input, I am using GPIO 36 as the analog input. A linear voltage regulator HT7333 is used to provide a stable 3.308v power source for the ESP32.

Here is the test code modified from Arduino tutorial, I averaged 1000 test points. The result is quite stable.

#include <Arduino.h>
#define ADC_PIN 36

const int numReadings = 1000;

float readings[numReadings];      // the readings from the analog input
int readIndex = 0;              // the index of the current reading
float total = 0;                  // the running total
float average = 0;                // the average

void setup() {
  Serial.begin(115200);
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    readings[thisReading] = 0;
  }
}

void loop() {
  total = total - readings[readIndex];
  readings[readIndex] = analogRead(ADC_PIN);
  total = total + readings[readIndex];
  readIndex = readIndex + 1;

  if (readIndex >= numReadings) {
    readIndex = 0;
  }
  average = total / numReadings;
  Serial.println(average * 3.3 / 4096 ,3);
  delay(1);        // delay in between reads for stability
}

And here is the result:

A linear region lies between 0.2v to2.5v, then I tried to add a compensation by shifting the readings by +0.153 and rotating the axis by a factor of 0.99. So the final readings should be: compensated result = (raw readings + 1.53) * 0.99.

In the region of 0.2v to 2.5v, we could archive a result of +/- 7mv. By using a voltage divider, we could limit the voltage below 2.5v. For example, when measuring a li-ion, we use a 470k ohm and 680k ohm resistor to convert the maximum battery voltage 4.2v to 2.483v.

${{battery voltage * 680k} \over {470k+680k}}=({{analogreadings * 3.3} \over 4096} + 1.53) *0.99$

And here is the wiring diagram:

Note that I removed the smoothing capacitor because the readings are quite stable.

After all the massive work and statistics, I only calibrated one of my many ESPs. And this compensation may not be suitable for all other boards!! Because they vary from each other. According to Espressif, each ESP32 has its own ADC reference voltage VRef (1100mV nominal), but it may range from 1000mV to 1200mV. This value is measured and burned into eFuse BLOCK0 during factory calibration. We can take this number as a reference and use it to compensate for our measurements.

By using espefuse.py we can find out the Vref stored in eFuse $IDF_PATH/components/esptool_py/esptool/espefuse.py --port /dev/ttyUSB0 adc_info. Here I checked two of my ESP32 boards, the first one has a 1100mV VRef and the second board is 1114mV.

C:\Users\Developer\AppData\Local\Programs\Python\Python310\Lib\site-packages>python espefuse.py --port COM4 adc_info
Connecting...
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
espefuse.py v3.3

=== Run "adc_info" command ===
ADC VRef calibration: 1100mV

C:\Users\Developer\AppData\Local\Programs\Python\Python310\Lib\site-packages>python espefuse.py --port COM4 adc_info
Connecting...................
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
espefuse.py v3.3

=== Run "adc_info" command ===
ADC VRef calibration: 1114mV

Before continuing to our code, we need to first include the library esp_adc_cal.h. And use the following structure variable and function.

struct esp_adc_cal_characteristics_t

Is a structure to store the ADC’s characteristics, such as ADC unit and channel (GPIO), ADC’s attenuation, resolution, etc.

While the function esp_adc_cal_characterize()used to initialize the above structure.

  esp_adc_cal_characteristics_t adc_chars; //declare a structure named adc_chars
  esp_adc_cal_value_t val_type = esp_adc_cal_characterize(ADC_UNIT_1, ADC_ATTEN_DB_11, ADC_WIDTH_BIT_12, 1100, &adc_chars); //Generate compensation curve of an ADC at a particular attenuation by VRef

ADC_UNIT_1: The ADC unit. Unit 1 includes 8 channels: GPIO32-GPIO39
ADC_ATTEN_DB_11: ADC Attenuation. 11dB, suitable rage 150mV ~ 2450mV
ADC_WIDTH_BIT_12: Bit capture width for ADC unit. 12 bit resolution
1100: Default VRef if eFuse value is not available
&adc_chars: The memory address of adc_chars

After we initialized the characteristic curve, we can take a look at the API Espressif has provided: esp_adc_cal_raw_to_voltage(). This function takes the compensation curve and raw ADC readings and returns voltage in mV. So let’s take out our ESP32 and check how accurate this function provided.

uint32_t esp_adc_cal_raw_to_voltage(uint32_t adc_reading, const esp_adc_cal_characteristics_t *chars)

Actually, the compensation was done in a similar manner on top: [y = coeff_a * x + coeff_b]. We can even check the coeff_a and coeff_b, they are stored as a public member of esp_adc_cal_characteristics_t. Read the coeff_a by Serial.println(adc_chars.coeff_a) and Serial.println(adc_chars.coeff_b)

For my ESP32, the coeff_a is 53470, coeff_b is 142. Let’s varify the how ESP32 do the compensation. Take a single measurement as an example, the analogRead() is: 2586, esp_adc_cal_raw_to_voltage() is: 2252.

$compensatedVoltage = {{53470} \over {65536}} * 2586 +142$

$compensatedVoltage = 2251.885$

The result 2251.885 in integer is 2252 which matches esp_adc_cal_raw_to_voltage().

In the following, I used two ESP32 to check the accuracy of the esp_adc_cal_raw_to_voltage() function. One with a VRef of 1114mV and one with 1100mV

Actual voltage vs **esp_adc_cal_raw_to_voltage**

Here is the code I used for testing. The compensated result looks accurate and could generate a reading with +/-15mV , which already exceeded the precision of my power supply.

To conclude, the esp_adc_cal_raw_to_voltage() does a great job of compensating the ADC, as long as the voltage goes between 150 ~ 2450mV which it can be easily done by adding a voltage divider.

#include <Arduino.h>
#include "driver/adc.h"
#include "esp_adc_cal.h"
#define ADC_PIN 35

const int numReadings = 100; 
uint32_t cal_sumReadings = 0; 
int vref = 1100;

void setup(){
  Serial.begin(115200);
}

void loop(){
  cal_sumReadings =0;
  for (int i = 0; i < numReadings; i++){
    cal_sumReadings += analogRead(ADC_PIN);
    delay(1);
  }
  
  esp_adc_cal_characteristics_t adc_chars; //declare a structure named adc_chars
  esp_adc_cal_value_t val_type = esp_adc_cal_characterize(ADC_UNIT_1, ADC_ATTEN_DB_11, ADC_WIDTH_BIT_12, 1100, &adc_chars); //Define compensation of an ADC at a particular attenuation.
  if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
    Serial.printf("eFuse Vref:%u mV", adc_chars.vref);
    Serial.println("");
    vref = adc_chars.vref;
    uint32_t averageReadingInt = cal_sumReadings/numReadings;
    Serial.print ("esp_adc_cal_raw_to_voltage: ");
    Serial.println(esp_adc_cal_raw_to_voltage(averageReadingInt, &adc_chars));
    Serial.print ("coeff_a: ");
    Serial.println(adc_chars.coeff_a);
    Serial.print ("coeff_b: ");
    Serial.println(adc_chars.coeff_b);
    Serial.print ("averageReadingInt: ");
    Serial.println(averageReadingInt);
  }
  Serial.println("~~~~~~~~~~~~~~~~~~~~~~~~");
  delay (1000);
}

Summary

In this post, 2 compensation methods are discussed. In the beginning, we measured the ADC output value with respect to the actual voltage, then we come up with a formula:

${{battery voltage * 680k} \over {470k+680k}}=({{analogreadings * 3.3} \over 4096} + 1.53) *0.99$

By applying this formula, we get a quite close result for a specific ESP32. This method is easy to implement to code and does not require additional libraries. However, the formula does not fit other ESP32, it could be troublesome to measure and record the ADC characteristics for each ESP32.

In the later part, we introduced the esp_adc_cal.h library, and discussed how the library does the compensation.

Finally, we used a handy function esp_adc_cal_raw_to_voltage() to compare the result with the actual voltage. It turns out to be quite accurate and could archive a +/-15mV result within the recommended voltage region. Such accuracy should be enough for a rough measurement without the need for external ADCs such as measuring battery voltage. If you want a more precise and accurate analog measurement, I still recommend going for an external ADC such as ADS1115.

Additional notes

The val_type variable above stores the method for generating the characteristics curve.

ESP_ADC_CAL_VAL_EFUSE_VREF: generated by reference voltage stored in eFuse
ESP_ADC_CAL_VAL_EFUSE_TP: generated by Two-point method
ESP_ADC_CAL_VAL_DEFAULT_VREF: generated by default VRef if eFuse value is not available
ESP_ADC_CAL_VAL_EFUSE_TP_FIT: generated by Two-point method and fitting curve coefficient

But all my ESP32 are using ESP_ADC_CAL_VAL_EFUSE_VREF, so I only discussed this type above. You can check Espressif’s documentation if you need more details.

By the way, with respect to the function, only Two-point method is supported, the parameter default_vref is unused. And only ADC_WIDTH_BIT_13 is supported.

Changing Node-red URL address

Add the following line to ~/.node-red/settings.js will set the admin panel from “/” to “/admin”, dashboard from “/ui” to “/”.

module.exports = {
    // the tcp port that the Node-RED web server is listening on
    uiPort: process.env.PORT || 1880,

    httpAdminRoot: '/admin', //added line
    ui: { path: "/"}, //added line
...
...
...

Monitoring CPU metrics on Grafana using Node-red

Grafana 8.2.2 seems does not support running a shell script natively. I would recommend parsing the CPU status via an external REST API. Node-red will be used in the following example.

In this article, you will learn:

How to monitor cpu status on Node-red
How to create a REST api on Node-red
How to parse the REST data using Grafana Infinity plug-in

Monitoring CPU status

We will be using exec node to run shell commands. In addition, we can add buttons to shutdown and reboot as well.

Notice that the script for running CPU usage is a bit complicated: top -d 0.5 -b -n3| grep "Cpu(s)"|tail -n 1| awk '{print 100-$8}'as well as ram free | grep Mem | awk '{printf "%.2f", ($2-$7)*100/$2}'. The final number is calculated in percentage, where 100 is overloaded. You may play around with the parameters, I will make a post to briefly introduce shell manipulation in the future.

[{"id":"2653a0b0.26d638","type":"ui_gauge","z":"1a2e114b.5f398f","name":"","group":"1890881e.83819","order":2,"width":0,"height":0,"gtype":"gage","title":"CPU Temperature","label":"C","format":"{{value}}","min":0,"max":"100","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":1090,"y":120,"wires":[]},{"id":"fba68adf.14e13","type":"exec","z":"1a2e114b.5f398f","command":"vcgencmd measure_temp","addpay":false,"append":"","useSpawn":"","timer":"","name":"raspberry pi temperature","x":630,"y":160,"wires":[["fa5b499.e176cb8"],[],[]]},{"id":"7c8379de.068868","type":"inject","z":"1a2e114b.5f398f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"2","crontab":"","once":false,"onceDelay":"","topic":"","payload":"","payloadType":"date","x":310,"y":200,"wires":[["fba68adf.14e13","972ece2a.3dbe8","6242be99.26ac88"]]},{"id":"fa5b499.e176cb8","type":"function","z":"1a2e114b.5f398f","name":"trim string","func":"str = msg.payload\nmsg.payload = str.substring(5,9);\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":860,"y":120,"wires":[["2653a0b0.26d638","28b64a2c.b32116","7d734a89.e1e164"]]},{"id":"683a2b05.736204","type":"ui_button","z":"1a2e114b.5f398f","name":"","group":"c5f1b8aa.45bc08","order":2,"width":0,"height":0,"label":"Reboot","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"","x":320,"y":580,"wires":[["1cf31554.2aaa63"]]},{"id":"1cf31554.2aaa63","type":"exec","z":"1a2e114b.5f398f","command":"sudo reboot","addpay":false,"append":"","useSpawn":"","timer":"","name":"sudo reboot","x":590,"y":580,"wires":[[],[],[]]},{"id":"38e53dbf.ca594a","type":"ui_button","z":"1a2e114b.5f398f","name":"","group":"c5f1b8aa.45bc08","order":3,"width":0,"height":0,"label":"Shutdown","color":"","bgcolor":"red","icon":"","payload":"","payloadType":"str","topic":"","x":320,"y":640,"wires":[["c409cae5.8e4128"]]},{"id":"c409cae5.8e4128","type":"exec","z":"1a2e114b.5f398f","command":"sudo shutdown 0","addpay":false,"append":"","useSpawn":"","timer":"","name":"sudo shutdown -h now","x":620,"y":640,"wires":[[],[],[]]},{"id":"28b64a2c.b32116","type":"ui_chart","z":"1a2e114b.5f398f","name":"","group":"1890881e.83819","order":3,"width":0,"height":0,"label":"","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":"12","removeOlderPoints":"1000","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"useDifferentColor":false,"x":1070,"y":160,"wires":[[]]},{"id":"972ece2a.3dbe8","type":"exec","z":"1a2e114b.5f398f","command":"top -d 0.5 -b -n3| grep \"Cpu(s)\"|tail -n 1| awk '{print 100-$8}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"cpu usage","x":590,"y":240,"wires":[["b9372186.ed1a5","8144e676.4d5d68"],[],[]]},{"id":"6242be99.26ac88","type":"exec","z":"1a2e114b.5f398f","command":"free | grep Mem | awk '{printf \"%.2f\", ($2-$7)*100/$2}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"ram usage","x":590,"y":320,"wires":[["d8ede4a4.507998","9b301b09.8c0468"],[],[]]},{"id":"b9372186.ed1a5","type":"ui_gauge","z":"1a2e114b.5f398f","name":"","group":"1890881e.83819","order":1,"width":0,"height":0,"gtype":"gage","title":"Processor","label":"CPU","format":"{{value}}","min":0,"max":"102","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":1080,"y":200,"wires":[]},{"id":"9b301b09.8c0468","type":"ui_gauge","z":"1a2e114b.5f398f","name":"","group":"9a96a8b1.92db78","order":1,"width":0,"height":0,"gtype":"gage","title":"Memory","label":"RAM","format":"{{value}}","min":0,"max":"100","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":1060,"y":300,"wires":[]},{"id":"d8ede4a4.507998","type":"debug","z":"1a2e114b.5f398f","name":"","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1070,"y":340,"wires":[]},{"id":"8144e676.4d5d68","type":"debug","z":"1a2e114b.5f398f","name":"","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1090,"y":240,"wires":[]},{"id":"57aad7a5.391708","type":"inject","z":"1a2e114b.5f398f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"60","crontab":"","once":false,"onceDelay":"","topic":"","payload":"","payloadType":"date","x":310,"y":460,"wires":[["8c19e802.e92e88"]]},{"id":"8c19e802.e92e88","type":"exec","z":"1a2e114b.5f398f","command":"df -h / | awk '{print $5}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"disk storage usage","x":610,"y":460,"wires":[["9869bc88.1d73f"],[],[]]},{"id":"9869bc88.1d73f","type":"function","z":"1a2e114b.5f398f","name":"trim string","func":"var test = msg.payload.substr(5);  //trim off Use%\nmsg.payload = test.substring(0, test.length - 1) //trim off the next line string\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":900,"y":460,"wires":[["1ba34040.c7d4","9c3b4510.f9fbc8"]]},{"id":"9c3b4510.f9fbc8","type":"debug","z":"1a2e114b.5f398f","name":"","active":false,"tosidebar":false,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1130,"y":480,"wires":[]},{"id":"1ba34040.c7d4","type":"ui_gauge","z":"1a2e114b.5f398f","name":"","group":"72fc319.cc425d","order":1,"width":0,"height":0,"gtype":"gage","title":"Disk","label":"Usage","format":"{{value}}","min":0,"max":"100","colors":["#00b500","#e6e600","#ca3838"],"x":1110,"y":440,"wires":[]},{"id":"7d734a89.e1e164","type":"debug","z":"1a2e114b.5f398f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1050,"y":80,"wires":[]},{"id":"1890881e.83819","type":"ui_group","name":"Col1","tab":"c3173234.2636e","order":1,"disp":false,"width":"6"},{"id":"c5f1b8aa.45bc08","type":"ui_group","name":"Actions","tab":"c3173234.2636e","order":4,"disp":true,"width":"6"},{"id":"9a96a8b1.92db78","type":"ui_group","name":"Col2","tab":"c3173234.2636e","order":2,"disp":false,"width":"6"},{"id":"72fc319.cc425d","type":"ui_group","name":"Col3","tab":"c3173234.2636e","order":3,"disp":false,"width":"6"},{"id":"c3173234.2636e","type":"ui_tab","name":"RPi Control","icon":"dashboard","order":1}]

Creating REST api on Node-red

We will use the http in and http response node to set up the REST API. Please note that you cannot initiate an HTTP response by itself. You should rather trigger the flow by going into the URL, or by an HTTP request node or by Grafana in this case. The http in node (Method: GET) will initiate the flow once the URL has been requested. Here is an example of setting up a simple payload as a content of the URL /cpumetrics. Open the browser and navigate to http://localhost:1880/cpumetrics.

[{"id":"c01ca5bf.a938b8","type":"http in","z":"9566abd8.7dc4d8","name":"","url":"/cpumetrics","method":"get","upload":false,"swaggerDoc":"","x":140,"y":140,"wires":[["1b98b2af.7af11d"]]},{"id":"8ccf75b3.ddd338","type":"comment","z":"9566abd8.7dc4d8","name":"REST endpoint","info":"","x":140,"y":80,"wires":[]},{"id":"1b98b2af.7af11d","type":"function","z":"9566abd8.7dc4d8","name":"simple payload","func":"msg.payload = {\"message\": \"this is the payload\"};\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":340,"y":140,"wires":[["26f72364.1a687c"]]},{"id":"26f72364.1a687c","type":"http response","z":"9566abd8.7dc4d8","name":"","statusCode":"","headers":{},"x":510,"y":140,"wires":[]}]

REST api with query parameters

Sometimes you want your reply to be changed dynamically based on different parameters. You could use an if-else statement on the function to give a specific reply to different parameters. After deploying the flow, navigate to the URL: http://localhost:1880/cpumetrics?metrics=cpu. If the metric parameter is equal to “cpu”, the page will reply a {“message”: “cpu loading”} otherwise, {“message”: “some other metrics”} will be replied.

[{"id":"59ff2a1.fa600d4","type":"http in","z":"1adc4b7f.bb9655","name":"","url":"/cpumetrics","method":"get","upload":false,"swaggerDoc":"","x":240,"y":200,"wires":[["a3d72dd3.05bc2","2ec4bd64.072302","e6e4ab4c.254ea8","8a2f5529.8baa88","54c1e70d.ab3e18","c4d1aa9b.1bb7c8"]]},{"id":"4eefdf83.b473c","type":"comment","z":"1adc4b7f.bb9655","name":"REST endpoint","info":"","x":240,"y":140,"wires":[]},{"id":"266c286f.d993d8","type":"http response","z":"1adc4b7f.bb9655","name":"","statusCode":"","headers":{},"x":1030,"y":220,"wires":[]},{"id":"a3d72dd3.05bc2","type":"function","z":"1adc4b7f.bb9655","name":"request with Query Parameters","func":"if (msg.payload.metrics === \"cpu\"){\n    msg.payload = {\"message\": \"cpu loading\"};\n}else{\n    msg.payload = {\"message\": \"some other metrics\"};\n}\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":690,"y":100,"wires":[["266c286f.d993d8","2ec4bd64.072302"]]}]

REST api with cpu metrics

In the last example, we will put the real-time cpu metrics as content. A join node is used to put together the disk space and ram space into a single payload. Besides, we will use a http request node to invoke the endpoint. The payload could be checked on the debug window.

[{"id":"4512cdf1.3cd794","type":"exec","z":"608a04eb.6011bc","command":"free | grep Mem | awk '{printf \"%.2f\", ($2-$7)*100/$2}'","addpay":"","append":"","useSpawn":"false","timer":"","oldrc":false,"name":"RAM","x":490,"y":220,"wires":[["c5b1a07e.aa5fb"],[],[]]},{"id":"1ade78d7.8383f7","type":"http in","z":"608a04eb.6011bc","name":"","url":"/cpumetrics","method":"get","upload":false,"swaggerDoc":"","x":120,"y":160,"wires":[["334a3882.c0e9b8","696e606d.dc86f"]]},{"id":"696e606d.dc86f","type":"function","z":"608a04eb.6011bc","name":"append topic","func":"msg.topic = \"ram\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":350,"y":220,"wires":[["4512cdf1.3cd794"]]},{"id":"334a3882.c0e9b8","type":"exec","z":"608a04eb.6011bc","command":"df -h / | awk '{print $5}'","addpay":"","append":"","useSpawn":"false","timer":"","oldrc":false,"name":"disk space","x":350,"y":160,"wires":[["c9c3ed28.056db"],[],[]]},{"id":"c9c3ed28.056db","type":"function","z":"608a04eb.6011bc","name":"trim off string","func":"var test = msg.payload.substr(5);  //trim off Use%\nmsg.payload = test.substring(0, test.length - 2) //trim off the next line string\nmsg.topic = \"storage\";\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":530,"y":160,"wires":[["c5b1a07e.aa5fb"]]},{"id":"c5b1a07e.aa5fb","type":"join","z":"608a04eb.6011bc","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":710,"y":180,"wires":[["a00ebb59.59d158","2b0fcd92.4fd282"]]},{"id":"a00ebb59.59d158","type":"http response","z":"608a04eb.6011bc","name":"","statusCode":"","headers":{},"x":870,"y":160,"wires":[]},{"id":"4899725b.041d9c","type":"comment","z":"608a04eb.6011bc","name":"REST endpoint","info":"","x":120,"y":100,"wires":[]},{"id":"5f21f1c0.0f39c","type":"http request","z":"608a04eb.6011bc","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"localhost:1880/cpumetrics","tls":"","persist":false,"proxy":"","authType":"","x":290,"y":320,"wires":[[]]},{"id":"7904b22.74ae64c","type":"inject","z":"608a04eb.6011bc","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":320,"wires":[["5f21f1c0.0f39c"]]},{"id":"2fc5dcef.b8ce74","type":"comment","z":"608a04eb.6011bc","name":"Invoke REST endpoint","info":"","x":160,"y":280,"wires":[]},{"id":"2b0fcd92.4fd282","type":"debug","z":"608a04eb.6011bc","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":890,"y":200,"wires":[]}]

Setting up Grafana plug-in

After preparing the data for display, we could set up the Grafana dashboard to display those data. Go to Configurations -> plugins, search for Infinity and install it.

Then go to Configurations -> Data sources -> Add data source.

The current version (Version 0.7.8) of Infinity does not require connectivity testing in this setup stage, so leave all fields blank and click Save & test.

Head over to your dashboard and add a new panel. Select a Stat graph, and select Infinity as the Data source. Put http://localhost:1880/cpumetrics in the URL field. Click Add Colums and put storage and ram as the selector, Number as type. You should now see the data shown in the preview panel.

Using TFT_eSPI library on esp32

TFT_eSPI library supports many drivers, I tested the following procedure with a 2.4 inch TFT LCD module with a ILI9341V driver.

Other than ILI9431, the library also supports:
ILI9341_2, ST7735, ILI9163, S6D02A1, RPI_ILI9486, HX8357D, ILI9481, ILI9486, ILI9488, ST7789, ST7789_2, R61581, RM68140, ST7796, SSD1351, SSD1963_480, SSD1963_800, SSD1963_800ALT, ILI9225, GC9A01.

Downloading the library

You can install the library on the Arduino IDE: Tools -> Manage Libraries…
Search for “TFT_eSPI”, and we are finding the library by Bodmer.

Editing the User_Setup.h file

By default, this library setup will work for NodeMCU using a ILI9341 driver. But in our case, we are using ESP32, so some of the setups have to be changed.

Open the User_Setup.h file with your favorite editor, it is ok to use notepad. (If you do not know where is the User_Setup.h file, you can File -> Preferences -> Sketchbook location:
Inside the directory, go into libraries -> TFT_eSPI. You could now see the User_Setup.h file.)

Uncomment the driver you are using, the default driver is already ILI9341, so we can skip Section 1.

//                            USER DEFINED SETTINGS
//   Set driver type, fonts to be loaded, pins used and SPI control method etc
//
//   See the User_Setup_Select.h file if you wish to be able to define multiple
//   setups and then easily select which setup file is used by the compiler.
//
//   If this file is edited correctly then all the library example sketches should
//   run without the need to make any more changes for a particular hardware setup!
//   Note that some sketches are designed for a particular TFT pixel width/height


// ##################################################################################
//
// Section 1. Call up the right driver file and any options for it
//
// ##################################################################################

// Define STM32 to invoke optimised processor support (only for STM32)
//#define STM32

// Defining the STM32 board allows the library to optimise the performance
// for UNO compatible "MCUfriend" style shields
//#define NUCLEO_64_TFT
//#define NUCLEO_144_TFT

// STM32 8 bit parallel only:
// If STN32 Port A or B pins 0-7 are used for 8 bit parallel data bus bits 0-7
// then this will improve rendering performance by a factor of ~8x
//#define STM_PORTA_DATA_BUS
//#define STM_PORTB_DATA_BUS

// Tell the library to use 8 bit parallel mode (otherwise SPI is assumed)
//#define TFT_PARALLEL_8_BIT

// Display type -  only define if RPi display
//#define RPI_DISPLAY_TYPE // 20MHz maximum SPI

// Only define one driver, the other ones must be commented out
#define ILI9341_DRIVER       // Generic driver for common displays
//#define ILI9341_2_DRIVER     // Alternative ILI9341 driver, see https://github.com/Bodmer/TFT_eSPI/issues/1172
//#define ST7735_DRIVER      // Define additional parameters below for this display
//#define ILI9163_DRIVER     // Define additional parameters below for this display
//#define S6D02A1_DRIVER
//#define RPI_ILI9486_DRIVER // 20MHz maximum SPI
//#define HX8357D_DRIVER
//#define ILI9481_DRIVER
//#define ILI9486_DRIVER
...
...
...

In section 2, we need to select the suitable pinout for your MCU. The default setup is for NodeMCU 8266, so comment out those lines by adding two slash characters //. And uncommenting those lines for esp32.

// ###### EDIT THE PIN NUMBERS IN THE LINES FOLLOWING TO SUIT YOUR ESP8266 SETUP ######

// For NodeMCU - use pin numbers in the form PIN_Dx where Dx is the NodeMCU pin designation
//#define TFT_CS   PIN_D8  // Chip select control pin D8
//#define TFT_DC   PIN_D3  // Data Command control pin
//#define TFT_RST  PIN_D4  // Reset pin (could connect to NodeMCU RST, see next line)
//#define TFT_RST  -1    // Set TFT_RST to -1 if the display RESET is connected to NodeMCU RST or 3.3V

//#define TFT_BL PIN_D1  // LED back-light (only for ST7789 with backlight control pin)

...
...
...

// ###### EDIT THE PIN NUMBERS IN THE LINES FOLLOWING TO SUIT YOUR ESP32 SETUP   ######

// For ESP32 Dev board (only tested with ILI9341 display)
// The hardware SPI can be mapped to any pins

#define TFT_MISO 19
#define TFT_MOSI 23
#define TFT_SCLK 18
#define TFT_CS   15  // Chip select control pin
#define TFT_DC    2  // Data Command control pin
#define TFT_RST   4  // Reset pin (could connect to RST pin)
//#define TFT_RST  -1  // Set TFT_RST to -1 if display RESET is connected to ESP32 board RST

// For ESP32 Dev board (only tested with GC9A01 display)
// The hardware SPI can be mapped to any pins
...
...
...

Connecting the Esp32 to the display

TFT Display side Esp32 side	ESP32 side
GND	GND
3.3v	3.3v
SCL (sometimes called SCLK)	GPIO18
SDA (sometimes called MOSI)	GPIO23
RES (sometimes called Reset)	GPIO4
DC (sometimes called data/ command)	GPIO2
CS ( sometimes called SS)	GPIO15
BLK-LED backlight signal	N/A

Now you are ready to use the display module, you may try some built-in examples, File -> Examples -> TFT_eSPI.

Assigning fixed names for USB devices in Raspberry Pi

You can find the USB devices by ls /dev/*USB*

pi@raspberrypi:~ $ ls /dev/*USB*
/dev/ttyUSB0  /dev/ttyUSB1
pi@raspberrypi:~ $

and more information by lsusb or dmesg | grep ttyUSB.

pi@raspberrypi:~ $ lsusb
Bus 002 Device 003: ID 05e3:0626 Genesys Logic, Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 006: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
Bus 001 Device 005: ID 046d:c534 Logitech, Inc. Unifying Receiver
Bus 001 Device 004: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 003: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
pi@raspberrypi:~ $
pi@raspberrypi:~ $
pi@raspberrypi:~ $ dmesg | grep ttyUSB
[    5.284638] usb 1-1.3: ch341-uart converter now attached to ttyUSB0
[    5.312424] usb 1-1.4: FTDI USB Serial Device converter now attached to ttyUSB1

But sometimes, you may want to use the /dev/ path to descript your USB, such as selecting the serial port in NodeRed.

Procedure of assigning fixed name

Finding unique attributes for your USB devices
Creating udev rules for your applications
Creating match keys and assignment keys to bind the USB devices to your desired name.
Applying rules

Finding unique attributes of USB devices

Using udevadm info [options] [devpath|file|unit…]to check the details of the USB devices. -a option stands for --attribute-walk.

pi@raspberrypi:/etc/udev/rules.d $ udevadm info -a /dev/ttyUSB0

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/ttyUSB0/tty/ttyUSB0':
    KERNEL=="ttyUSB0"
    SUBSYSTEM=="tty"
    DRIVER==""

  looking at parent device '/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/ttyUSB0':
    KERNELS=="ttyUSB0"
    SUBSYSTEMS=="usb-serial"
    DRIVERS=="ch341-uart"
    ATTRS{port_number}=="0"

  looking at parent device '/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0':
    KERNELS=="1-1.3:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="ch341"
    ATTRS{bNumEndpoints}=="03"
    ATTRS{bInterfaceProtocol}=="02"
    ATTRS{bInterfaceSubClass}=="01"
    ATTRS{bInterfaceClass}=="ff"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{supports_autosuspend}=="1"
    ATTRS{authorized}=="1"

  looking at parent device '/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3':
    KERNELS=="1-1.3"
    SUBSYSTEMS=="usb"
    DRIVERS=="usb"
    ATTRS{bcdDevice}=="0264"
    ATTRS{busnum}=="1"
    ATTRS{authorized}=="1"
    ATTRS{bDeviceProtocol}=="00"
    ATTRS{urbnum}=="31"
    ATTRS{bConfigurationValue}=="1"
    ATTRS{bDeviceClass}=="ff"
    ATTRS{devspec}=="(null)"
    ATTRS{bMaxPower}=="98mA"
    ATTRS{idProduct}=="7523"
    ATTRS{rx_lanes}=="1"
    ATTRS{tx_lanes}=="1"
    ATTRS{product}=="USB Serial"
    ATTRS{bNumInterfaces}==" 1"
    ATTRS{avoid_reset_quirk}=="0"
    ATTRS{bDeviceSubClass}=="00"
    ATTRS{speed}=="12"
    ATTRS{removable}=="unknown"
    ATTRS{bNumConfigurations}=="1"
    ATTRS{configuration}==""
    ATTRS{devpath}=="1.3"
    ATTRS{quirks}=="0x0"
    ATTRS{bMaxPacketSize0}=="8"
    ATTRS{bmAttributes}=="80"
    ATTRS{ltm_capable}=="no"
    ATTRS{version}==" 1.10"
    ATTRS{devnum}=="65"
    ATTRS{maxchild}=="0"
    ATTRS{idVendor}=="1a86"

...
...
...

Some unique attributes are ” idProduct”, “idVendor”. Make sure you are searching these attributes under the corresponding Port, in my case, usb1/1-1/1-1.3. Repeat the udevadm info command and search for attributes for the other devices. Remember to change the /dev/ttyUSB0 to /dev/ttyUSB1 or else only plug a single device at a time.

To understand more about USB ports, please visit Extra readings down below.

Creating udev rules

Linux stores file-like device nodes in /dev directory. Each node points to a part of a system or device, no matter it exists or not. Because the /dev directories contain every device that may exits in the system, it may be very large and become difficult to manage.

udev plays an important role to manage /dev directories by providing a path forward, by matching information provided by sysfs and rules provided by users.

udev files should be kept in /etc/udev/rules.d directories and with a .rulessuffix. Files in /etc/udev/rules.d are parsed in lexical order. In general, you want your rule file to be parsed first, so it is suggested that /etc/udev/rules.d/10-local.rulesis a good choice.

sudo touch /etc/udev/rules.d/10-local.rules to create a rule file.

In a rules file, lines starting with “#” are treated as comments. Every other non-blank line is a rule. Rules cannot span multiple lines.

To learn more about udev, you may man udev man udevadmor visit http://www.reactivated.net/writing_udev_rules.html#udevinfo

Writing rules

Each line of rule should contain at least one match key and at least one assignment to construct a key-value pair. When all match keys are fulfilled, the rule will apply and the action of assignment will be performed.

Do not insert any line breaks in the rules, udev will see your one rule as multiple rules. And here are my rules to assign a new name for the 2 USB devices. Just add the following lines in your 10-local.rules file.

SUBSYSTEM=="tty", ATTRS{idProduct}=="7523", ATTRS{idVendor}=="1a86", SYMLINK+="ttyUSB_HL-340_device"
SUBSYSTEM=="tty", ATTRS{idProduct}=="6001", ATTRS{idVendor}=="0403", SYMLINK+="ttyUSB_FT232_device"

SUBSYSTEM will match against the subsystem of the device.
SYMLINK is the alternative name(s) you want to assign to the USB device. This will not change or hide the original name but only provide an alternative name to link to the device.

Besides an exact matching of strings, you could also use shell-style pattern matching, such as *, ? and [].

Applying rules

Run udevadm trigger to apply the new rules.

sudo udevadm trigger

To check the result.


pi@raspberrypi:/etc/udev/rules.d $ ls -l /dev/ttyUSB*
crw-rw---- 1 root dialout 188, 0 Dec  3 12:05 /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 1 Dec  3 12:05 /dev/ttyUSB1
lrwxrwxrwx 1 root root         7 Dec  3 12:05 /dev/ttyUSB_FT232_device -> ttyUSB1
lrwxrwxrwx 1 root root         7 Dec  3 12:05 /dev/ttyUSB_HL-340_device -> ttyUSB0

If you know the top-level device path, you can use udevadm test to show the action. Or else, just use

pi@raspberrypi:/etc/udev/rules.d $ udevadm test -a -p  $(udevadm info -q path -n /dev/ttyUSB_FT232_device)

Serial port name in Node-red

Now you can specify the USB you are pointing to in Node-red rather than guessing which is ttyUSB0 and ttyUSB1.

———————-This is separator———————–

Extra readings

You may use lsusb -t to explore what devices are connected to your pi. The following corresponds to no USB device connected.

pi@raspberrypi:~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

And the following corresponds to 4 USB devices connected. Dev 31 is a wireless mouse and keyboard. All devices are via USB2.0.

pi@raspberrypi:~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 27, If 0, Class=Vendor Specific Class, Driver=ch341, 12M
        |__ Port 2: Dev 31, If 0, Class=Human Interface Device, Driver=usbhid, 12M
        |__ Port 2: Dev 31, If 1, Class=Human Interface Device, Driver=usbhid, 12M
        |__ Port 3: Dev 26, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 12M
        |__ Port 4: Dev 30, If 0, Class=Hub, Driver=hub/4p, 480M

If you have a USB hub, then those connected devices will be under the hub port. In my case, the hub is connected to Port 3.


pi@raspberrypi:~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 3: Dev 36, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 1: Dev 37, If 0, Class=Human Interface Device, Driver=usbhid, 12M
            |__ Port 1: Dev 37, If 1, Class=Human Interface Device, Driver=usbhid, 12M
            |__ Port 2: Dev 41, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 12M
            |__ Port 3: Dev 42, If 0, Class=Vendor Specific Class, Driver=ch341, 12M

If you have USB 3.0 devices, they will be using Bus 02. In my case, both the USB hub and USB flash drive support USB 3.0.

pi@raspberrypi:~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 13, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 4: Dev 14, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 56, If 0, Class=Hub, Driver=hub/4p, 480M

The number after the word ‘usb’ in dmesg is actually the USB Bus and Port. In my case, the ch341-uart converter is under bus 1, port 1.3. In similar manner, usb 3.0 devices will be usb 2-X.X.

[161104.469392] usb 1-1.3: ch341-uart converter now attached to ttyUSB1