As it happens, Haaga-Helia University of Applied Sciences has a full-fledged studio for our communication students and other needs related to audio and video. The studio manager happened to come by and asked us at the 3D + Robo Lab to print him a stand for a studio clock. That was a nice little interlude, resulting in this:

While this was being printed, he happened to think aloud: if the VMIX system he uses to mix live video with inserts is already broadcasting the length and position of any insert that is running inside a live show, would it be possible to build a countdown clock for this, wireless being the preferred method? Currently he has no way of telling the guests in the studio about the counting down of the insert, so they look at him quizzically and he uses hand signals to try to tell them that they are about to go live again.

It’s not an ideal situation to either party.

I thought this a nice challenge. I had already fooled around with the ESP32 system on a chip long enough to know that it would be a solid foundation to this system. I asked him about the XML, and he said that the mixing software outputs a steady stream at the localhost address of the machine running the software. I had a look, and lo and behold, it had this at the address http://127.0.0.1:8088/api/?Function:

This is the start of the file, there is more, but the beef is seen here already. The vMix API is well documented at https://www.vmix.com/help23/. Note that the port 8088 must be part of the request.

Inside the XML you have the element <input> and that has an attribute “position”. There is also “duration”. I ventured to guess that both are in milliseconds, and sure enough, the duration of 6976980 milliseconds converts to 697 seconds, or 11 minutes 36 seconds.

There are other elements too that have the word “position” in them, but the first occurrences are the ones that matter here. If you had to refer to the others, you could employ the power of XML to full extent using TinyXML2, or, just start searching for the substrings later in the XML string, since you already know where the first occurrence was in the string.

It was clear by now that there would be six phases in the process to build the device.

1. figure out how to use the 57mm green LED displays with TPIC6B595N drivers
2. figure out the traffic between ESP32 and VMIX
3. figure out the remaining duration of the insert
4. turn values like 03:22 into separate numbers for printing on the LCD and the LEDs
5. show that and some other info on a LCD screen, and show the time data info on four LED displays

Let’s go through the phases one at a time.

1. Using the 57mm LED displays

for this project I chose the green 7 segment displays from Mouser, which is a very good purveyor of fine electronic thingies. These take in 9 volts, so they cannot be fed from the ESP32, which only gives out 5V anyway, and would rather flamboyantly fry if you tried to run these off the ESP32. These are also of the common anode type, so you need to feed them 9 volts, and ground the segments you want to show. These cost something in the region of 8 euros apiece. You also need shift registers of the type TPIC6B595N, which you can source from Mouser too. Every digit you use needs to have its own shift register.

WARNING:

For electricity, you can for example take an old laptop charger that gives out something like 12V and 2A, then use two buck-down regulators to put put 5V for the ESP and the logic part of the TPIC6B595N shift registers, and another toned to 9V for powering the common anode 7 segment displays.

Do not issue 9V to either the shift registers or the ESP32, or you will see the magic smoke escape the ESP32 and the shift registers, after which they will cease to work for you.

This video shows how I tested the displays first. The pins on the displays are five on the top and five on the bottom. The five on the bottom are 1 to 5, left to right, and the ones on the top are 6 to 10, respectively. Pins 1 and 5 are for the +9V, and when you connect the other pins to the ground, the corresponding segments light up. Here I have given 9 volts to pins 1 and 5, and am just poking the rest of the pins in the Ground holes.

The full pin table is as follows:

1 and 5: 9V
2: segment C
3: segment D
4: segment E
6: segment B
7: segment A
8: decimal point, not used in this project
9: segment F
10: segment G

The segments are counted from the top bar as A, clockwise until G, which is the middle bar.

When you use these in a real system, you must use shift registers. This is covered in more detail in another blog post, but I will add the relevant parts here.

Shift register operation is explained in this great tutorial. A shift register takes in one byte and delivers eight bits out of the pins in the chip. When these pins are connected to the segments of the 7 segment display, it will light the desired set of segments. The ESP32 sends out commands on four wires:

const int _7segCLK = 16; //Connected to TPIC pin 13: SRCLK (aka Clock)
const int _7segLATCH = 17; //Connected to TPIC pin 12: RCLK (aka Latch/load/CS/SS…)
const int _7segOE = 18; //Connected to TPIC pin 9: OE (Output Enable)
const int _7segDOUT = 19; //Connected to TPIC pin 3: SER (aka MOSI aka DATA IN)

The Clock pin is 16 on the ESP32 and it times the operation. 17 is Latch, and that is the gatekeeper. 18 is Output Enable, and that pin tells the chip to either turn on or off the output pins. 19 is Data Out, and that pin sends the actual data to the chip.

Note that you will govern all four displays with just four wires. This is because the 4 shift registers are chained together; their Data Out pin is connected to the next register’s Data In pin, and the other three wires are similarly connected forward from the first register to the other three. If you are using a breadboard to build this, as I did, simply attach another jumper wire from shift register 1’s governing pins to the next shift register. Do pay attention to these wirings, because debugging these is a real pain in the posterior, and get one display to run before attaching the next one.

To make the letters and numbers you need, you create a byte array:

const byte numTable[] =
{
  B11111100,  //numeral 0
  B01100000,  //numeral 1
  B11011010,  //numeral 2
  B11110010,  //numeral 3
  B01100110,  //numeral 4
  B10110110,  //numeral 5
  B10111110,  //numeral 6
  B11100000,  //numeral 7
  B11111110,  //numeral 8
  B11110110,  //numeral 9
  B01101110,  //capital letter H
  B01111010   //small letter d
};

The last two ones, the H and the d, are for vanity reasons alone. With these you can make the LED display show HH3d to advertise our Haaga-Helia 3D + Robo Lab, where this was made.

When you want the first display to show the numeral 5, for example, you do it like this:

digitalWrite(_7segLATCH, LOW);   //Tells all SRs that uController is sending data
delay(10);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[5]);
digitalWrite(_7segLATCH, HIGH);

First the pin LATCH is pulled low to tell the shift registers that the times they are a-changin’, and then the actual data gets sent in the shiftOut command. The pin is the one designated as the Data Out, and the clock signal is then sent via the Clock pin. LBSFIRST means the data is sent out as Least Significant Bit first, and the actual character byte is the numTable[5].

In essence, this is all there is to it. However you must remember that you cannot send data out as 3:22 if you have 3 minutes and 22 seconds running in the insert. That will not compute. You must parse your numerical data into the string variables led10min, led1min, led10sec, and led1sec as 0, 3, 2 and 2, and send it out to the shift registers in a function like this:

send7seg(led10min.toInt(), led1min.toInt(), led10sec.toInt(), led1sec.toInt());

Note that the strings are turned into integers at the sending time with the toInt() function. This is because the table that contains the segment combinations need bytes, and single digit integers can be used here. The function that does the magic of displaying 0322 is then this:

void send7seg(int myX, int myY, int myZ, int myQ) {
  digitalWrite(_7segLATCH, LOW);   //Tells all SRs that uController is sending data
  delay(10);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myQ]);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myZ]);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myY]);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myX]);
  delay(10);
  digitalWrite(_7segLATCH, HIGH);
}

This video shows you a single digit cycling 0 to 9. Incidentally, in the start you see the two buck down regulators that give 5 and 9 V respectively. Note how they enter the breadboard’s two voltage strips labeled 5 and 9, and how power is taken from both strips to the single shift register I had connected at this time. The yellow jumper wire is just a bystander that doesn’t take part in the action, but happened to fall on the set.

The connections of the TPIC6B595N are as follows:

NC – Not connected
Vcc – 5V
SER IN – Data Out pin 19 of the ESP32
SER OUT – to the next SER IN of the chain of shift registers
DRAIN0 to DRAIN7 – the segments A to G, with the decimal point at DRAIN7
GND – all three to Ground
SRCLR – to 5V
G – Output Enable, to pin 18 of the ESP32
RCK – Latch pin 17 of the ESP32
SRCK – Clock pin 16 of the ESP32

I have found it best to connect the 3 ground wires first, then the 2 voltage wires, and then the four ESP32 wires. Then I insert the drain segment wires, and label them, and only connect them to the display pins when all other wires are done. Pay special attention to the segments when you connect the display pins as they do not run linearly, but in the order I listed above.

Also, if you use a different type of segmented display from mine, bear in mind that I am using a common anode type here. Therefore I always give 9 volts to the two pins, 1 and 5, on the display, and the segments light up when the shift register pins are pulled low. In a common cathode system, you would supply power to each segment from the shift register, and the ground would be always on. You must pay careful attention to which type of LEDs you are using, and select the shift register accordingly. For example, I have another blog on using home made 7 segment displays and a 74HC595 shift register.

2. The traffic between VMIX and ESP32

ESP32 is a natural choice for any project that needs WLAN traffic. It also has Bluetooth for projects that don’t need WLAN but have to be able to communicate somewhat. My first impression here was that while the VMIX machine is on cabled Ethernet, we’d need to use some tunneling software like ngrok to allow us access to the VMIX machine’s localhost service. I did some research, and while it is fully possible to tunnel to a machine, it is unnecessary here. ESP32 is able to connect to a hotspot running on a desktop machine, if it has a USB WLAN adapter installed.

For testing purposes on your laptop, you need to have Apache or Internet Information Server running on the laptop, otherwise browsers will report unable to connect. A good way is to install XAMPP, which gives you a full Internet server locally. Its main folder is located in c:\xampp\htdocs, so the test file I used went there. Note that in the code below, the URL is not 127.0.0.1, but 192.168.137.1. You can find that address by opening a command window and using the command ipconfig, as below. The address you need is the IPv4 address:

C:\Users\heikki.hietala>ipconfig

Windows IP Configuration


Ethernet adapter Ethernet:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : haagahelia.amk

Wireless LAN adapter Local Area Connection* 9:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Local Area Connection* 10:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::b0f4:e206:62aa:7297%6
   IPv4 Address. . . . . . . . . . . : 192.168.137.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

This is then used in the actual URL to get the data. You can set the IP address of the dongle to a static IP if you want, the only critical thing is that the ESP32 must be in the hotspot’s coverage, and it must know the IP address. In that sense it is a good idea to first fix the IP to a static one, because it is hard-coded in the ESP32 program.

My test version had the XML in a file, which was copied to the XAMPP home directory as above, so the URL points to that file:

#include <WiFi.h>
#include <WiFiMulti.h>
#include <HTTPClient.h>

WiFiMulti wifiMulti;

//WIFI
void setup() {
  Serial.begin(115200);
  wifiMulti.addAP("MY_SSID", "MY_PASSWORD");
  while (WiFi.status() != WL_CONNECTED) { //WAITING FOR WIFI
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected.");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

}
void loop() {
  // wait for WiFi connection
  if ((wifiMulti.run() == WL_CONNECTED)) {
    HTTPClient http;
    http.begin("http://192.168.137.1/vmixdemo.xml"); //HTTP
    int httpResponseCode = http.GET();
    if (httpResponseCode > 0) {
      Serial.print("HTTP Response code: ");
      Serial.println(httpResponseCode);
      String payload = http.getString();
      Serial.println(payload);
      Serial.println("Returned XML in characters: " + String(payload.length()));
      Serial.println();

    }

At the top there are just a couple of libraries, after which a WifiMulti client is created. The local IP is then written on the serial monitor so that you know it is connected all right, and then in the start of the loop, a HTTPClient object called… wait for it… http is created. It is then passed the URL to the vmixdemo.xml file, which I had saved from a real live VMIX feed for testing purposes. The http.GET() function goes asking for the server, and the response is stored in an integer variable. If all goes well, it is 200, and it is also printed on the serial monitor. The actual payload from the server is retrieved via http.getString(), and stored in a String variable called payload.

The payload is written on the Serial monitor for debugging purposes, and its length is found by the String(payload.length()) function.

So far, so good.

I bought a TP-LINK dongle, which set us back a daunting 7.49 euros. This particular model has the added benefit of only reaching the 2.4GHz network – ESP32 doesn’t have the 5 GHz band, and this saves the hassle of having to force the dongle down. With the dongle installed, click the network icon on Windows, and push the mobile hotspot button there. Give it the same credentials you used in the code, restart the adapter, and start your ESP32. The code I have written shows you what’s happening in the Serial monitor. At this stage you may want to give the dongle a static IP in the dongle’s properties in Windows.

When you then change the URL to read http://192.168.137.1:8088/api/?Function, the system outputs the entire XML that contains the state of the vMix session at that moment the request is made. What happens next is discussed in the following paragraph.

3. Duration of the insert that is running

If you want to do this the proper way, you should use an XML parser. That would be able to climb up and down the XML tree and return all child node data and attribute data that you could ever ask for. Well, I tried that. I installed a library called TinyXML2, and while it is powerful and neat, I found it unnecessarily complex for my purposes.

This is because the XML I was sent from localhost is actually very simple. Reading from the beginning of the string, there is one occurrence of the word “position”; one occurrence of “duration” and one occurrence of “loop”, and these carry the meaningful data. Later occurrences can be ignored Instead of using a parser to get these values from the XML, I went the quick and dirty way as I so often do.

In a string, you can use the indexOf() function to locate a substring in a string.

The payload string starts like this:

<vmix><version>24.0.0.58</version><edition>HD</edition><inputs><input key="69c9bac2-1a42-45ba-9ded-5d049a523ac6" number="1" type="Video" title="capture - 05 marraskuu 2019 - 10-06-40 ap..mp4" shortTitle="capture - 05 marraskuu 2019 - 10-06-40 ap..mp4" state="Running" position="25379" duration="6976980" loop="False" ........

If I wrote

  myPosition = payload.indexOf("position");

I was given the number of characters at which point the word Position starts, ie. 268. Asking

  myDuration = payload.indexOf("duration");

gives the position of the word Duration as an integer. And finally, asking for

  myLoop = payload.indexOf("loop");

gives the position of the word Loop in the string. A bit of maths then separates the value from the text:

  myPosString = payload.substring(myPosition + 10, myDuration - 2);

The string containing the position data, which I want to store in myPosString, in the XML is between the myPosition + 10 and myDuration -2. myPosition is followed by a quote, eight characters (position), an equal sign, and another quote. After the value, there is another quote and a space, so I need to go back two characters from myDuration as shown.

The same method is used to extricate the value of the duration of the insert:

  myDurString = payload.substring(myDuration + 10, myLoop - 2);

The myLoop variable is just the location of the word behind the duration value. The values are then turned into integers:

  myFinalPosition = myPosString.toInt();
  myFinalDuration = myDurString.toInt();

just to be able to do the remaining insert length:

  myCountDown = myFinalDuration - myFinalPosition;
  Serial.println("Remaining duration in milliseconds: " + String(myCountDown));
  strMinutes = int(myCountDown / 60000);
  strSeconds = int(int(myCountDown / 1000) % 60);
  Serial.println("Remaining duration in time: " + String(strMinutes) + ":" + String(strSeconds));

Now, the myCountDown variable contains the remaining running time of the insert and that can be passed to the next two parts in the process, first the LCD, and finally the LED displays.

4. Using the LCD display

The LCD displays prevalent in the market are really easy to use. All you need is a library to drive the LCD, and then some code to write the data on the display. I have written another blog post on using LCDs, so please refer to this one if you have never used LCDs.

The relevant code in this product is as follows:

#include <LiquidCrystal_I2C.h>
int mySDA = 21;
int mySCL = 22;
int lcdColumns = 20;
int lcdRows = 4;
LiquidCrystal_I2C lcd(0x27, lcdColumns, lcdRows);

The LiquidCrystal_I2C library allows you to create the object lcd using the parameters above it. In the I2C communication method, mySDA and mySCL are the only two wires you need to supply data to the LCD display, not counting the+5V and Ground, of course. Make sure you plug them in the right order. The 0x27 in the lcd object is the hexadecimal address of the display. If for some reason your display doesn’t start to work with these setup values, you can try 0x3c, which is another common value, but the 0x27 should work. If things get really hairy, there are code snippets to discover all I2C protocol addresses, but that probably isn’t necessary.

Then, in the Setup function, write this:

// initialize LCD
lcd.init();
// turn on LCD backlight
lcd.backlight();

This enables the device and turns on the backlight. To write anything onto the device, the syntax is always the same:

Serial.begin(115200);
wifiMulti.addAP("MY_SSID", "MY_PASSWORD");
Serial.print("connecting to ");
Serial.println(ssid);
lcd.setCursor(0, 0);
lcd.print("Connecting to ");
lcd.setCursor(0, 1);
lcd.print(ssid);

To write, you must place the cursor first, with the lcd.setCursor() function. The top left corner is (0,0) and the leftmost column on row 2 is (0,1), since the count starts from row 0.

To get something like this to show, you need a little more code:

lcd.clear();
lcd.setCursor(0, 0);
lcd.print("Insert data found.");
myPosition = payload.indexOf("position");
lcd.setCursor(0, 1);
lcd.print("Position:");
lcd.setCursor(11, 1);
lcd.print(myPosition);
lcd.setCursor(0, 2);
lcd.print("Duration:");
myDuration = payload.indexOf("duration");
myLoop = payload.indexOf("loop");
myPosString = payload.substring(myPosition + 10, myDuration - 2);
myDurString = payload.substring(myDuration + 10, myLoop - 2);
lcd.setCursor(11, 2);
lcd.print(myDurString);
 

This code has been purged of values I send to the Serial monitor, but this code is the actual LCD for writing the screen you see in the image. You need to use the lcd.clear() function whenever you want to refresh the values – there will be leftover data on the screen otherwise and it will not look good. Also note that I convert everything into strings before attempting to write on the LCD, just to save my remaining hair.

So, by now there is the way to find out the remaining insert length and serve it out to the LCD and LEDs.

5. Preparing data for sending it to LCDs and LEDs at the same time

After I have figured out the remaining insert length in seconds, it’s a good idea to use a function called countdown() to do the actual presentation of the data on both the LCD and the LEDs. It looks like this:

void countdown(int timer) {
  delay(10);
  int minutes, seconds;
  String strMinutes, strSeconds, led10min, led1min, led10sec, led1sec;
  for (int x = int(timer); x > -1; x--) {
    minutes = int(x / 60);
    seconds = int(x % 60);
    strMinutes = String(minutes);
    strSeconds = String(seconds);
    //figuring out the LED segment numbers
    if (strMinutes.length() > 1) {
      led10min = strMinutes.substring(0, 1);
      led1min = strMinutes.substring(1, 2);
    }
    else {
      led10min = "0";
      led1min = strMinutes;
    }
    if (strSeconds.length() > 1) {
      led10sec = strSeconds.substring(0, 1);
      led1sec = strSeconds.substring(1, 2);
    }
    else {
      led10sec = "0";
      led1sec = strSeconds;
    }

The function gets the remaining length in the parameter timer. This is in seconds. It’s then used as the countdown timer in the for loop, and inside the loop, there are integers minutes and seconds that have the relevant time. To get the minutes from the running time, the variable is divided by 60, and the seconds result from a modulo division by 60c, which gives the remainder. The strMinutes and strSeconds string variables are needed for parsing the time data into separate digits.

If the strMinutes is 2 characters long, then the ten character of the string can be extracted by strMinutes.substring(0, 1); If this is not the case, there are only single minutes, and the ten character can be set to “0”. This same logic is applied to the strSeconds. In essence this will turn 182 seconds into four characters, “0”, “3”, “0”; and “2”.

In the LCD screen, this is handled as follows:

lcd.setCursor(0, 3);
lcd.print("Remaining: ");
if (minutes < 10) {
  lcd.setCursor(11, 3);
  lcd.print("0");
  lcd.setCursor(12, 3);
  lcd.print (String(minutes));
} else {
  lcd.setCursor(11, 3);
  lcd.print (String(minutes));
}
lcd.setCursor(13, 3);
lcd.print(":");
if (seconds > 9) {
  lcd.setCursor(14, 3);
  lcd.print (String(seconds));
}
else {
  lcd.setCursor(14, 3);
  lcd.print("0");
  lcd.setCursor(15, 3);
  lcd.print (String(seconds));
  lcd.setCursor(16, 3);
  lcd.print (" ");
}

To get this to show as the LED numbers, the program calls another function:

send7seg(led10min.toInt(), led1min.toInt(), led10sec.toInt(), led1sec.toInt());

As you can see, the strings are turned into integers, because the LED display function can take in integers as bytes, but not strings as such. This is the actual LED function:

void send7seg(int myX, int myY, int myZ, int myQ) { //sends "0.00" to 7 seg display  digitalWrite(_7segLATCH, LOW); //Tells all SRs that uController is sending data
  delay(10);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myQ]);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myZ]);//numeral 0 with dp
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myY]);
  shiftOut(_7segDOUT, _7segCLK, LSBFIRST, numTable[myX]);
  delay(10);
  digitalWrite(_7segLATCH, HIGH);}

So, when the countdown function runs, it first sends out the LCD data, and then does the same with the LED data. This video shows you what happens:

And this is the result. All that remains is the 3D printed chassis for this, and some real world testing, but I don’t think either of them will turn into issues. I will upload all the code and the STL files for printing up on my GitHub account some time in the near future.

I hope you enjoyed this presentation and can use it as a starting point for your own forays into IoT devices.