Author Archives: Tim Stableford

PCF8574 Arduino Keypad With Interrupt

Interfacing with a keypad takes a lot of IO doesn’t it? There isn’t really space to do much else on an Arduino of ESP8266 after having 7 or 8 pins taken for a keypad. Luckily there’s a cheap solution for this one, using an I2C I/O expander. In this case the PCF8574AP· The ‘A’ being the alternate address version.

The first part to do, if it’s not supplied with your keypad is to figure out which pins connect to which keys. This is nearly always done in a grid configuration with rows and columns. To read this configuration you set all the pins to high/input-pullup except for a single column and then see which row is pulled low.

To figure out my keypad layout I attached a multi-meter to each of the pins on the keypad while in buzzer mode and noted which pins correlated to which button. Then it’s just a matter of finding the common theme and making your grid as below.

Keypad pin out claculator

Most keypads don’t tend to have such a complicated pin-out and will often group the columns together.

Next step is to wire it all together, for this you will need:

  1. 2x 4.7kOhm Resistors
  2. Arduino Uno/Due
  3. Keypad
  4. PCF8574
  5. Misc cables
  6. (Optional) Breadboard

Then wire it up to the PCF8574 as follows:

PCF8574 Pin out

PCF8574 Pin Arduino Pin Keypad Pin
SDA/15 A4 (and 5V through 4.7k resistor)
SCL/14 A5 (and 5V through 4.7k resistor)
INT/13 2
VDD/16 5V
VSS/8 GND
INT/13 2
P[0..6] [0..6]

After that it should like something like the following:

PCF8574, keypad, and Arduino connected together

Then the last step, the code:

#include <Arduino.h>
#include <Wire.h>

// This is the address for the 'A' variant with all address pins not connected (high).
// Check the datasheet for other addresses. http://www.ti.com/lit/ml/scyb031/scyb031.pdf
#define PCF_ADDR (0x27)
#define INTERRUPT_PIN (2)
#define DEBOUNCE_DELAY (200) // milliseconds

bool keyPressed = false;
int32_t lastPress = 0;

// Interrupt called when the PCF interrupt is fired.
void keypress() {
  if (millis() - lastPress > DEBOUNCE_DELAY) {
    keyPressed = true;
  }
}

// The pins for each of the columns in the matrix below. P7 to P0.
uint8_t kColumns[] = { B00000100, B00000001, B00010000 };
uint8_t kRows[] = { B00000010, B01000000, B00100000, B00001000 };
char kMap[][3] = {
  { '1', '2', '3'},
  { '4', '5', '6'},
  { '7', '8', '9'},
  { '*', '0', '#'}
};
// This is the value which is sent to the chip to listen for all
// key presses without checking each column.
uint8_t intMask = 0;

// The PCF chip only has one I2C address, writing a byte to it sets the state of the
// outputs. If 1 is written then they're pulled up with a weak resistor, if 0
// is written then a transistor strongly pulls it down. This means if connecting an LED
// it's best to have the chip as the ground terminal.
// If passing a byte value to this, eg B11111111 the left most bit is P7, the right
// most P0.
void write8(uint8_t val) {
  int error = 0;
  // This will block indefinitely until it works. This would be better done with a
  // timeout, but this is simpler. If the I2C bus isn't noisy then this isn't necessary.
  do {
    Wire.beginTransmission(PCF_ADDR);
    Wire.write(val);
    error = Wire.endTransmission();
  } while(error);
}

uint8_t read8() {
  uint8_t read = 0;
  do {
    read = Wire.requestFrom(PCF_ADDR, 1);
    if (read == 1) {
      return Wire.read();
    }
  } while (read != 1);
}

void setup() {
  Serial.begin(115200);

  Wire.begin();
  // When a pin state changed on the PCF it pulls its interrupt pin low.
  // It's an open drain so an input pullup is necessary.
  pinMode(INTERRUPT_PIN, INPUT_PULLUP);
  attachInterrupt(digitalPinToInterrupt(INTERRUPT_PIN), keypress, FALLING);

  // Calculate the interrupt mask described above.
  for (uint8_t c = 0; c < sizeof(kColumns) / sizeof(uint8_t); c++) {
    intMask |= kColumns;
  }
  intMask = ~intMask;
  write8(intMask);
}

// This goes through each column and checks to see if a row is pulled low.
char getKey() {
  for (uint8_t c = 0; c < sizeof(kColumns) / sizeof(uint8_t); c++) {
    // Write everything high except the current column, pull that low.
    write8(~kColumns);
    uint8_t val = read8();
    // Check if any of the columns have been pulled low.
    for (uint8_t r = 0; r < sizeof(kRows) / sizeof(uint8_t); r++) {
      if (~val & kRows[r]) {
        return kMap[r];
      }
    }
  }
  return '\0';
}

void loop() {
  if (keyPressed) {
    char key = getKey();
    // Key may not be read correctly when the key is released so check the value.
    if (key != '\0') {
      Serial.println(key);
      lastPress = millis();
    }
    keyPressed = false;
  }
  // After getKey this needs to be called to reset the pins to input pullups.
  write8(intMask);
}

When uploaded and everything wired together the sketch will output the pressed key to the serial console.

Lightweight Multi-language RPC

Many times have I found myself needing to communicate with an Arduino over serial, often wanting to send complex data types. Often too simpler types such as a 4 byte int. Then there is the code necessary to figure out when you’ve started sending that int and what to do with it. Then what if you want to send more than one int?

I’ve implemented an asynchronous RPC library for Arduino, and C++/Java on PC. It allows invoking remote methods and passing complex data types over serial. I’m sure has been done before, this library though attempts to find a middle ground between minimal and usable. There are limitations, such as a lack of return types, limitations on packet sizes (65535) and string sizes (255), it should suite most purposes though.

The Ping-Pong Example

Running The Examples

For running the examples I’m going to assume the host is a Linux environment (tested on Ubuntu 16.04).

First download and install the Arduino library.

mkdir -p ~/Arduino/libraries
git clone https://github.com/timstableford/LRPC-Arduino-C ~/Arduino/libraries/RPC

Then you’ll need to restart the Arduino IDE. Following that, in the Arduino IDE under File -> Examples -> Examples from Custom Libraries -> RPC you will find rpc_ping. It will give a few compiler warnings about shifting bytes, that’s expected. Upload it and you’ll be good to go.

Next install and run the Java example that matches the sketch.

cd ~/Documents
git clone https://github.com/timstableford/LRPC-Java
cd LRPC-Java
./gradlew runPing

Arduino Client Sketch

On my Github I’ve put up the Arduino library, which includes a ping pong example. This sketch waits for a ping RPC call from the server (Java) and responds with a pong RPC call. The following paragraphs will do a breakdown of some important points in the code.

The serialReader and serialWriter functions are simple wrappers around Serial read and write, these are here so the library could be used over alternate communication mechanisms, such as I2C.

The next item of importance is the RPC function declaration table.

RPC::RPCContainer rpcs[] = {
  { 1, rpcPing, NULL },
  { 2, rpcPong, NULL }
};

This lists RPC function ID’s, pointers to the functions, and some user data you may wish to pass to the function, in this case it’s NULL because we don’t want to. Inside these functions is a reference to an Object, this contains various functions for getting and setting data. In RPC callbacks, the 0th item is the function ID and any items after are user data. In this case the 1st item is the time in milliseconds as an int64.

There’s a few different ways to respond to RPC calls if you dig deep enough you can construct Objects manually. However, this library includes a wrapper similar to printf that supports many data types. A list of supported data types is near the end of this post.
An example is:

rpc.call(2, "m", (int64_t)millis() + time_offset);

The first item is the function ID to call. The second item is like the printf format specifier. In this example though all types are dictated by a single character for ease of parsing. “m” is the type specifier for an int64. Following the format specifier is a list of data items to send.

Skipping over the constructors because they’re mostly the same in all sketches. You may vary the buffer size, depending on the size of the RPC calls you’re making to the Arduino sketch.

The final important part to making it all work is

while(parser.parse() >= 0);

Each call to parser.parse reads one byte. I recommend doing it like this, it will return a number less than 0 when there are no bytes left for it to read. If you have other time sensitive operations though, it would be possible to limit the number of bytes to read in a single loop.

Java Example

I’ve also put a Java example on Github that connects over serial to the Ardunio sketch and sends regular ping commands. This is a little more verbose than the Arduino sketch because it contains multiple threads. The general flow is the same though. The biggest difference is how RPC calls are made.

rpc.call(PONG_FID, new LSerializer(LObjects.Int(LType.INT64, System.currentTimeMillis())));

An LSerializer object is the equivalent of an Object class in the Arduino sketch, it handles most of the serialising and de-serialising. However, type specific code is handled in the LObjects class. This is a far easier way to add additional types, but adds a lot of overhead. So interfaces are avoided in the C++/Arduino implementation.

Packet Format

The RPC protocol has two parts. A low-level packet layer called Stream Parser which has a header containing information about how much data is to be received. It buffers all that data and then passes it along to a registered callback dependant on the packet type specified in the header. For instance, RPC is 8.

The other portion of the called called RPC/Object takes a buffer and parses it into a more accessible form, avoiding data duplication where possible. The Object class serialises and de-serialises data, whereas the RPC class routes incoming RPC calls to the correct functions, and in the C++ implementation provides a wrapper around creating objects using a printf like format.

Stream Parser

A stream parser packet consists of 6 bytes, all data types are in network byte order. The first two bytes are a uint16 containing the packet type, the second two bytes are another uint16 dictating the size of the packet to be received, and the final two bytes are a 32-bit crc of the first 4 bytes. This means a header can be picked up at any point in a stream. After reading the header, the stream parser attempts to read all of the data bytes. If successful it gets passed along through a callback registered to the type.

NameSize (bytes)TypeDescription
type2uint16The type of the incoming packet.
size2uint16The size of the data to receive after the header.
crc2uint16A CRC 32 of the type and size.

Object

The Object class is a way to transfer different data types while preserving what sort they are and providing a wrapper to easily set and get items from a buffer. The first piece of data in a serialised Object is an unsigned byte which specifies the number of items to be transmitted. For instance two ints would be ‘2’, one string would be ‘1’. (Not encoded as characters.) After that, there’s a 1 byte type specifier for each piece of data to be transmitted. For data types with fixes size, that’s all. For data types of variable size such as strings there’s an additional table after the type specifier that specifies sizes. Currently there’s a maximum string length of 255. After that is just plane data, in network byte order where applicable.

NameSize (bytes)TypeDescription
item_count1uint8A count of the data items in the object.
data_type1 (per data item)uint8Type specifier's for each data item.
string_size1 (per array/string item) [0 if no strings]uint8Specifies the length of a string/array element.
datavariablevariableSerialized data, ints are in network byte order, strings are raw.

RPC

RPC packets are an Object class. The first piece of data is a uint16 to specify the RPC function ID. The rest is the encoded data.

The table below lists the type specifiers for rpc.call:

CharacterData type
sString
cint8
Cuint8
dint16
Duint16
lint32
Luint32
mint64
Muint64
ffloat

 

 

Video Streaming With Zebkit JS UI

Following on from my previous post, so what if I now want to make a nice client side, but web-based application for playing videos. We can avoid using HTML for this purpose because we don’t need any search engine indexing, this means we get to use HTML5 canvas’s and a UI framework to speed up development.

Zebkit happens to be my favourite Javascript Canvas UI framework, it’s pretty quick to develop with and has most of the features of a standard desktop UI framework. What I found though is the video streaming is a bit lackluster, sure the support is technically in there but it made me long for the wonderful on-screen controls of the HTML5 video tag.

So then we can’t use the video tag in full as far as I know within a canvas, we could have a UI canvas element and a separate video element but that’s just messy. Although the video panel included with Zebkit doesn’t support drawing overlays, luckily they’ve made a very extensible API. So what we can is extend the existing video panel, overwrite the draw method and throw in some simple controls, such as seeking, pausing etc. The result of which is below…

 

The source for this Zebkit plugin can be found on Github, here.

HTML5 Split Video Streaming

Recently I’ve wanted to look into how video streaming of local content can be done with NodeJS and HTML5’s video tag. I’ve only found one example to base upon, here. This demo though didn’t really cut it for me, for one it grabbed the file and split it in the client Javascript and then appended it to the video. Ideally, this would be done server side. It also has another bug where the split file doesn’t append at a time offset, and overwrites the original buffer.

I’ve created a demo that addresses these issues by creating a server in NodeJS that scans a videos directory and transcodes and splits the videos on the fly with ffmpeg. And uses mse_webm_remuxer to fix the format of these files in some magical way to make them compatible with HTML5’s MediaSource.

The demo below shows the client side code, it can be found here. This is a client-side only demo of sorts, in that it mimics the NodeJS server statically because the server (being a demo) is incredibly insecure.

So, how the whole process works:

  1. The client makes a request to video/ to see what videos are available.
  2. This propogates a list in the client, the user then clicks a video.
  3. Upon clicking a video, the client makes a request for the videos metadata, returning data such as duration, chunk size and information on the streams.
  4. The client then sets up the video, mostly the duration and requests the first chunk.
  5. The server would then get the source video and transcode the first 30 seconds to a webm format and run mse_webm_remuxer on it, sending the URL of this new file to the client.
  6. The client then buffers this chunk at the correct location in the video.
  7. As the client gets to different points new chunks are transcoded and buffered, allowing jumping around the video and transcoding the necessary sections on the fly.

Video chunks? But why?

There’s a few benefits to using video chunks, such as how YouTube does it. The primary advantage to YouTube I suspect is bandwidth. By only buffering sections of the video they can control how much data to send to the client in advance, this saves a heck of a lot of data if they don’t buffer the whole thing without quitting.

The reason for chunks in this demo though is timing. To transcode a 320×240 30s length of video it will take about 15s, that means the user can start viewing fairly quickly, rather than waiting for a full transcode operation to complete. By measuring the time it took to receive a video chunk in the client side, the program can know when to request the next chunk to ensure smooth playing.

Source Code

The source code for this demo has been put up on my Github, here.

Finished Dissertation

I’ve finally finished my dissertation and done my final presentation. I’ve previously detailed what the dissertation involved here. And this post is just going to be some additional documentation, and my dissertation.

I’ve created several Github repositories making all work public. The source code is split into three sections: the code for the radio board, the code for the sensor board and the configuration program. All can be found under source/code in the main Github repository:

https://github.com/timstableford/Dissertation

The dissertation itself can be found here and in the Github repository.