{"id":296,"date":"2016-11-08T12:15:58","date_gmt":"2016-11-08T12:15:58","guid":{"rendered":"https:\/\/tstableford.co.uk\/wordpress\/?p=296"},"modified":"2016-11-08T12:16:13","modified_gmt":"2016-11-08T12:16:13","slug":"lightweight-multi-language-rpc","status":"publish","type":"post","link":"https:\/\/tstableford.co.uk\/wordpress\/?p=296","title":{"rendered":"Lightweight Multi-language RPC"},"content":{"rendered":"

Many times have I found myself needing to communicate with an Arduino over serial, often wanting to send complex data types. Often\u00a0too simpler types such as a\u00a04 byte int. Then there is the code necessary to figure out when you’ve started sending that int and what to do with it. Then what if you want to send more than one int?<\/p>\n

I’ve implemented an asynchronous RPC library for Arduino, and C++\/Java on PC. It allows invoking remote methods and passing complex data types over serial. I’m sure has been done before, this library though attempts to find a middle ground between minimal and usable. There are limitations, such as a lack of return types, limitations on packet sizes (65535) and string sizes (255), it should suite most purposes though.<\/p>\n

The Ping-Pong Example<\/strong><\/h2>\n

Running The Examples<\/em><\/h3>\n

For running the examples I’m going to assume the host is a Linux environment (tested on Ubuntu 16.04).<\/p>\n

First download and install the Arduino library.<\/p>\n

\r\nmkdir -p ~\/Arduino\/libraries\r\ngit clone https:\/\/github.com\/timstableford\/LRPC-Arduino-C ~\/Arduino\/libraries\/RPC\r\n<\/pre>\n
Then you’ll need to restart the Arduino IDE. Following that, in the Arduino IDE under File -> Examples -> Examples from Custom Libraries -> RPC you will find rpc_ping. It will give a few compiler warnings about shifting bytes, that’s expected. Upload it and you’ll be good to go.<\/p>\n
Next install and run the Java example that matches the sketch.<\/p>\n
\r\ncd ~\/Documents\r\ngit clone https:\/\/github.com\/timstableford\/LRPC-Java\r\ncd LRPC-Java\r\n.\/gradlew runPing\r\n<\/pre>\n
Arduino Client Sketch<\/em><\/h3>\n
On my Github I’ve put up the Arduino library<\/a>, which includes a ping pong example. This sketch waits for a ping RPC call from the server (Java) and responds with a pong RPC call.\u00a0The following paragraphs will do a breakdown of some important points in the code.<\/p>\n
The serialReader<\/em> and serialWriter<\/em> functions are simple wrappers around Serial read and write, these are here so the library could be used over alternate communication mechanisms, such as I2C.<\/p>\n
The next item of importance is the RPC function declaration table.<\/p>\n
\r\nRPC::RPCContainer rpcs[] = {\r\n  { 1, rpcPing, NULL },\r\n  { 2, rpcPong, NULL }\r\n};\r\n<\/pre>\n
This lists RPC function ID’s, pointers to the functions, and some user data you may wish to pass to the function, in this case it’s NULL because we don’t want to. Inside these functions is a reference to an Object, this contains various functions for getting and setting data. In\u00a0RPC callbacks, the 0th item is the function ID and any items after are user data. In this case the 1st item is the time in milliseconds as an int64.<\/p>\n
There’s a few different ways to respond to RPC calls if you dig deep enough you can construct Objects manually. However, this library includes a wrapper similar to printf that supports many data types. A list of supported data types is near the end of this post.
\nAn example is:<\/p>\n
rpc.call(2, "m", (int64_t)millis() + time_offset);<\/pre>\n
The first item is the function ID to call. The second item is like the printf format specifier. In this example though all types are dictated by a single character for ease of parsing. “m” is the type specifier for an int64. Following the format specifier is a list of data items to send.<\/p>\n
Skipping over the constructors because they’re mostly the same in all sketches. You may vary the buffer size, depending on the size of the RPC calls you’re making to the Arduino sketch.<\/p>\n
The final important part to making it all work is<\/p>\n
while(parser.parse() >= 0);<\/pre>\n
Each call to parser.parse reads one byte. I recommend doing it like this, it will return a number less than 0 when there are no bytes left for it to read. If you have other time sensitive operations though, it would be possible to limit the number of bytes to read in a single loop.<\/p>\n
Java Example<\/em><\/h3>\n
I’ve also put a Java example on Github<\/a> that connects over serial to the Ardunio sketch and sends regular ping commands. This is a little more verbose than the Arduino sketch because it contains multiple threads. The general flow is the same though. The biggest difference is how RPC calls are made.<\/p>\n
rpc.call(PONG_FID, new LSerializer(LObjects.Int(LType.INT64, System.currentTimeMillis())));<\/pre>\n
An LSerializer object is the equivalent of an Object class in the Arduino sketch, it handles most of the serialising and de-serialising. However, type specific code is handled in the LObjects class. This is a far easier way to add additional types, but adds a lot of overhead. So interfaces are avoided in the C++\/Arduino implementation.<\/p>\n
Packet Format<\/strong><\/h2>\n
The RPC protocol has two parts. A low-level packet layer called Stream Parser which has a header containing information about how much data is to be received. It buffers all that data and then passes it along to a registered callback dependant on the packet type specified in the header. For instance, RPC is 8.<\/p>\n
The other portion of the called called RPC\/Object takes a buffer and parses it into a more accessible form, avoiding data duplication where possible. The Object class serialises and de-serialises data, whereas the RPC class routes incoming RPC calls to the correct functions, and in the C++ implementation provides a wrapper around creating objects using a printf like format.<\/p>\n
Stream Parser<\/em><\/h3>\n
A stream parser packet consists of 6 bytes, all data types are in network byte order. The first two bytes are a uint16 containing the packet type, the second two bytes are another uint16 dictating the size of the packet to be received, and the final two bytes are a 32-bit crc of the first 4 bytes. This means a header can be picked up at any point in a stream. After reading the header, the stream parser attempts to read all of the data bytes. If successful it gets passed along through a callback registered to the type.<\/p>\n
Name<\/th>Size (bytes)<\/th>Type<\/th>Description<\/th><\/tr><\/thead>
type<\/td>2<\/td>uint16<\/td>The type of the incoming packet.<\/td><\/tr>
size<\/td>2<\/td>uint16<\/td>The size of the data to receive after the header.<\/td><\/tr>
crc<\/td>2<\/td>uint16<\/td>A CRC 32 of the type and size.<\/td><\/tr><\/table><\/p>\n
Object<\/em><\/h3>\n
The Object class is a way to transfer different data types while preserving what sort they are and providing a wrapper to easily set and get items from a buffer. The first piece of data in a serialised Object is an unsigned byte which specifies the number of items to be transmitted. For instance two ints would be ‘2’, one string would be ‘1’. (Not encoded as characters.) After that, there’s a 1 byte type specifier for each piece of data to be transmitted. For data types with fixes size, that’s all. For data types of variable size such as strings there’s an additional table after the type specifier that specifies sizes. Currently there’s a maximum string length of 255. After that is just plane data, in network byte order where applicable.<\/p>\n
Name<\/th>Size (bytes)<\/th>Type<\/th>Description<\/th><\/tr><\/thead>
item_count<\/td>1<\/td>uint8<\/td>A count of the data items in the object.<\/td><\/tr>
data_type<\/td>1 (per data item)<\/td>uint8<\/td>Type specifier's for each data item.<\/td><\/tr>
string_size<\/td>1 (per array\/string item) [0 if no strings]<\/td>uint8<\/td>Specifies the length of a string\/array element.<\/td><\/tr>
data<\/td>variable<\/td>variable<\/td>Serialized data, ints are in network byte order, strings are raw.<\/td><\/tr><\/table><\/p>\n
RPC<\/em><\/h3>\n
RPC packets are an Object class. The first piece of data is a uint16 to specify the RPC function ID. The rest is the encoded data.<\/p>\n
The table below lists the type specifiers for rpc.call:
\n
Character<\/th>Data type<\/th><\/tr><\/thead>
s<\/td>String<\/td><\/tr>
c<\/td>int8<\/td><\/tr>
C<\/td>uint8<\/td><\/tr>
d<\/td>int16<\/td><\/tr>
D<\/td>uint16<\/td><\/tr>
l<\/td>int32<\/td><\/tr>
L<\/td>uint32<\/td><\/tr>
m<\/td>int64<\/td><\/tr>
M<\/td>uint64<\/td><\/tr>
f<\/td>float<\/td><\/tr><\/table><\/p>\n
 <\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
Many times have I found myself needing to communicate with an Arduino over serial, often wanting to send complex data types. Often\u00a0too simpler types such as a\u00a04 byte int. Then there is the code necessary to figure out when you’ve started sending that int and what to do with it. Then what if you want […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/296"}],"collection":[{"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=296"}],"version-history":[{"count":14,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/296\/revisions"}],"predecessor-version":[{"id":310,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/296\/revisions\/310"}],"wp:attachment":[{"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tstableford.co.uk\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}