Firmata: strings sent from the computer become damaged after Arduino

Question

I have a Python program that acts as a Firmata client. This program connects to the Arduino at a user-specified COM-port (with Firmata onboard, the sketch can be seen below), sends the user-requested string message to the Arduino, and awaits Arduino's reply. If the reply is present, the program prints it out as a list of ASCII symbol codes.

I expect the program and the sketch to work this way (block scheme is below), but the strings coming from the Arduino are damaged. They don't contain all symbols which were sent, but they contain only the first symbols in some random order, with a \0 symbol present after each character in the string. For example,

String sent to the Arduino	Received as ASCII symbol codes	Received as a string
`A`	`(65, 0)`	`A\0`
`AB`	`(65, 0)` where is the `B`?	`A\0`
`ABC`	`(65, 0, 67, 0)` the `B` lacks presence	`A\0C\0`
`ABCD`	`(65, 0, 67, 0)` the received data length remains the same, but `B` and `D` aren't present	`A\0C\0`
`ABCD(10)`	`(65, 0, 67, 0, 40, 1, 48, 1)` WHAT IS THIS?	`A\0C\0(\10\1`
`DHTt(4)`	`(68, 0, 84, 0, 40, 0, 41, 1)`	`D\0T\0(\0)\1`

Program's screenshot:

Awaited work algorithm block scheme:

Python program code:

import pyfirmata, time
class CustomArduino(pyfirmata.Arduino):
    def init(self, port: str):
        pyfirmata.Arduino.init(self, port)
        self.iterator = pyfirmata.util.Iterator(self)
        self.iterator.start()
    self.add_cmd_handler(0x71, self.receiveString)


def receiveString(self, *args, **kwargs):
    print(f'Received: {args}')


def sendString(self, data):
    self.send_sysex(0x71, bytearray(data, 'utf-8'))


def send_sysex(self, sysex_cmd, data=[]):
    &quot;&quot;&quot;
    Sends a SysEx msg.
    :arg sysex_cmd: A sysex command byte
    : arg data: a bytearray of 7-bit bytes of arbitrary data
    &quot;&quot;&quot;
    msg = bytearray([pyfirmata.pyfirmata.START_SYSEX, sysex_cmd])
    msg.extend(data)
    msg.append(pyfirmata.pyfirmata.END_SYSEX)
    self.sp.write(msg)



print('STRING TESTER\nPlease input your COM port: ', end = '')
myCom = str(input())
device = CustomArduino(myCom)
while True: 
    device.sendString(input('Send: '))
    time.sleep(1)

Arduino sketch:

#include <Firmata.h>
void stringCallback(char myString)
{
  //The function accepts one charset.
  Firmata.sendString(myString); //send the same* charset  back
}
void sysexCallback(byte command, byte argc, byte *argv)
{
  Firmata.sendSysex(command, argc, argv);
}
void setup()
{
  Firmata.setFirmwareVersion(FIRMATA_FIRMWARE_MAJOR_VERSION, FIRMATA_FIRMWARE_MINOR_VERSION);
  Firmata.attach(STRING_DATA, stringCallback); //when there is a STRING incoming to the Arduino, call 'stringCallback'
  Firmata.attach(START_SYSEX, sysexCallback);
  Firmata.begin(57600);
  pinMode(13, OUTPUT);
}
void loop()
{
  while (Firmata.available()) {
    Firmata.processInput();
  }
}

How I can deal with this?

score 3 · Answer 1 · answered Feb 05 '21 at 14:28

You have to understand the Firmata protocol to write your own client library. The spec is here: https://github.com/firmata/protocol.

In this particular case, you need to know that data that is sent (the string in this case) can never use bit 7 of the data, since these values have special meanings. Therefore, Firmata sends every byte as two bytes, first bits 0-6 and second bit 7 as bit 0 of the next byte. Therefore, the conversion to UTF-8 is not sufficient. You need to do a two-byte-per-8-bits encoding. The same applies for the reply.

score 1 · Answer 2 · answered Feb 05 '21 at 15:02

1

Alternatively, if you want a system without Firmata's baggage of ancient MIDI constraints, like its bizarre 7-bit limit, you might consider DaqPort https://www.daqarta.com/dw_rraa.htm

answered Feb 05 '21 at 15:02

Boggyman

619
1
4
4

score 1 · Accepted Answer · answered Feb 05 '21 at 15:19

Instead of using the UTF-8 encoding,

def receiveString(self, *args, **kwargs):
    print(f'Received: {args}')
def sendString(self, data):
    self.send_sysex(0x71, bytearray(data, 'utf-8'))

it is necessary to use the pyfirmata special bytearray manipulation methods: two_byte_iter_to_str and str_to_two_byte_iter:

def receiveString(self, *args, **kwargs):
    print(f'Received: {pyfirmata.util.two_byte_iter_to_str(args)}')
def sendString(self, data):
    self.send_sysex(0x71, pyfirmata.util.str_to_two_byte_iter(data))

Firmata: strings sent from the computer become damaged after Arduino

3 Answers3

Linked