Standard service - a fixed amount of work that is performed at a fixed price.  Post Service

  

Sunday, 15 September 2019 11:52

Chirp Chirp… transmitting data via sound – now 2.0!

Written by  https://cxlabs.sap.com/2019/09/16/chirp-chirp-transmitting-data-via-sound-now-2-0/
Rate this item
(0 votes)

Back in 2013 (!), we’ve experimented with data transmission via sound. We called this prototype Stream and it was super well received

at that time. Stream used ultrasonic sounds embedded in video files to transmit the IDs of products, which were then pulled from SAP Commerce and displayed on a tablet screen. Take a quick look, it’s really a bit like magic! It was (and is) a great example for a highly relevant Customer Experience technology that connects brands and their products to consumers – in a very innovative way!

Roughly 6 years later, the startup library we used at that time is probably long gone. But there is a great new company out there that offers even more options for ‘data-over-sound’ applications: the UK-based Chirp.

The purpose of this blog post to quickly introduce this exciting technology to you and to reason about potential applications in the Customer Experience space. As always, please share your ideas via comments or simply via Twitter! To make this a bit more fun, I’ve created a bunch of quick demos that should be fun to watch.

Data-over-Sound – the basics

Before we dive deeper, why would one choose data to communicate between devices? There are many options out there create a link between devices – scan via BLE/iBeacons, present QR codes or NFC Tags, via geofencing, data-over-light (like Electric Imp), etc. – so data-over-sound is definitely just one way to create that linkage. These are the characteristics:

  • Data-over-Sound can use audio files in the hearable sound frequencies or ultrasonic frequencies, which most humans cannot hear. Using ultrasound, not all device / application combinations are easily supported. For example, the Chirp WebAssembly SDK is only interoperable with the 16kHz-mono frequency, which is in the hearable audio spectrum. For ultrasonic on a mobile smartphone, you’ll have to go the native way.
  • It typically takes a few seconds to transmit a few bytes. This means data-over-sound is typically used to share identifiers: IDs to shared spaces (virtual game rooms, physical shopping locations, etc.), IDs to services or products, the ID of a service ticket, the ID to a payment transaction… there are also protocol restrictions which depend on the frequency chosen.
  • There are SDKs for the web (Javascript send-only, WebAssembly), native (Android, iOS, also cross-platform via Flutter), Python3 libs and python-based command-line tools that are compatible with all major OSes. Also, after a bit of experimentation, I was able to use most of them pain-free.

One recent new SDK caught my attention: Chirp for Arduino! There is now an official Chirp SDK for Arduino, which of course now makes it even easier to add data-over-sound features to physical creations.

Demo: sending IDs in all kind of flavors

As a quick showcase, I came up with the idea of sending product IDs via sound. I decided for a smartphone, camera and laptop as the products and simply made up an imaginary ID (e.g. 01/02/03). The sounds you hear in the videos used the 16kHz-embedded-mono profile, because that’s the one used on the Arduino and it happes to be compatible with the Javascript and WebAssembly SDKs, too.

Sender: Bare Conductive Touch Board
Receiver: Arduino Nano 33 BLE Sense

I found it a bit boring to go the easy way and just use the Javascript SDK initially, so I first decided to send my audio signals via a Bare Conductive Touch Board initially. After you have setup your system for your specific Chirp application (chirp key, secret and config have to be written to .chirprc) and you have installed the Chirp command line tools, I generated the audio files similar to this:

chirp-audio-write -x 01020304 output.wav

The result being a .wav file, I needed to convert the wav file to an mp3 for the Bare Conductive Touch Board via Audacity and the Lame mp3 encoder.

Note: unfortunately instagram misses some of the action due to the square format of the video – please enable sound and also make sure you view the full post/video.

Sender: Chirp command line on a MacBook Pro
Receiver: Arduino Nano 33 BLE Sense

Happy with the results, I was keen to spin it further. So next I replaced the Touch Board with a pure command line audio-generation. The receiving part stayed the same, the Arduino with built-in mic.

Sender: Chirp Javascript SDK in a localhost-served HTML-page
Redeiver: Arduino Nano 33 BLE Sense

While the command line is fine for demos, a more realistic use case is obviously to generate dynamic “chirps” in a web page. For example, an in-store product display would emit the sounds to help customers identify products or link to games/vouchers being advertised. Some interesting learning here: the usage of the Chirp JS SDK will fail, if it is not initialised with a customer action (e.g. a button press linked to the init function).

Sender: Touch Board, Web-Page with Chirp JS SDK, sound files…
Receiver: Chirp WebAssembly SDK in a locally served HTML-page

After so much fun and successes with the various Chirp SDKs, I had to give it a final spin. I ended the testing by integrating their WASM SDK , which allows you to listen to chirps and to decode the chirps right in the browser. Obviously this will ask for the microphone permission, which needs to be granted.

Ultrasonic frequencies – not with your browser

As quickly mentioned above, to play in the ultrasonic frequencies range, you’ll have to go native. Neither the Arduino C SDK nor the WASM SDK for the web will allow you to receive chirps via the ultrasonic (non-hearable) frequencies. For the web, this seems to be due to how the browsers pass the mic input down to the Web Assembly script and it is most likely a security setting which disables audio-tracking.

Chirp/Flutter integration failed on me

Needless to say, I wanted to try the Flutter integration which would have allowed me to try out the ultrasonic frequencies. Sadly, with my installation of the latest iOS/Android toolchains and the latest Flutter 1.9, I hit a hard stop here. The application compiles and starts up fine on my Pixel 3 Android Q, but fails upon receiving or generating a chirp. I ended stuck on this issue here, hopefully they come back to me on this quickly.

Applications and Use Cases for CX

I want to end this blog post by ideating a bit about the potential use cases for Customer Experience. While we’ve shown the applicability already in 2013 with the Stream Prototype, I think there might be many more both on the B2C and B2B area.

In the B2C IoT area, I can well imagine setting up consumer facing devices such as smart speakers or home hubs via sounds. While this is typically done by scanning and joining ad-hoc Wifi networks, the data-over-voice features of Chirp would allow the consumer to stay on the current WiFi network and not switch back and forth between both. Similar solutions are of course already applied via BLE or NFC.

With the widely played game Roblox using data-over-sound by Chirp, it seems all fun & engaging end-user interactions that depend on some kind of shared data such as a channel id (for chatting, for receiving vouchers, etc) could be a good fit, too. In the end, the chirps are pretty nice to listen to :-)

One thing to keep in mind is that for receiving data on a web or native application, the microphone permission needs to be granted. Due to this, the perfect sweet spot for this technology might even be in the B2B or professional area, where let’s say handymen have to check-in to a work setting. I’d also argue that there are environments, which have a totally polluted 2.4 GHz frequency spectrum (Wifi, BLE) where sound is an interesting alternative for data transmission. Keep in mind that for my testing I’ve used the hearable chirps, but especially with native Apps and custom hardware ultrasonic frequencies should not be an issue.

If you have more great ideas, please really let me know via Twitter or just leave a comment! I hope you enjoyed this trip into the data-over-sound space.

Read 73 times

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.