Making a Clap-Sensing Web Thing

The Project Things Gateway exists as a platform to bring all of your IoT devices together under a unified umbrella, using a standardized HTTP-based API. We recently announced the Things Gateway and we’ve started a series of hands-on project posts for people who want to set up a Gateway and start playing around with the Web of Things. Earlier this month we began with a high-level overview of how to build a Gateway add-on.

In this post we’ll take what we’ve learned so far and build a real add-on for the Gateway. This add-on will provide a clap-sensing Web Thing that we can use to control our lights and other devices.

Clap Sensing Web Thing

For this walkthrough we’ll be writing our Gateway add-on in JavaScript with the goal of making our lights clap-activated. This requires a microphone as input, so I’ll be plugging one into my Gateway and setting it up as a clap-sensing Web Thing. If you want to follow along you can plug any basic USB microphone into your Raspberry Pi. The eLinux-verified USB sound cards are all good options for gaining a microphone input.

We’ll start off with evaluating what kind of microphone add-on we want to create, then copy the example add-on to use as a skeleton for our code. Next, we’ll flesh out that skeleton with a bit of code and finish by testing out the add-on with the Gateway.

Outlining our add-on

We need to think about what we want as the “device” our add-on’s adapter provides. While our add-on could provide an adapter that handles multiple devices, we can simplify our work because we only have one device: the microphone listening for claps. We’re effectively using the microphone as our “clap sensor” device. Next, we have to consider how we want to perform the clap detection. A quick search of npmjs.com for libraries comes up with clap-detector, an open source library developed by Thomas Schell. Our adapter can include this library to use the microphone as a clap sensor.

We now can consider whether our idea of a clap sensor fits into any existing Web Thing types. An important part of the Web of Things specification is that each Thing can have a device type which tells the Gateway and other consumers of the API the properties, actions, and events available. For example, the type dimmableLight represents a light that can be turned on and off or dimmed. From the list of Web Thing types, it looks like a binarySensor with its active and inactive states is a great fit for the clap sensor’s clapping and silent states.

Writing the code

We can now begin the implementation. We can start our development on any computer with the Gateway software installed. If you have a Raspberry Pi flashed with the 0.3.0 release, you can log into it by following these instructions, and follow along from the ~/mozilla-iot/gateway/src/addons directory. Otherwise, if you have the Gateway software setup installed locally according to the instructions in the GitHub project page, you can follow along from the src/addons directory in that copy instead.

The second step is to download a copy of the example-adapter code into this src/addons directory so we can edit an existing adapter instead of writing everything from scratch. On the Pi, we can do this by making sure we have git installed then cloning the example-adapter repo.

cd ~/mozilla-iot/gateway/src/addons
sudo apt install git
git clone https://github.com/mozilla-iot/example-adapter

Once this step is completed, we should have an example-adapter directory in src/addons. We can now move on to getting the clap-detector library to update the active property of our clap sensing binarySensor. The only file we need to edit is example-plugin-adapter.js in the example-adapter directory.

First, let’s remove some of the generic-ness of the example adapter. We’re going to be doing a lot of renaming, so now is the time to use the Find-and-Replace function of your editor if you know how. We really only want one property–whether the sensor is active–so let’s rename ExampleProperty to ActiveProperty.

class ActiveProperty extends Property {

We also know that we want ExampleDevice to only ever be a clap sensor, so let’s rename it to ClapSensor. Note that we also change ExampleProperty to ActiveProperty in the ClapSensor‘s constructor and ExampleDevice to ClapSensor in addDevice.

class ClapSensor extends Device {
  constructor(adapter, id, deviceDescription) {
    super(adapter, id);
    this.name = deviceDescription.name;
    this.type = deviceDescription.type;
    this.description = deviceDescription.description;
    for (var propertyName in deviceDescription.properties) {
      var propertyDescription = deviceDescription.properties[propertyName];
      var property = new ActiveProperty(this, propertyName,
                                         propertyDescription);
      this.properties.set(propertyName, property);
    }
  }
}

Our final bit of modification for now is to make sure we’re making a binarySensor in the loading function. Therefore, we update the uses of ExampleDevice in addDevice and loadExamplePluginAdapter as follows:

class ClapSensorAdapter extends Adapter {
  // ...
  addDevice(deviceId, deviceDescription) {
    return new Promise((resolve, reject) => {
      if (deviceId in this.devices) {
        reject('Device: ' + deviceId + ' already exists.');
      } else {
        var device = new ClapSensor(this, deviceId, deviceDescription);
        this.handleDeviceAdded(device);
        resolve(device);
      }
    });
  }
  // ...
}
function loadExamplePluginAdapter(addonManager, manifest, _errorCallback) {
  var adapter = new ExamplePluginAdapter(addonManager, manifest.name);
  var device = new ClapSensor(adapter, 'clap-sensor-0', {
    name: 'Clap Sensor',
    type: 'binarySensor',
    description: 'Clap Sensor',
    properties: {
      active: {
        name: 'active',
        type: 'boolean',
        value: false,
      },
    },
  });
  adapter.handleDeviceAdded(device);
}

Now let’s figure out how clap-detector works. Based on clap-detector‘s documentation we have to install sox and clap-detector before we can begin using it in our adapter.


cd ~/mozilla-iot/gateway/src/addons/example-adapter
sudo apt-get install sox
npm install --save clap-detector

Then, we can use its documentation’s API example to get whether clapping is occurring:


const clapDetector = require('clap-detector');

// Start clap detection
clapDetector.start();

// Register on clap event
clapDetector.onClap(function(history) {
    console.log('clapping is happening', history)
});

We can then hook this up to our ActiveProperty by telling it to toggle its state whenever it detects clapping.


var clapDetector = require('clap-detector');
// Start clap detection
clapDetector.start();

class ActiveProperty extends Property {
  constructor(device, name, propertyDescription) {
    super(device, name, propertyDescription);
    this.unit = propertyDescription.unit;
    this.description = propertyDescription.description;
    this.setCachedValue(propertyDescription.value);
    this.device.notifyPropertyChanged(this);

    clapDetector.onClap(function() {
      console.log('clap!');
      this.value = !this.value;
      this.setCachedValue(this.value);

      this.device.notifyPropertyChanged(this);
    }.bind(this));
  }
  // ...
}

Clapping for lights

We’re done with the code part of this project. Now all we need to do is make sure our clap-sensing version of example-adapter installs and does what we want it to do. First, let’s restart our Gateway to make sure it cleanly loads our add-on. Run sudo systemctl restart mozilla-iot-gateway.service and wait until your Gateway restarts. Now we can verify that the add-on is working by going to our Gateway’s Settings screen. If you don’t see the add-on listed in the Settings screen, run tail -n +0 -f ~/mozilla-iot/gateway/run-app.log for logs that you can read to find out what went wrong:

Adapter List

Next, we add the ClapSensor device by clicking on the plus sign on the main Things page of our Gateway and saving the binarySensor named “Clap Sensor” that shows up:

Adding the Clap Sensor

Now we get to test our device by clapping near the microphone. If it’s working as intended, the sensor should turn on or turn off every time you clap. Otherwise, try adjusting the clap sensor configuration or compare your version to the official ClapSensor code.

Once everything is working we can get creative and set up a rule to turn on and off our lights every time we clap. Go to the Rules page in your Gateway and click the plus sign to add a new rule. On the bottom devices list, select your clap sensor and drag it into place as a trigger. Select “on” as the trigger’s property so that it triggers every time the sensor is active instead of when it is inactive. Next, drag whichever light you want to control into the rule area’s effect section. Select that you want to turn the light “on”. If you’re having any trouble, the completed rule is shown below for reference. We can now clap to turn on and off our light:

Clap Rule

Thank you for reading! If you want to learn more, check out the main Mozilla IoT page for more information about the Web of Things and how you can contribute!

About James Hobin

Level 25 Computer Wizard on a quest to keep the Internet of Things free and open.

More articles by James Hobin…


15 comments

  1. Rei Vilo

    The project turns the light on with a clap of the hands.

    But does it turn the lights off on a second clap?

    The idea would be off- clap – on – clap – off – clap – on – …

    February 22nd, 2018 at 14:21

    1. James Hobin

      Yes, the device senses the clap and uses that to toggle its “on” property. This makes my calling it a “clap sensor” a bit of a misnomer since it’s actually a toggle-upon-clap sensor if anything but makes the clap to turn on then turn off work well. Eventually I want to support a “clap” event and “toggle” action that would create this behavior in a much more intuitive way.

      February 22nd, 2018 at 14:39

  2. Thomas Schell

    Hi James, great article ! Many thanks for mentioning my name along with my package. I’m glad you found it useful.

    February 23rd, 2018 at 10:22

    1. James Hobin

      Thanks for writing such a quality library! It was great to have it work perfectly out of the box.

      February 23rd, 2018 at 10:54

  3. noor

    can you spek arapac

    March 20th, 2018 at 04:14

    1. James Hobin

      No, sorry.

      March 20th, 2018 at 08:19

  4. james

    that is great

    March 20th, 2018 at 07:03

    1. James Hobin

      Thanks!

      March 20th, 2018 at 08:20

  5. Ben

    Great job! I wonder if you could do a similar thing but voice activated. You could even implement some of this to clap for the computer’s attention!

    March 22nd, 2018 at 01:45

    1. James Hobin

      Thanks! Right now our voice interface works by clicking a button manually but we have some great experiments in the work with wake-word detection and other fancy features. One of our goals in this area is to expose our voice controller as a Web Thing so that if someone wants to clap to get the computer’s attention it’s just a matter of creating a rule.

      March 22nd, 2018 at 15:22

  6. priya

    such a wonderful job james

    March 22nd, 2018 at 04:17

  7. Matteo

    Hi there! Good article. Unfortunately i couldn’t manage to get the addon working. Not even the example-adapter is recognized, even though everything looks good. I’m on MacOS, i think there could be something wrong with this.
    Addon folder is not even there, I had to manually create it.
    Any idea?
    Thanks a lot

    March 22nd, 2018 at 06:31

    1. James Hobin

      Oh no, sorry that you’re encountering issues! I think the problem you’re running into is that you downloaded the latest version of the Gateway off of GitHub. This post was written for the Gateway’s 0.3 release which is our stable version. We recently introduced a change to where the Gateway stores add-ons, moving from inside the gateway’s directory to `~/.mozilla-iot/addons` folder. This folder is accessible through terminal or through Finder’s Go > Go to Folder option.

      tl;dr: On the 0.4.0 version of the Gateway, `gateway/src/addons` is instead `~/.mozilla-iot/addons`

      March 22nd, 2018 at 15:28

  8. Ashwin

    Great article James.

    One question: Can we run gateway on a windows PC?

    March 22nd, 2018 at 18:16

    1. James Hobin

      Yes, we have support for windows thanks to an awesome community contribution in https://github.com/mozilla-iot/gateway/pull/731. Unfortunately I’m not sure what the equivalent of the addons directory is on windows. It should be something like `$HOME/.mozilla-iot/addons`. Good luck!

      March 22nd, 2018 at 18:38

Comments are closed for this article.