Build your own web things with the Things Framework

Last year Mozilla started Project Things to help developers connect physical “things” to the web in a safe, secure and interoperable way.

A diagram of the Things Framework

In February we announced the 0.3 release of the Things Gateway which bridges existing smart home devices to the Web of Things. Today we’re excited to tell you about the beginnings of the Things Framework which aims to help developers build their own “native web things”, and the 0.4 release of the Things Gateway which now supports a wider range of smart home devices.

Things Framework

The Things Framework is a collection of reusable software components to help you build your own web things, which directly expose the Web Thing API. That means they can be discovered by a Web of Things client or gateway, which can monitor and control them over the web.

An illustration of the Things Framework by Mozilla

A web thing has a Web Thing Description which describes the device’s capabilities, and exposes a Web Thing REST API and/or WebSocket API, so that it can be monitored and controlled. The Thing Description provides machine-readable metadata about a device and its available properties, actions and events. The Web Thing API lets a client read and write its properties, request actions and subscribe to its events.

You can get started today by turning Android things into web things using our Java web thing library, or if you prefer to build things with Python or NodeJS, we also have you covered there. We have some early examples of how to build web things using WiFi-enabled microcontrollers like the ESP8266, and a serial gateway adapter for chipsets with more constrained resources. We’re releasing these libraries at a very early stage of development so that you can provide us with feedback and help us to help you build better web things.

In the coming days we’ll be blogging about how to use each of these new web thing libraries, to help you get hands-on building your own devices.

These are still experimental technologies in the process of standardisation at the W3C, but we hope our early open source implementations will help developers try out the Web of Things and help us to improve it.

Add Web Things

With the 0.4 release of the Things Gateway, you can now add native web things to your gateway, to control them alongside all your other smart home devices. The advantage of native web things is that they don’t need a custom gateway adapter because they follow a common standard using existing web technologies.

The interface when "scanning for new devices" to add a Web Thing

Web things can broadcast their web address using mDNS or a Bluetooth beacon so that they can be discovered by the gateway, or they can be manually added by their URL. Simply click on the “+” button in the Things screen of the gateway and either allow it to scan for devices, or manually copy and paste a web thing URL using the “Add by URL…” link.

New Add-ons

The 0.4 gateway release also comes with a larger selection of add-on adapters which add support for smart home protocols like Apple HomeKit, and devices from LIFX and Broadlink.

New add-ons for the Things Gateway

You will also notice that, like the gateway itself, add-ons now get automatically updated so you’ll always have the latest version.

Other Changes

There are a host of other changes in the 0.4 release. You can now create rules based on numerical and color properties, we’ve added support for color temperature light bulbs and there’s a new configuration UI for add-ons. There are new developer features like Windows support, the ability to view and download logs and a new local token service. The local token service provides a simple onboarding experience for people who want to use OAuth to access the gateway’s Web Thing API.

You can see a full changelog for the 0.4 release here.

We want to say a big thank you to our growing Mozilla IoT community for contributing some of the new add-ons, providing us with feedback, reporting bugs, writing documentation and for generally helping us push Project Things forward. We can’t do this without you!

You can find out more about the Things Framework at iot.mozilla.org/things, and feel free to head over to Discourse with any questions and comments.

About Ben Francis

Full time UK-based Mozillian, working on the Web of Things.

More articles by Ben Francis…


9 comments

  1. Attila Csibi

    Hi,

    Great stuff this Things Framework! :)
    I would like to know if it’s possible to run a WebThing or more alongside the Gateway on the same Raspi, differentiating them by name or/and port.

    Thanks in advance!

    April 17th, 2018 at 02:54

    Reply

    1. Ben Francis

      Yes that is technically possible, as long as you run the web thing on a different port to the gateway.

      April 17th, 2018 at 07:50

      Reply

  2. Randy Constan

    A buddy pointed me to this link, as a possible lead to something I’ve wanted to look into for some time, which is purely browser based control panels for various IOT devices I’ve planned for development. Over 2 decades ago I had created a demo showing how a simple web page could be used for remote control. Back then about all you could do is move or hide various screen objects (with different methods for every browser), and about the only way to update screen metrics without reloading the page was by using fields. To reach a server, a java applet was used with something called a “shared byte array”, and so a crude demo web page was formed, and did allow a user to control and monitor a device.

    So these days, between CSS and ajax, it seems it would be hugely easier to do something similar, and with much easier security options. The advantage, of course, is that instead of having to create OS and IOS dependent apps, and hassling with whatever rules Apple and Android have for their app store and google play store, a web page based control panel could be made compatible with a huge span of browsers and devices, with no “app” download needed.

    So I’ve been looking to explore such a methodology for my own IOT projects. But having grown tired of having to invent or re-invent every wheel, I’ve been trying to convince a few web coding gurus I know to consider making an actual web-control panel “construction set”. And it was one of those buddies that pointed me to this page.

    But I’m late to the party here, and I don’t know whether your system is intended to help people build device specific apps, or browser based control pages like I’m discussing. Please let me know if we’re on the same page here, or if I’m barking up the wrong tree. :-)

    April 17th, 2018 at 14:02

    Reply

    1. Ben Francis

      Hi Randy,

      That is exactly the kind of use case we would like to enable. Our Things Gateway (https://hacks.mozilla.org/2018/02/how-to-build-your-own-private-smart-home-with-a-raspberry-pi-and-mozillas-things-gateway/) can provide a web interface for a wide range of existing off-the-shelf IoT devices as well as “native web things” created using the Things Framework.

      If you want to build your own dashboard web interface then the gateway exposes all the devices through the Web Thing API (https://iot.mozilla.org/wot/), which is available to third party web applications using OAuth.

      April 18th, 2018 at 07:02

      Reply

      1. Randy Constan

        Thanks Ben. Sounds interesting. I’d ask to move the conversation to email eventually, but I’m sure others can benefit from this discussion.

        Back to my projects. If you’ve watched the various microcontroller forums such as Arduino, you probably have heard of the very versatile and low cost esp8266 wifi modules. You can put your own code into it, or control it through another CPU board. I’ve yet to fully explore it yet, so forgive me if my discussion reveals some brain gaps :-)

        As an electronics and code developer, it seems that such a module would allow connectivity of any number of small “widgets” or a”appliance” type of products to the internet, through your home broadband service. With a paid or free DNS service and a port forwarding adjustment in your router, you don’t even need any special tricks to discover your device, as it can be a server to at least one remote connection.

        But, here’s where it gets a little fuzzy for me. While I’m sure the little esp8266 wifi could hold and handle a small web page, I would think the tools such as Ajax, needed to enable convenient two way communication between screen objects and the server, might require a bit more overhead than would comfortably fit in the esp8266 wifi. Even if it can, its possible the libraries don’t exist to do it, which of course means taking the time to invent more wheels.

        I’ll explore what’s available for the esp8266 on some other forums. But bottom line if this were a problem, the obvious (and maybe easier) solution would be to run the web server on a PC that is also connected to the same home router. From there, the PC could maintain a TCP/IP link between itself and the MCU, so it would only have to deal with simple supervisory monitoring and control commands.

        But you see where I have some gaps in understanding there, about how much CPU horsepower (and libraries) are available for little boards like that esp8266 module, for handling Ajax or Json requests. Like I said, I’ll investigate this more on some of those forums, but I’d imagine you’d agree that it would be far easier to implement the web server on a local PC, and let it talk to the MCU on a separate link with a simpler home brewed command set, no?

        Incidentally, as I do a lot of low level electronics, including PCB layouts and build. I mostly do electronics for musicians these days, but I’ve worked over 20 years doing SCADA type programming and electronic design for electric utilities, and am bringing a lot of experience (though some of its old school experience) to the table. So its possible we collaborate in the future on some IOT software and hardware combination offerings. My website is in my email address, should you care to touch base with me there as well.

        April 20th, 2018 at 11:07

        Reply

        1. Ben Francis

          Hi Randy,

          What you describe is basically how the Things Framework (https://iot.mozilla.org/things/) and Things Gateway (https://iot.mozilla.org/gateway/) fit together.

          The Things Framework provides libraries for IP-capable devices, like webthing-esp8266 (https://github.com/mozilla-iot/webthing-esp8266) for the chipset you mentioned, to help them expose a simple standard HTTP or WebSocket API (a WebSocket is actually quite similar to a raw TCP socket). Then the gateway (you can use an always-on headless Raspberry Pi for this rather than a PC) does most of the heavy lifting to host the HTML5 web interface and provide a range of other functions like the rules engine and voice controls.

          For more constrained MCUs there are a range of non-IP low power wireless protocols available including ZigBee, Z-Wave and Bluetooth for which the gateway has adapters. Or you can simply plug the MCU directly into a serial port on the gateway and use the serial adapter.

          April 20th, 2018 at 11:27

          Reply

          1. Randy Constan

            I’m well aware of the many “wireless options” for simple non TCP communication. But let’s concentrate on those that do, like the ESP8266. Please keep my goal in mind here… to be able to design electronic devices that can be controlled and monitored remotely, using a modern browser, rather than a platform dependant “app”.

            If I understand your last message correctly, though an ESP based product may technically be able to connect to the internet, it would likely need help from a more fully capable web server. Am I correct? If so, is this why you mentioned the Raspberry pi instead of a PC?

            Though I’m familiar with the raspberry, I’d want to avoid it for an end product if possible. Besides the added costs, its a safe bet any household that already has a WiFi router will also have a PC, and free web servers for PCs abound.

            So just to be clear, if an ESP board itself would not have the resources needed to directly BE the web server for a browser based control panel, do you have a PC based solution?

            April 23rd, 2018 at 09:41

          2. Ben Francis

            An ESP8266 is capable of connecting to the Internet itself via a home router, but you’d probably want to use TLS, add some authentication and configure DNS and your firewall to make it accessible via a web address. The Things Gateway can provide all of that by acting as a Internet-facing proxy.

            You can build and run the gateway software yourself on a PC (see the instructions on the README https://github.com/mozilla-iot/gateway/blob/master/README.md), but I wouldn’t recommend that for an end product because the PC would need to be turned on and connected 24/7 and you’d need to be very careful about security.

            A dedicated gateway has some benefits. Apart from the Raspberry Pi we’re currently working on an OpenWrt build to run on a range of existing router-like hardware so in future your IoT gateway and home router could be the same device.

            We’re also working to improve the Things Framework to add authentication etc. to things themselves so they can be directly connected to the Internet, without the need for a WoT gateway.

            April 23rd, 2018 at 10:03

  3. Amber

    I want to build my own dashboard but its complicated help please

    April 19th, 2018 at 07:30

    Reply

Post Your Comment