Building remote temperature sensors with Nodemcu/ESP8266 DHT22 and MQTT

The Concept

I knew I'd need remote temperature and humidity monitoring for my house, and it'd need to be wireless. I've previously used DHT22 sensors directly connected to a raspberry Pi at work so knew they were cheap and would work.

But how to connect them wirelessly? Over the last few years I've considered Xbee (Zigbee) modules like: (I even bought a book about how to make your own wireless mesh networks) but it looked very complex and the sensors wouldn't directly talk to the radio modules.

I recently found the wonderful MySensors website, which uses Arduinos as sensor controllers and a gateway, and it was compatible with most of the Home Automation platforms I've considered using (e.g. HomeAssistant, OpenHAB, etc) so I asked for an Arduino starter kit for Christmas 2015 which included some useful parts and an Arduino Uno.

The Solution

But then I realised, thats a lot of microcontrollers! I had seen the ESP8266 radio chip mentioned as a module to connect Arduinos together, but then I found out the ESP8266 can be loaded with Arduino code itself! Further research revealed the perfect solution for combining a WiFi connection, lots of digital IO connections (for DHT22 sensors) and arduino compatible firmware: The NodeMCU

So, off to Google I went, and searching for DHT22, MQTT and Nodemcu eventually revealed the following blog entry: https://odd-one-out.serek.eu/esp8266-nodemcu-dht22-mqtt-deep-sleep/

Perfect! So I ordered 2 NodeMCU boards from eBay and some DHT22 sensors, and waited for them to arrive and for a quiet weekend when I could plug them in. That day was yesterday (Sat 9th January 2016) and it wasn't as easy as the blog entry I'd read had implied! Below is what happened:

The Prototype

So, to start with I have:

• 10 year old MacBook Pro running OS X 10.6.8 (which can't be upgraded) 
• Breadboard and all relevant cables
• NodeMCU Dev Kit 1.0 module

The instructions I followed were slightly different, as the NodeMCU LUA firmware now includes DHT support: https://odd-one-out.serek.eu/esp8266-nodemcu-dht22-custom-modules-firmware/

1) Check the NodeMCU boards work

Luckily the eBay seller I bought pre-loaded them with an Arduino program to scan for wireless networks and print it out on the serial console. So, connect to the NodeMCU and read it's output. Sounds simple, doesn't it? Not on a 10 year old MacBook Pro!

https://github.com/nodemcu/nodemcu-devkit/wiki/Getting-Started-on-OSX was where I started, but I quickly found I had none of the software or tools installed.

1. First download esptools to flash the Nodemcu firmware to the development board. Doesn't work. No python installed!
2. Download python2.7
3. Download the correct usb serial port driver to connect to the nodemcu board (following these instructions http://plugable.com/2011/07/12/installing-a-usb-serial-adapter-on-mac-os-x)
4. Don't mess around with screen or telnet, download coolterm (lucky compatible with OS X 10.6.8) to see the serial port output. 

Awesome! Now I can connect to the nodemcu board and see the output of the test program preloaded by the ebay seller. It works. Now, onto making it do what I want.

2) Flash the NodeMCU boards with the latest firmware

Based on the blog post I read, it seems a lot simpler to code in LUA, so I want to flash the latest firmware which also includes native MQTT and DHT22 support. Full instructions are at: 

https://odd-one-out.serek.eu/esp8266-development-kit-nodemcu-firmware-update-os-x/

I built a custom firmware at http://nodemcu-build.com/

With no components attached to the nodemcu, it's time to wipe it and load the custom firmware. This is quite simple and worked first time. I hit the reset button on the nodemcu and got the following boot up message in coolterm:

NodeMCU custom build by frightanic.com
.branch: master
.commit: c8037568571edb5c568c2f8231e4f8ce0683b883
.SSL: false
.modules: node,file,gpio,wifi,net,tmr,uart,mqtt,dht
 build .built on: 2016-01-09 15:59
 powered by Lua 5.1.4 on SDK 1.4.0
lua: cannot open init.lua

Hurrah! Blank firmware works! Now to program it to read the DHT22 sensor and report the data with MQTT.

3) Program the NodeMCU 

The preferred software to program the board is esplorer. I very quickly I discovered I can't run it because I don't have Java 7 on my mac. Officially it's not supported by Apple or oracle on anything older than OS X 10.7. Some Google searching later, and this turns out to be a mostly political issue, and there is a way to force Java 7 to install on OS X 10.6.8 so cant run eplorer (to program the mcu)

I found the following instructions to trick the java installer that I was running OS X 10.7 lion: http://jksha.blogspot.se/2013/09/java-7-and-snow-leopard-osx-106.html

Some scary config file editing later and I've got Java 7 installed, but esplorer still didn't work. It turns out I downloaded and install jre (java runtime) when really I needed jdk (java development kit). Whoops. So try again.

Fake the OS X version again, download Java 7 DK, install it, revert to correct OS X version.... still doesn't work! java-version in terminal reports version 6. I've not restarted the MacBook Pro yet, so that can't hurt.

One restart later, and Java 7 is working, but when opening esplorer I now get the error:
NSInvocation: warning: object 0x108b8a3c0 of class 'ThreadUtilities' does not implement methodSignatureForSelector: -- trouble ahead

Whoops. Sounds pretty nasty! Some more Google searching and it turns I can't use anything later than Java DK 1.7u25 on osx 10.6.8 otherwise it just spits out the error above.

Ugh. Repeat above steps, but this time I need an account to log into the Oracle Java download site, 

download the correct version Java SE Development Kit and install it. Then, open esplorer with the correct java version using the command:

/Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/bin/java -jar ESPlorer.jar 

SUCCESS!

Now, I can use the code from https://odd-one-out.serek.eu/esp8266-nodemcu-dht22-mqtt-deep-sleep with the IP address, WiFi and MQTT details updated to match my network. Load it into an init.lua file, upload it to the NodeMCU board, hit the reset button and away it goes!

4) Check the remote sensor works

There should be some debugging output via the serial port, but this only worked for me if I uploaded the code and executed it. After a hard reset of the NodeMCU board it didn't output anything over the serial port. Instead, I watched the MQTT feed with mosquitto_sub and saw the following:

ESP8266/temperature 20.8
ESP8266/humidity 61.1
ESP8266/temperature 20.7
ESP8266/humidity 61.1
ESP8266/temperature 20.7
ESP8266/humidity 61.1

Hurrah! Prototype complete! For now I'm saving the raw MQTT data into a text file with timestamps, so I can verify it works and manually check temperatures for different times of day.

5) Next Steps

To complete the sensors and set them up permanently I need to:

• Change the code to use 2 sensors attached to one NodeMCU board, one on an extension cable into the room next to the sensor. This allows monitoring of 2 rooms from one board and saves on components.
• Find a 5V power supply (possibly an old smartphone charger?)
• Buy an enclosure for the board, and solder and wire it up properly. Possibly this one: http://www.maplin.co.uk/p/vented-psu-enclosure-4-n06kh

• Setup OpenHAB to subscribe to the MQTT feed and process/display it.
• Enable persistance in OpenHAB to record historical data.
• Create some pretty graphs.

GoPro camera fun and table hack idea

So, the new year is here and I've not done much to the house over Christmas. The main bedroom is almost ready, but we need to pick a carpet. We have everything else ready (furniture, fittings, etc).

I did add an extra security light to the garage, which is connected to a mains remote controlled switch. This means Becky can turn the light on from her car as she reverses along the dark driveway behind the row of houses (as the normal security light only gets activated once someone is outside of their car). I might hook into this remote control system in the future, either by decoding the wireless signals used or by adding a spare remote and patching its buttons into an Arduino or raspberry pi or something else.

I did however get a nice shiny GoPro HD Hero 2 camera for my birthday from Becky. After getting used to its wide angle, and the fact that it has no viewfinder, 

I've finally got some good photos of the house! I've also spent plenty of money on a WiFi BacPac add on, Class 10 32Gb SDHC memory card, and a few other bits and pieces.

 

It is going to be perfect for running a time lapse recording of me and Becky doing significant changes to a room (e.g. the main bedroom), although I have to calculate the photo interval properly. On a recent trial time lapse I recorded I set the interval for 2 seconds, and ended up with over 9,000 3mb photos! It almost filled the new memory card up! I will post completed time lapse videos and photos on here soon.

Still, there are many ideas and plans for things that can be done in 2013. One of the best ones yet was suggested by Becky. I was fed up of the amount USB cables cluttering up our coffee table when charging phones, PS3 remotes, etc. I thought of adding a USB powered hub into the IKEA LACK coffee table we have. Then Becky suggested "why not just add mains extension sockets so we can plug in our laptop chargers as well.

So, I've decided I can recess some sockets under the table (as it is mostly hollow anyway) and add some cable management hooks. A powered USB charger/hub and some spare USB cables. If I'm cunning enough I can hide all the wires in the table frame, with a power cable for the mains coming out of a table leg and under the rug/sofa.

Better heating control?

I have a completion and moving date of 22nd October 2012!

Realistically I won't get any of the cool things done straight away, and will probably spend most of my time decorating and getting the essentials done to make the house liveable  This doesn't mean I can't still plot and scheme!

I read another post on hackaday that mentioned using an old android phone as a thermostat to control your heating.

This would be a perfect use for my old HTC Desire mobile phone.

http://androidthermostat.com

A few more home hack ideas

Getting very close to completion of the house purchase now!

On a long drive back from a family holiday in St Ives last month I had the following ideas/thoughts. I forgot to post it at the time, but here it is:

• A house-wide CCTV system. Probably cheaper to use individual ip webcams than a composite video and separate DVR recording system. This would cover the: front door, garden, garage and alleyway behind the garden (where there is access to the back garden and car parking)
• PIR motion sensors in the alleyway, front garden, front door and inside the house in the hallway. This can be hooked into the CCTV and/or security system, as well as detecting people when they come in (potentially hooked into the automatic lighting system).
• Temperature sensors in all rooms and under stairs cupboard. This can be linked to the thermostat/central heating to allow for more efficient heating of the house.
• All equipment to run the hacked house to be hidden out the way in the under stairs cupboard (hence the temperature sensor in the cupboard). UPS compatible wiring, if I wish to add mains power cut protection later.
• Cooling system (probably temperature controlled) to vent from the under stairs cupboard into the adjacent boiler cupboard or out of the side of the house
• Run cat5 and power out to the garage (makes other ideas easier to implement)
• All the living room entertainment systems to be in shelving unit to the left of the fireplace (where the TV is going to be wall mounted). Wiring will be hidden as much as possible. Potential to have virgin media coax hidden through ceiling or floor to the cable tv box on the shelves.
• A 5 channel surround speaker system in corners of the living room and auto calibrated to balance the sound for optimum listening on the sofa, opposite the tv/fireplace.
• TV wall mounted above fireplace.
• 2 additional speakers in the kitchen running off main amp in the living room.
• A Raspberry pi running raspbmc or a silent mini-itx media centre running xbmc in living room (to replace the existing PS3 which doesn't support all media codecs).
• Google nexus 7 tablet as remote control for xbmc. Can control music playback in kitchen/living room. Can also access web based house interface with cctv, temperature monitors, lighting control, etc
• A computer controlled (from server or media pc) IR sender to act as dummy remote control. Used to control amp, tv, and other Ir media equipment in living room to provide seamless media experience

Possible ideas, but not sure about these. Probably best as separate stand alone projects

• Brewing control/monitoring in garage to control my planned micro homebrewery. Could use aebcam and Arduino connected to temperature sensors and boilers to automate the brewing process. It would appear that most of the hard coding work on this project has been started at http://brewpi.com (although this would mean buying another RaspberryPi)
• A modified roomba style Hoover with camera and remote control. Can clean carpets downstairs and monitor house/cat. Another one upstairs? Linked by WiFi to server for out of house control

Very Effektiv Garbage & Recycling Bins

I've seen something similar to this a while ago on the internet, but I can't remember where it was now. Then I saw this on IKEAHackers.net - Very Effektiv Garbage & Recycling Bins:

I have to do this to the kitchen for the food waste recycling. I'm fed up of the small brown plastic bin that Bristol Council provide, and this is an almost perfect solution!

Cat warmer for naps

Cat palace with an automatic heat lamp via Hack a Day


Me and Becky would like to get a cat when we're settled down in our new house. This seemed like a nice idea, and automatic heat lamp for a cat to nap under.

Hopefully something like this would stop a cat from napping in unwanted places or on warm electronic items like laptops, HiFis, etc

Arduino + PWM Shield = Easy LED Fading

I'm lazy! Why make my own circuit to control the individual steps fading in and out on the automatic stair lighting when I can use a pre-designed circuit board.

The best way I've found of fading LED strip lights is with Pulse Width Modulation. Arduinos can do this (and have a few pin headers dedicated to PWM), but I'm going to have at least 10 stairs and will run out of PWM connectors on a standard Arduino.

An Arduino Starter Kit from oomlout costs £40.50 and includes an Arduino Uno, breadboard, pre-made jumper wires and other bits and bobs needed to get prototyping.

The great thing about Arduinos is you can connect shields to their pin outs to quickly and easily add extra functionality. There are almost 300 known shields available (a fully searchable list is available at http://shieldlist.org).


If I use a PWM control shield from Practical Maker then it gives me 32 PWM outputs, which are more than enough!

A pre-assembled shield costs $36.75 from the US, but a naked PCB only costs $11.25. If the other components can be sourced from the UK then this would be a lot more affordable.

Simple and easy, and then just the hard bit of coding and wiring up the LED strips!

Adafruit + Raspberry Pi = Home Automation Server

Just spotted this on one of my blogs (I think hackaday).

This looks like a potential tweaked OS to install onto my RaspberryPi so that I can easily coordinate all the other systems I will eventually install into the house

Overview | Adafruit Raspberry Pi Educational Linux Distro | Adafruit Learning System:

Npower SmartPower Meter Hack

I got a free wireless electricity meter monitor from npower a few years ago and was thinking of ways in which to intercept the signals so I can add electricity usage to my house hack.

It looks like it's just a re-badged Prelude 1 made by GEO

This is a shame, as GEOs "Base Level" devices don't have any way of exporting the data, whereas their "intermediate" devices do.

I was inspired by this hack, but found that they were reading the LED light that flashes when the electricity meter records a unit of power going through it.

At first I thought I could physically open up the case and tap into it's circuitry, but this would be messy.

30 minutes of Googling later and I've found out it uses Wireless M-BUS as a radio standard between the battery powered receiver (attached to my mains electricity meter) and the little black npower reciever (with LCD display).

I doubt I can easily connect to the transmitter using M-BUS as the two devices are paired using encryption keys. Looks like I'll have to open up the wireless meter after all and see whats inside. Maybe there are some data tags I can connect to and steal the data that way, but it's unlikely.

Or I could just go and buy a new wireless electricity meter (defeating the point of a hack) or build one from scratch using an electricity clamp current meter. That would be more fun, but less of a hack and more of a build-from-scratch.

I've bookmarked everything and will shelve this plan for now :)

Initial Ideas

It's early in the morning and I can't fall back asleep, so I might as well start with my initial ideas for hacking a house. (I'll clean up this post and add links later)

Hacking can sometimes be seen as a bit of a derogatory word, but I like its underlying meaning. Wikipedia describes a hacker as follows:

In home and hobby circles, a hacker is a person who heavily modifies the software or hardware of their own computer system. It includes building, rebuilding, modifying, and creating software (software cracking, demo scene), electronic hardware (hardware hacking, modding), or anything else, either to make it better or faster or to give it added features or to make it do something it was never intended to do. Hobby home computer hacking originated around the MITS Altair, yet the spirit of hacking has been embodied in real projects since the advent of simple tools.

I wouldn't describe myself as a creator, more of a hacker/bodger/kludger. I like to fix broken things, but I'm lazy so I usually borrow something that has already been made and adapt it to suit my needs (rather than create something from scratch).

The purpose of this blog is do document my attempts to hack and improve a house.



So far I've only ever rented flats or houses, and I've been unable to make any permanent improvements as my landlords would frown upon my ideas (and I'd have to remove the mods when I moved out anyway). I'm a geek and love technology, so I've tried to bring the benefits of this into places I've lived.

Over the years I have (with help from various geeky housemates, especially Dan) done many hacks and projects. A majority of this happened in a 7 bed shared house in Redland, Bristol:

• We installed a house wide cat5e Gigabit network (dubbed Wonderland)



Circa 2006: At its maximum we had 2Tb of shared music, films and tv across 14 PCs

• Setup a touch screen based HTPC system (integrated with the existing tv and hifi system) with network sharing and synchronization called Nucleus. Dan did most of the hard coding work that made this possible. It was very close to being finished, but we all moved out and the project lost momentum, however it is still available for download at SourceForge!
• Configured a mini-itx based Linux based router to share and monitor/graph internet usage (based on the vnStat project). This was very useful in a shared student house, and I continued to use it for the next few years of house sharing.
• Built a pair of homemade speakers and portable battery powered amp (which, thanks to Dan, sound absolutely amazing!) We made a separate project website at: http://ppamp.dineley.com. This was designed for audiophiles and it had the potential of being a commercial success, but the momentum ran out and the market was flooded with cheap iPod docks.
• And other things (which I can't remember now)

Back to the present. I'm in the process of buying my first house with my fiance Becky. We've not exchanged contacts, so the house isn't ours yet, but I keep on getting carried away and thinking ahead of the potential awesome technology I could add to a house.



Home automation
I've seen some cool projects on the internet (usually via hackaday) and the home automation has always caught my attention. In fact, I've been thinking of automating a house since I was in primary school (I wanted automatic closing curtains in my bedroom).

Below are some of the things I would like to hack in my house:

Floor level LED lighting on the stairs, with automatic triggers when walking up/down. This is ideal for evening/night time (and if I'm too lazy to turn on the light switch).
Inspired by:
Sean's Automated hallway Skirtingboard Lighting
a StarScape customer project
ElJefeUno's Stairway LED lighting with IR trip sensor
Illia's Automatic Led Stairs Controller
Alan Parekh's Stair Lights

LED lighting above work surfaces in kitchen (mounted under cupboards above the surfaces).
Inspired by:
Esterill's How to fit LED kitchen lights with fade effect
DJJules's Build LED Under Counter Lighting that ROCKS!

LED lighting along the tops of cupboards in kitchen, and possibly around the edge of the ceiling in rooms to give nice gentle indirect lighting. Remote controllable from smartphones? 

Inspired by:

dlewisa's LED Indirect Lighting

Integrated LED lighting in garden. Possibly built into the garden/patio itself. 

Inspired by:
cw-kid's LED Strip Lighting Project: Part 1, Part 2, Part 3, Part 4, Part 5

• CCTV system linked to home server/internet/smartphone
• Keyless entry to house (has major potential security risks, so not too sure about this)
• Fully integrated HTPC/media system in living room which also links to speakers across the house/garden for distributed media playback
• Central media server for all music and video file storage and backups (probably a commercial NAS solution as they are more power efficient than a full PC)
• Computer controlled heating/climate control
• Wireless charging for smartphones on bedside tables. (As I write this on my smartphone I have a micro USB cable wedged uncomfortably against my hand)

And there is probably more. I will try to make a proper to do list of things that is more organised. Maybe install a blogger todo list plugin (if they exist).



Technologies/tools/parts at my disposal
I've done a little research into other peoples home related hacks and the Arduino seems to be incredibly popular. I'm not a natural programmer, but the coding seem to be relatively straightforward and there are plenty of resources and support online for hackers like me.

I also picked up one of the first RaspberryPi Linux development boards. While not as mature as the Arduino, I think this could make an ideal 'overlord' style controller that oversees various independent Arduino sub-systems and coordinates the overall house hack.

LED Dioder strip lights (think IKEA, but cheaper ones available from eBay) are perfect for most of my lighting ideas and saves time on manually soldering and designing a lighting system from scratch (and they're waterproof!)

The Internet! Without Tim Berners Lee's invention of the world wide web none of this would be possible!



Anyway, I've spent far too long waffling on about my ideas. More important things need to be done before I can start (or afford) any of these hacks (like actually getting the house and moving in)!

I am determined to make at least some of these hacks possible. If I plan ahead properly and make the early prototypes in a modular fashion then it should be easier to adapt and link them all together with some form of computer control (and I'll actually stand a chance of getting some of it completed)!

I will add as many links to this post as I can find, and probably share interesting related hacks on this blog until I get started on the house hacking (which hopefully will be in the next few months).

Here are some more links to things that that are useful reference:

• Commercial home control system:
  http://www.embeddedautomation.com/products/mcontrolv2.asp

• LadyADA LED Strip light tutorial
  http://www.ladyada.net/products/rgbledstrip/

• Controling LED lights with a RaspberryPi:
  http://www.raspberrypi-spy.co.uk/2012/06/control-led-using-gpio-output-pin/#more-255

• Calculate how to wire up LEDs without blowing them up:
  http://ledcalculator.net/

• http://www.instructables.com/id/Control-Ikea-Dioder-LED-Strip-with-Arduino-16X-P/?ALLSTEPS

• A brilliant guide to using/controlling Dioder style LED strips: http://www.ledlightsworld.com/page.html?id=32

• A good seller of Dioder LED strips on eBay (much cheaper than IKEA): http://www.ebay.co.uk/itm/261070133115

• Hacking IKEA Dioder lights:
  http://hackaday.com/2011/08/19/adding-usb-control-for-ikea-rgb-led-strips/
  http://hackaday.com/2009/12/29/ikea-dioder-hack/