ESP Data WiFi Logging
...ESP and MeDecember 18, 2016
For some time I've had some DHT11 devices logging data using long wire runs and an Arduino ProMini reading values and providing data to my home automation server via RS232 serial on demand. I.e. I have a ProMini via USB hooked up to the server. The ProMini is wired to the DHT11 sensors along with other things like moisture sensors, rain bucket, wind speed, etc. Every five minutes a VBScript runs that queries the ProMini for the latest readings and it serves up the values which are then logged into a SQL table and/or updated in the dedicated HA system variables. This works, but does have limitations. The main limitation is my laziness of running more wires to be able to log values inside parts of the house. No doubt it's a pain trying to run a CAT5 cable through the attic and into some room somewhere just to read temps and humidity.
I actually had purchased some ESP devices in middle of 2015 and had planned on messing with them to see how these "IoT" things actually work. I lose interest and a year or more rolls by before I even get them out of the bag to see how to do anything with them. I purchased some ESP8266-01's, some -05's with antennas and some -08 devices not really know what they were all about. I had mistakenly thought there were serial to WiFi solutions only and didn't realize you could program them standalone. I guess the -05 devices are still kind of limited like that but even the -01 devices are quite powerful obviously.
Once I dug only a little into the -01 devices I realized these could very easily do the remote logging I want to without much effort. Most of the online examples use ThingSpeak.com or some other cloud solution but I already have my OWN cloud solution with my HA web server. Logging information and even controlling back to the ESP devices was actually going to be pretty easy using the examples online. That along with my own .ASP code and local SQL database for logging made it all super simple.
One of the challenges of working with the ESP-01 devices is I guess the odd pin layouts and arrangements. I had already ordered some 8 pin headers from the slow boat world but didn't want to wait that long and realized I could just use two header shells back to back and glue them to get an easier result. This actually allows me to make my own short cables without extra soldering which has likely worked out better.
Instead of building out things on a perf board or like a pro would design an actual PC board and have it manufactured I just free wired the sensors together. Good enough for me and with the simple needs of a 3.3v regulator, DHT22 sensor, and ESP-01 CPU there really isn't much to it.
You can see from the pic that it's really pretty straight forward. I did run a jumper on top of the ESP-01 devices to jump the CH_PD in over to the 3.3v power supply as needed to make them work and used that to my advantage to use that pin to run power over to the DHT22 device. This lets me tape things up and stuff it into a 3D printed box with only the +5v and Ground needing a connection to the world using any 5+v wall wart. I did want to make sure the ESP device could be unplugged to re-program it as I do not have any over the air options setup. I'm sure some pro builders are cringing out there as I have no pull resistors, reverse polarity protection, etc but I have seen these hooked up like this all over the net and it just works for me.
Logging The Data
Of course this stuff doesn't do any good if it can't push the information somewhere and that is where the web server side comes in. I've already had a "logging" table in my SQL database for ages pushing data into the database is very easy. The ESP devices just post to a deep page with the data and my code stuffs that data into the logging page. A few security approaches are used to keep it basically secure as the ESP devices have registered themselves when they come online. I then use that information to make sure the device calling for the post is coming from that particular local IP before posting anything. Also right up front I check that the http call is coming ONLY from an internal network IP and simply shut down the page if not.
Otherwise the web site of the posting is standard http get and SQL writes that log the data to my logging tables. From there, either scripts running on a regular basis or the HA system itself can use the values to determine what to do. I do update variables in the old HA embedded device so it can make decisions on the temperature values if needed. For example if the garage temperature is lower than 40 degrees F and the garage electric heater is off then turn it on. Flip that around when the temp is greater than 45 degrees and turn it off if needed.
I have a little dashboard I can check to see if the devices are on or offline and when the last time they booted/checked in or posted and update. If a device hasn't reported in after a certain period the LastPost field turns red just as a reminder. In this capture I hadn't updated the "Description" fields yet. I intentionally left any location information out of the actual ID values. This way I can move devices around without having to worry about update firmware and can just rename the device using the description values.
Presenting the Values
The most recent values can be seen on the Other Temps page of the HA website and tag the ESP device to a particular location. Clicking on the description value shows a chart of the history of the logged data for a visual of the information. One thing I learned the hard way trying to use the DHT11 devices was that the devices I had reported invalid humidity, which was an apparently known issue. They also do not report temps under 0 degrees C or 32 degrees F so are kind of useless for my area for outside use.
There are some issues which appear to mainly be related to my WiFi coverage as the devices can miss reporting way to often. I am installing a WiFi extender / amp soon as this has been an issue on the other end of the house anyhow which I think will help resolve the drops. Overall I am happy with the results so far. I can now put loggers, alarms, monitors, etc pretty much anywhere inside or out and just have the post to the web server for actions.
Next Steps - Turn on some Inputs or OutputsWith the logging seemingly working I plan on updating the hardware end points with the simple low volt relays that can be controlled back from the HA server by HTTP. I've already tested this with one device and an LED and could also set these up as remote INPUTs either buttons, motion, or similar that could log back or trigger web actions.
Motion / Activity SensorsThe first other input may be to add PIR sensors into the remote 3D printed boxes and have them trigger logs for motion back to the system. This data can be used by the HA system to determine if a light has been on an extended time without any local room motion and turn it off.
Plant MonitoringOne could also have a web connected plant monitor rather easily. Use a moisture sensor and have the input checked on a regular basis and trigger a notice via web services if water is needed.
Replacing Door SensorsAnother use for ESP devices as sensors will be to replace my refrigerator and freezer door sensors that currently use X10 technology. Currently I am using some old dry contact to X10 modules that sometimes work and sometimes do not. Using an ESP device may be more stable by just having the input toggle do an HTTP get to update the status and log the activity.
The ESP-01 devices ARE limited to the two pins so using them for fancy solutions isn't feasible but for basic temp/humidity and an IN or OUTput is super easy. The ESP-08 devices have many more input / output pins and could be used to built more powerful end points. As the ESP-05 devices are serial to WiFi only they can't stand along BUT with their extended antenna I think they would pair up nicely with a ProMini to upgrade my home brew anemometer and air moisture setup soon.
Time and interest will determine where their use goes next.. or maybe doesn't go anywhere but I did get this far at least.