Full disclosure: I’m a Scalyr DevOps engineer, but I’d be geeking out over the sheer number of possible uses for Scalyr even if I wasn’t. It’s more than a log analysis tool—it’s a platform. Scalyr now monitors the temperature inside my house, as well as the history of my thermostat and HVAC system usage. I’m one of very few homeowners in the world with real-time access to information about my HVAC system’s energy usage.
What compelled me to do this? Siri.
And a desire to harness home automation to improve my house’s energy efficiency. Here’s the story.
I’d recently installed a Nest thermostat in my ongoing and evolving quest to improve energy efficiency and be greener. It was leaps and bounds above the 15-year-old “programmable” model that came with the house. I thought it was pretty awesome that I could control the house temperature from my phone, no matter where I was.
But I wanted more. Heating and cooling are responsible for a significant percentage of the average American household’s utility bills, and if having access to real-time temperature data can help people save even a few percent on temperature-control-related expenses, it would be a huge net win for the planet. “Knowing is half the battle,” after all (thanks for the lesson, GI Joe).
Yet despite the advent of smart home technology, we generally have very little insight into what’s actually going on in our homes. The technology to collect this data is actually pretty cheap (check out my cost breakdown below), and HVAC is a good place to start.
Plus, I’d just gotten a new iPhone 7. I’d heard Apple had opened up the Siri API quite a bit with iOS 10 and I was itching to play around with it. Integrating the Nest with Siri would be a good first project. Life would be so much more pleasant if I could say “Hey Siri, set the thermostat to 72,” instead of going through the awkward business of opening up the Nest app or manually tweaking the dial to set the temperature. All that swiping, tapping, and twisting… so primitive.
This integration project wasn’t quite as easy as I’d hoped…
…thanks, Apple and Google. Nest and iOS haven’t played well together since Google acquired Nest. Apple has its own proprietary home automation framework, HomeKit, while Google is rolling out another. And these two tech behemoths are not exactly the best of friends.
Fortunately, when it comes to integrating Nest and HomeKit, there is a solution. There’s an open-source NodeJS application called Homebridge. It emulates the HomeKit API and basically provides a way for Siri to communicate to non-HomeKit-enabled devices (such as the Nest).
Open-source NodeJS application?
Fantastic! This kind of project was right up my alley. I’m a DevOps guy, after all. I could install the NodeJS application and keep it running somewhere. Where, though?
I don’t really need to keep machines running 24/7 at my house, and keeping a desktop or laptop running all night seemed like a waste of energy (which kind of defeats the purpose). Then it hit me—I’d been looking for an excuse to buy a Raspberry Pi for a few years now, and this was the perfect opportunity. To further justify my decision, the cute little machine only uses about three watts per day. It’s almost like I’m producing energy instead of consuming it!
After a quick check to make sure I’d actually be able to run Homebridge on the Pi, I ordered a kit. It arrived two days later and I set it up—all pretty straightforward. I had to install various lower-level software prerequisites on the Pi, then NodeJS itself, and then Homebridge. Also pretty straightforward.
After setting up a JSON config file with my Nest credentials, I started Homebridge up. And it recognized my Nest devices on the first run! In case you’re curious, here’s some of the terminal output I saw:
It was nice to see that Homebridge recognized the Nest stuff. But I was on a quest.
I was going to control the Nest with Siri, dammit.
And nothing was going to distract me from that goal. I pulled out my iPhone and launched HomeKit. I was pleasantly surprised to see that Homebridge showed up in the HomeKit app and I was able to register my Nest. So far, so good. Then, the moment of truth.
I said the magic words: “Hey Siri, set the thermostat to 75 degrees”.
And Siri said, “Okay, Christopher, I’ve set the Living Room Thermostat to 75.” And I heard the furnace whirr to life in the background.
Success!
I also noticed that this appeared in the terminal window:
11:12:04.889 raspberrypi1 /var/log/syslog Sep 24 11:12:02 raspberrypi1 homebridge[26550]: [9/24/2016, 11:12:02 AM] Target temperature for Living Room Thermostat is: 75 F
And my thinking went something like…
Hmm, that’s a log line.
Wait a minute…
Homebridge isn’t just a translation layer between Siri and the Nest. It’s also logging all the events and state changes for my Nest (and probably any other smart home devices I hook up to it).
Hmm, my company takes exactly this kind of data and allows me to do interesting things with it… such as graph it and set up alerts.
But would it work?
In theory, all I needed to do was install the Scalyr agent on the Raspberry Pi, set it up to monitor Homebridge’s logs, write a parser to interpret the log lines, and I’d be able to produce nice graphs of temperature and HVAC usage. Sweet!
The next question was: would the Scalyr agent work on the Raspberry Pi? I hadn’t ever heard of anyone installing it on a Pi, but there didn’t seem to be any reason it wouldn’t work. After all, our agent is pure Python. And the default Raspberry Pi OS (Raspbian) is a flavor of Debian, which is one of the Linux flavors we support.
After trying the automatic install script, I looked at scalyr_install.log. At the end of the log file, I found this:
W: Failed to fetch https://scalyr-repo.s3.amazonaws.com/stable/apt/dists/scalyr/InRelease Unable to find expected entry 'main/binary-armhf/Packages' in Release file (Wrong sources .list entry or malformed file)
Okay, so I guess we didn’t build an ARM distribution.
I guess nobody anticipated this use case (myself included). No problem. I figured I’d just build it myself. It actually wasn’t too bad… we have a script for this. I had to install fpm and some of its dependencies, but that was all pretty straightforward.
# git clone # sudo apt-get install ruby-dev gcc make # sudo gem install fpm # cd scalyr-agent-2/
After that, I used fpm (which is awesome) to build a package for the ARM architecture. The build script currently requires some git settings to be in place, but those are easy enough to set up:
# git config --global user.email “[email protected]” # python build_package.py deb
At this point I had a package I could install locally:
# sudo dpkg -i /var/cache/apt/archives/scalyr-agent-2_2.0.20.pre21.1_all.deb
And voilà, I had the agent installed.
# ls /etc/scalyr-agent-2/ agent.d agent.json #
When I installed Homebridge, I set it up as a systemd-managed service, so it was sending its logs to /var/log/syslog. In theory, all I would need to do is point the Scalyr agent at that file, and I’d be off and running.
Here’s the damage I had to inflict upon /etc/scalyr-agent-2/agent.json (changes in green):
// Configuration for the Scalyr Agent. For help: // // httpshttps://www.sentinelone.com/wp-content/uploadswww.scalyr.com/help/scalyr-agent-2 { // Enter a "Write Logs" api key for your account. These are available at httpshttps://www.sentinelone.com/wp-content/uploadswww.scalyr.com/keys api_key: "MY_SECRET_API_KEY_THAT_YOU_WILL_NEVER_SEE", // Log files to upload to Scalyr. You can use '*' wildcards here. logs: [ { path: "/var/log/syslog", attributes: {parser: "home"} } ], monitors: [ ] }
As you can see, I declared a custom parser called “home” to process /var/log/syslog. My next step was to go to Scalyr and define that parser. There was a bit of trial and error involved here, but this is what I came up with. Trust me, it works. if you want to use this, just copy and paste.
{
formats: [
{
id: "targetTemperature",
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Target temperature for
$itemLocation$ is: $temperature$ F"
},
{
id: "setTemperature",
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Setting target temperature for
$itemLocation$ to: $temperature$"
},
{
id: "currentTemperature",
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Current temperature for
$itemLocation$ is: $temperature$ F"
},
{
id: "currentHumidity",
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Current humidity for
$itemLocation$ is: $humidity$%"
},
{
id: "hvacStateChange",
attributes: {hvacStatus: 1},
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Current heating for
$itemLocation$ is: Heating",
},
{
id: "hvacStateChange",
attributes: {hvacStatus: 0},
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Current heating for
$itemLocation$ is: Off",
},
{
id: "hvacStateChange",
attributes: {hvacStatus: -1},
format: "$aHost$ homebridge[$homebridgePid$]: [$raspdate$, $rasptime$] Current heating for
$itemLocation$ is: Cooling",
}
]
}
So now I had the agent installed and the appropriate configuration in place. The next step was to start the agent:
# sudo service scalyr-agent-2 start Okay. #
So, the agent was running.
I restarted Homebridge (to do a basic sanity check), then I went to Scalyr to see if things were working:
Success! Everything was proceeding according to plan.
I played around with the thermostat on Siri for a while to test things out. Sure enough, log lines showed up on the console and in Scalyr. Here’s a week’s worth of temperature data:
The teal line represents actual temperature in the house, while the orange line represents the temperature the thermostat was set to at any given time. I’d configured the Nest to gradually turn down the heat to 65° at night, bump up the temperature to 75° between 6:30 and 8am, and then keep things at 72° for most of the day.
Notice that the temperature stayed a lot closer to 72° on the last day, as compared with the other six days shown on this graph. The reason? A storm came in last night and the outside temperature dropped dramatically. We woke up to the first snow of the year that day. It would be really interesting to add the outdoor temperature to this graph, but that’s a project for another day.
But I did add another data line to the graph—the HVAC state. It’s in dark blue below and a bit harder to read because the scale is so different, but those spikes are actual activations of the furnace. I can monitor system performance AND tell exactly for how many minutes the furnace has run (and when). Not bad, eh?
Cost breakdown
So how much did it cost me to do this project (in terms of time and money)? About four hours and $225. Much of that four hours was discovery, but if you follow my instructions, it will probably take you just an hour or so.
Nest thermostat: $250
Rebate from Questar (local gas company): -$100
Raspberry Pi kit: $75
Open source software: $0
Net cost: $225
Geek fun factor: Priceless
Do you have to be a DevOps professional to do this? No, not really, but a basic familiarity with Linux (and some basic handyman skills) will help. You should probably know where your circuit breakers are and which ones are connected to your HVAC equipment. And of course, set up your Scalyr account.
