Ep 1: Home Automation System Architecture

Discussion of the online video show SuperHouseTV, where Freetronics co-founder Jonathan Oxer hacks on his house using various Open Source hardware and software. [SuperHouseTV site]
User avatar
jonoxer
Freetronics Staff
Freetronics Staff
Posts: 297
Joined: Sat Oct 15, 2011 11:31 am

Ep 1: Home Automation System Architecture

Post by jonoxer » Thu Apr 12, 2012 11:54 am

Ep 1 is finally up. Please check it out! Yes, I know it's just a boring talking-head piece, but I already have other eps partly filmed that will show much more.

http://www.superhouse.tv/episode/home-a ... chitecture
--
Jon

User avatar
geekscape
Posts: 7
Joined: Tue Nov 01, 2011 4:58 pm
Location: Melbourne, Australia
Contact:

Re: Ep 1: Home Automation System Architecture

Post by geekscape » Fri Apr 13, 2012 1:11 am

hi Jon,

Nothing like putting some pressure on ourselves ... by going public :)

You mentioned that on your wall plates ... that you'll use two buttons (on and off) for each light ... rather than just one button (toggle). Any reason for that approach ?

For those listening closely, you may have noticed that Jon started talking about MQTT "channels" then switched to talking about MQTT "topics". They are usually called "topics", not "channels". Speaking consistently for 19 minutes straight isn't easy ... so, that's a trivial quibble.

For anyone wanting to know more about MQTT (light-weight publish-subscribe message transport), please see http://mqtt.org ... and the MQTT server used can be found at http://mosquitto.org

An important advantage of this approach is that anything (hardware or software application or user interface) can subscribe to an MQTT topic and be immediately notified of any changes (light on or off). This is very hard to replicate efficiently using HTTP, which is a request / response design ... fine for web-pages, but not for devices everywhere.

Jon mentioned "federation", which is a group of services, such as MQTT, that communicate with each other to form a larger system. Key design goals for this project are security and robustness. Rather than adding that later, we're building them in from the start. It's really hard to make an insecure system become secure later. It's really hard to make a single-point-of-failure design become more reliable over time.

Whilst the result needs to look like one system ... it is critical to distinguish between a typical approach, which is to have one instance of the most important parts, which we call "single point of failure". Instead, it is better to have a collection of replicated services with automatic fail-over that appear as a single "unified" system, not "centralized" (which implies just one).

Within the local network, there will be replication of services (unified). Across the wide-area network (between home, office, vehicles) there will be interconnection (federation). Dynamically creating new connections (and often application logic) between different parts of the system will allow new ideas to be quickly prototyped. As well as enable remote control of any part of the system from anywhere else.

akhe
Posts: 4
Joined: Fri Apr 13, 2012 7:03 am
Location: Los, Sweden
Contact:

Re: Ep 1: Home Automation System Architecture

Post by akhe » Fri Apr 13, 2012 7:05 am

Have a look at VSCP - http://www.vscp.org

mr-russ
Posts: 16
Joined: Wed Jan 18, 2012 8:46 am

Re: Ep 1: Home Automation System Architecture

Post by mr-russ » Sat Apr 14, 2012 6:39 am

Great to see it happening, I've been following the blog waiting for this for quite a while. Nice to see you using the whiteboard you made Jon :)

It was great to see the whole design philosophy and understand the direction it's going in. I'm in the designing the system phase and trying to understand the most reliable way to automation without the requirement for a pc.

I have been looking a vscp and wondering how does that compare with the MQTT approach as I only heard about MQTT as part of the episode. Do anybody have any feedback on the reasons for this approach?

I agree with the question regarding 2 buttons vs 1. I thought you could do on/off/fade all with one button if you program it correctly.

From previous video blogs I've seen you have 1 arduino for a single light switch. Is that correct? I would have thought you could share arduino hardware where switches are near each other. Is there a reason for that or do I have it wrong?

User avatar
jonoxer
Freetronics Staff
Freetronics Staff
Posts: 297
Joined: Sat Oct 15, 2011 11:31 am

Re: Ep 1: Home Automation System Architecture

Post by jonoxer » Sun Apr 15, 2012 12:34 pm

mr-russ wrote:It was great to see the whole design philosophy and understand the direction it's going in. I'm in the designing the system phase and trying to understand the most reliable way to automation without the requirement for a pc.
Great! I'd love to see more discussion about overall architecture, which is one of the reasons I started talking about my projects publicly. I hope it'll stimulate more public discussion about other approaches because I'd love to have seen that myself. My idea with doing the videos was to consider what *I* would want to know about but haven't been able to find much documentation on, and just start talking about it. From my point of view if I found someone who was publishing in-depth videos about their home automation projects as a holistic series, not just a quick "hey look what I built!" thing on YouTube, I'd watch every second of it looking for clues to help me with my own projects.

There's nothing wrong with those quick one-off videos about specific projects, of course, and I love watching those too, but they tend to be just about non-integrated hacks rather than going in depth into a system that has developed over time. I would *love* to see a complete, in-depth series explaining all the different aspects of how things are done, but I haven't found one yet.

So, anyone who sees these videos and has ideas or examples of how you'd like to build something or (better still) how you've *actually* built it, please speak up. I don't have all the answers, and I don't even know what many of the questions should be, so I'm relying on sucking the brains of people with more experience than me in order to pull this off.
mr-russ wrote:I have been looking a vscp and wondering how does that compare with the MQTT approach as I only heard about MQTT as part of the episode. Do anybody have any feedback on the reasons for this approach?
And that sort of thing is *exactly* why I want to hear from other people! I hadn't even heard of VSCP until @akhe's post, and I don't know anything about it other than what I've gleaned from a quick look at the website. From there I've learned that @akhe is actually the founder of the VSCP project so he's pretty much the best expert we could possibly have on the topic. I love it when I'm made aware of cool things I hadn't heard of before.

Just at first glance it appears that VSCP and MQTT are trying to solve very similar problems, and they seem to do it in quite similar ways. I just happen to have come across MQTT through people I know who have experience with it including @geekscape (Andy Gelme) but it looks like VSCP would be a very admirable alternative. If I'd come across VSCP first I may well have ended up using that.
mr-russ wrote:I agree with the question regarding 2 buttons vs 1. I thought you could do on/off/fade all with one button if you program it correctly.
Sure, easy. That would just be a software change in the logic that processes the incoming events from panels and dispatches events to the switchboards. In future I could certainly do that, but the reason I went for pairs of buttons initially was just to avoid having to maintain any concept of "state" centrally. Right now the logic code is sessionless and has no local storage: it just does a lookup from incoming button events and dispatches something as a result. It couldn't be much simpler (or dumber) which is intentional.

Maintaining a concept of state is certainly something I want to do, because it will also allow indication back to the panels. Right now the buttons are constantly illuminated, but their illumination could change to indicate the state of the device they're associated with. Changing the state of a device using one panel would require other panels associated with the same device to also change their displayed state.
mr-russ wrote:From previous video blogs I've seen you have 1 arduino for a single light switch. Is that correct? I would have thought you could share arduino hardware where switches are near each other. Is there a reason for that or do I have it wrong?
You're right, that would certainly be possible. In practice though I haven't found many situations where it actually happens: I can't think of a single location in the house where there are switches nearby on adjacent sides of a wall, or somewhere that it would be trivial to run a cable between them. The obvious scenario would be if you have a switch inside a bedroom door and another switch in the hallway on the other side of the door, but due to the geometry of our house that hasn't happened. Even switches that are physically within a meter of each other are in positions where they're on different segments of wall, or otherwise isolated.
--
Jon

evan
Posts: 3
Joined: Wed Nov 02, 2011 11:19 pm

Re: Ep 1: Home Automation System Architecture

Post by evan » Mon Apr 16, 2012 1:22 am

[...] the reason I went for pairs of buttons initially was just to avoid having to maintain any concept of "state" centrally. [...] Maintaining a concept of state is certainly something I want to do [...]
It's funny, but "What about current state?" was the first thing I thought of while watching your video wrt MQTT. (I admit I assumed I was either missing something, or there was something already in MQTT to address this.)

What happens when everything comes back up after a power outage? How do all the lights (and other systems) figure out what state they should be in, or do they just go for a sensible default? (e.g., Hallway and large room lights on?)

I was wondering if something more like SNMP would be useful here? Or a persistent state maintainer using something like CouchDB that can spread it among various nodes in your house?

While I'm here, some other questions/comments:

Are you able to control brightness of the lights (i.e. dimming) and how do you do that with your two button model?

Have you considered putting light sensors in rooms and controlling light brightness compared to available daylight (à la the adaptive house?)

Recently saw this video (youtube) where they guy set up his entire house with DC lights because the flicker in AC lighting bothered him (not much technical detail though).

E.

ralight
Posts: 1
Joined: Tue Apr 17, 2012 5:21 pm

Re: Ep 1: Home Automation System Architecture

Post by ralight » Tue Apr 17, 2012 5:38 pm

evan wrote:
[...] the reason I went for pairs of buttons initially was just to avoid having to maintain any concept of "state" centrally. [...] Maintaining a concept of state is certainly something I want to do [...]
It's funny, but "What about current state?" was the first thing I thought of while watching your video wrt MQTT. (I admit I assumed I was either missing something, or there was something already in MQTT to address this.)
I think you could do this using retained or "last known good" messages. Each topic in MQTT can have a single retained/last known good value associated with it (just set the retained flag when you publish to the topic to create this value). When a client subscribes to the topic it immediately gets the retained message sent to it if it exists. Retained messages are persisted to disk at intervals so can recover from a loss of power.

So the control could publish to lounge/light/command and subscribe to lounge/light/status or something similar.

Cheers,

Roger

akhe
Posts: 4
Joined: Fri Apr 13, 2012 7:03 am
Location: Los, Sweden
Contact:

Re: Ep 1: Home Automation System Architecture

Post by akhe » Mon Apr 23, 2012 10:30 am

Sorry, I actually lost the URL for the forum....

I will add MQTT support to VSCP & Friends (the software we distribute). VSCP is an application layer protocol and use other protocols such as MQTT for transport. So they are not "one of" but rather a "together" solution. We try to implement as many ways as possible for communication. VSCP can be run nativly over Ethernet, TCP/IP, CAN, USB, RS-232 or whatever also of course. This is not the mainpoint though why it was created. What we wanted was a way to

- Make things that can be setup once and then will work autonomous.
- Make it possible to abstract the actual hardware from the control logic/ user interface software.
- Make it possible to configure everything in a common way.
- Let the actual device hold the information about how it can be used, how it can be configured etc etc.
- Make it possible to write software once.

So thinking this way you can build a software based/hardware based user interface that displays temperature and be able to interchange the actual hardware between different vendors.

The Paris relay module http://www.auto.grodansparadis.com/paris/paris.htmlis a typical example and also http://www.auto.grodansparadis.com/kelv ... c10ka.html

There are some vidoes here that shows some features http://vscp.org/docs.php

Currently I build some widgets that will make it easier to create usr interfaces in a standard way on phones, pc's and other devices. Some info here http://youtu.be/acbI14Fx_5E and the channel here http://www.youtube.com/user/eurosource The goal here is to create a tool with which it is possible to easily create an interface and the deploy it to a device. This will work with any hardware as long as it in somewhere have a driver that abstracts it into a VSCP device (which is possible for all devices).

It is a lot more to say about this but I better stop now.

I like you project a lot Jon and will follow it with interest.

Cheers
/Ake

akhe
Posts: 4
Joined: Fri Apr 13, 2012 7:03 am
Location: Los, Sweden
Contact:

Re: Ep 1: Home Automation System Architecture

Post by akhe » Mon Apr 23, 2012 10:46 am

Have to make an addition and will try to be short this time...

So VSCP is more about what you have as payload in MQTT or whatever protocol you decide to use.

Cheers
/Ake

mlinnen
Posts: 3
Joined: Tue Apr 24, 2012 1:09 am

Re: Ep 1: Home Automation System Architecture

Post by mlinnen » Tue Apr 24, 2012 2:32 am

Hi Jon,
Awesome to see your progress on superhouse. I love the idea about detailing the whole infrastructure of the complete solution. There is so much someone can learn from such detailed content.

I really think the pub/sub messaging infrastructure is the way to go. It makes so much more sense to have an infrastructure where multiple subscribers can react to a single message. A simple message like a change in the weather forecast condition could trigger a whole bunch of things. The HVAC system might decide on changing the temperature setting due to these weather changes. The lawn irrigation system might decide not to water the lawn if rain is in the forecast. The porch awning might roll up due to a forecast of high winds.

I agree with a previous post that VSCP looks like a message format rather than a transport mechanism (although I just learned about VSCP here too). I have also looked into what transport and message format I should use in my future automation project. I am looking at leveraging Windows Azure Service Bus as the back bone for my message bus. I have also looked at using xPL http://xplproject.org.uk/ for the message format (as well as standards on maintaining state). Of course if I tie myself to a cloud service as a bus infrastructure that has good an bad points. Good - I can gain access to my house from anywhere, Bad - I am dependent on my in house system to always have an internet connection.

I also struggle on where the state of the system should be maintained. Either the distributed devices are smart and maintain their own state and are capable of exposing that state through messaging or these devices are somewhat dumb and they rely on a central processing unit to maintain their state and help them make the smart decisions. I am leaning toward the centralized smarts and state management. I find it more troublesome to push too much logic into the distributed devices. This just makes it difficult to troubleshoot problems and rolling out new software upgrades to the system. So I am a fan of your central decision making logic on the main server.

I am looking forward to the future videos!!

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests