Wow ... that's a lot of data!

If you thought that we've gone away ... you're partly right, we've been away but not in a bad way!
About a week ago we got our first full set of data from a few of the participants in Rangiora and just to give you an idea ... for every home  we have about one million records with between 5 and 10 parameters each ... that's a lot of numbers!
So, yes, we've been away but it has been trying to get our heads around this large dataset ... with various levels of success... as a German general said in the 1800s: "No plan survives contact with the enemy".

At the beginning of this campaign we thought that the temperature data obtained through the iButtons and BRANZ's loggers was going to be simple to interpret. High temperature = heater on, low temperature = heater off. The next level in complexity would be ECan's temperature and $PM_{10}$ data as it is standard compliant and we have a long time series for that location that would give us some context (outside temperature) for the heater use identified from the indoor sensors. Next up is the ODIN data that we'd need to correct and normalize before using it but it would give us information about the spatial gradients in the town. Finally, the most complex data are coming from PACMAN that would give us insights on the levels of $PM$ indoors and relate it to the home heating activities and the outdoor concentrations reported by ECan.

When we got the first set of temperature data we saw that things weren't going to be that straight forward. Here you can see a plot of all the indoor temperature data grouped by type of sensor, with the outdoor data reported by ECan.

So, the temperature didn't go as high as we were expecting (except for one home) which makes our analysis a little difficult if we are to define a threshold that identifies heater on/off. Then is the relationship between indoor and outdoor temperature that we were expecting to exploit to identify when people are more likely to turn on their burners. The next plot is from one home (I'm not saying which one) where we can see that when it gets cold outside, it gets warm inside, suggesting that the heater is on. However, the temperature at which that home lit their burner is not consistent. Most of the time, when the outdoor temperature dropped below 10C, the indoor temperature started increasing (heater on) but there are many instances where the ambient temperature is above 10C and the heater seems to be on (BRANZ temperature above 40C!)...

So much for a simple relationship!

Next time I'll introduce you to the joy of PACMAN data ... you've been warned!