how much space does logging take?

Submitted by boelle on Fri, 12/09/2014 - 20:41

the big box of fun from OEM have just been scanned first time here so i will get it monday...

now juggling with data so i can get a HDD free to log data on...

but then how much space will the data take?

i plan to have one emontx logging:

temp from my hot water cylinder
Energy, Current Power, Temperature t1, Temperature t2, Temperature diff, Flow, Volumen 1
pulse/blinker from the electricity meter

i plan to update to emoncms no less than each 5 sec, it gives the heat meter enough time to be ready to send data again. I think i could go as low as every 2-3 sec. not sure yet.

then another emontx will monitor my water meter and update also every 5 sec. i plan to send to emoncms how many liters used between readings. flowrate in liters/sec. on emoncms i will then calc total liters since start of logging.

on top of this i'm part of OpenTRV dev team and i plan to send in the folllowing from each box: temp, bat. level, light level, valve % open

what would a good educated guess be for data consumption be? ie how many mb each day

Re: how much space does logging take?

Submitted by boelle on Tue, 16/09/2014 - 10:25.

could also ask how much space logging of a single temp for one day takes and then multiply with how many different things i log to get an approx number

Re: how much space does logging take?

Submitted by Robert Wall on Tue, 16/09/2014 - 10:54.

I think you'll find some answers in Trystan's investigation into the various database engines. There's a current thread that should help you http://openenergymonitor.org/emon/node/5387

Re: how much space does logging take?

Submitted by TrystanLea on Tue, 16/09/2014 - 11:34.

Here's the documentation on the storage engines: https://github.com/openenergymonitor/documentation/tree/master/BuildingBlocks/TimeSeries

PHPFina (fixed interval no averaging) 4 bytes per datapoint, interval is fixed so if logging is every 10 seconds thats:

86400 seconds in a day

8640 datapoints in a day

8640 x 4 = 34560 bytes per day (34kb/day)

PHPTimeSeries is variable interval and records the timestamp, it uses 8 bytes per datapoint (68kb/day)

PHPFiwa has additional average layers (60 second, 600 second and 3600 second averages)

10s layer: 8640 dp
60s layer: 1440 dp
600s layer: 144 dp
3600s layer: 24 dp

TOTAL 40kb/day

Re: how much space does logging take?

Submitted by boelle on Tue, 16/09/2014 - 13:02.

so about 10 datapoints i have (star wars nerd yes)

so logging every 4 sec must mean i log 21600 times a day

an each time it should be 80 bytes i calc correct... it must be 1728000 bytes each day.

i get it to roughly 1,65 mb each day... am i correct so far?

for a whole year non stop it would be about 601.5 mb?

Archived Forum