Add data from csv file to emoncms

Hi

I have got data logged on cvs file I would like to read it and use in emoncms .

Any ideas?

Max

Jérôme's picture

Re: Add data from csv file to emoncms

I don't think emoncms has this kind of import feature.

However, you could write a script that would read the .csv file and generate URLs according to the input API. That would be a nice and not so difficult way to learn the insides of the project.

There's an API help once you're logged in emoncms. In the input tab, if I remember well.

bumax's picture

Re: Add data from csv file to emoncms

yes

it works now I am able to insert data from my weather station :-) with my pi

Thanks

Jérôme's picture

Re: Add data from csv file to emoncms

Great.

Maybe you can post your script here in case it would be useful to anyone.

What kind of weather station is it ? Is it common ?

If the csv format is mainly used, other people may be interested.

Rickenbacker's picture

Re: Add data from csv file to emoncms

I for one would be interested.

bumax's picture

Re: Add data from csv file to emoncms

The weater station is TFA Dotsmann DV928.

I don 't know if it s common but I found some idea on web.  I connect it to my Rpi with usb.  Using lusblib and TE923 tools.

I made a litle script that read data from usb with TE923 and generate urls.  And cron every 5 minutes.  I am able to use T° inside outside, humidy inside ouside, windspeed, pressure, rain gauge, and some other informations (wind direction,...).

All run on Rpi, EmonCms too.  I put db on usb harddrive.  Hope for along live :-)

I would like to make a différent widget for them like raingauge, or wind direction.  I have to read documentation ;-)

Hope it could help someone.

Max

Jérôme's picture

Re: Add data from csv file to emoncms

It may be interesting to integrate this feature in the gateway of the emonBase.

I can't do that myself without the test setup, but if anyone is interested, I can help. In this case, your script and maybe example files would help.

bumax's picture

Re: Add data from csv file to emoncms

Here is my script. 

Be indulgent I don't dev on linux ;-) normally.

te923con return value from weather station

here is the result

1385397125:17.90:46:3.50:80:i:i:i:i:i:i:i:i:1007.1:i:5:0:1:0.2:0.5:3.1:1069

Unixtime:T0:H0:T1:H1:T2:H2:T3:H3:T4:H4:T5:H5:PRESS:UV:FC:STORM:WD:WS:WG:WC:RC

-  Unixtime - Current date / time in a unixtime stamp format.
-  T0    - temperature from internal sensor in °C
-  H0    - humidity from internal sensor in % rel
-  T1..5 - temperature from external sensor 1..4 in °C
-  H1..5 - humidity from external sensor 1...4 in % rel
-  PRESS - air pressure in mBar
-  UV    - UV index from UV sensor
-  FC    - station forecast, see below for more details
-  STORM - stormwarning; 0 - no warning, 1 - fix your dog
-  WD    - wind direction in n x 22.5°; 0 -> north
-  WS    - wind speed in m/s
-  WG    - wind gust speed in m/s
-  WC    - windchill temperature in °C
-  RC    - rain counter (maybe since station starts measurement) as value

   weather forecast means (as precisely as possible)
     0 - heavy snow
     1 - little snow
     2 - heavy rain
     3 - little rain
     4 - cloudy
     5 - some clouds
     6 - sunny

from: http://www.redeo.nl/2012/03/connecting-your-weather-station-to-your-nas-...

He does the same thing but without emoncms :-(

#!/bin/bash

OIFS=$IFS;
IFS=":";

var=$(/data/te923tool-0.5/te923con);
varArray=($var);

IFS=$OIFS;

curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFATempInt:${varArray[1]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFAHumiInt:${varArray[2]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFATempExt:${varArray[3]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFAHumiExt:${varArray[4]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFAPressur:${varArray[13]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFAWindSpe:${varArray[18]}}"
curl "http://xxx/emoncms/input/post?apikey=xxx&json={TFARainGau:${varArray[21]}}"

First time curl post 2 values, first one correct value and second time only 0.  So graphic was so ugly.  But now it's ok.  At the moment I don't know why it send that in the same time.

It works since yeasterday.  For the moment without problem just with cron and email...  I would like to sens a daily repport by mail.  But I solved that too.

I am interesting by new widget ;-)

Max

Jérôme's picture

Re: Add data from csv file to emoncms

Thanks.

I think I get the idea. It should be possible to add a listener type to the gateway, to enable this in a few clicks.

(Be careful, it seems you just pasted your apikey publicly. You may want to replace it by xxxxxxx or anything to hide it.)

bumax's picture

Re: Add data from csv file to emoncms

You are right,

Now it's ok thank for advise

Max

rsegoly's picture

Re: Add data from csv file to emoncms

I use Arduino for measurement .

Data sent in CSV format via Serial to PC and logged PC.

PC logs to file and also send to Emoncms to be presented.

I use Python  so it straight forward 

In same manner data will be stored on local database maybe as hosted Econcms or as 

Python script

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.