Archived Forum

Anybody know where I did wrong?

If only I can understand how emonhub works, I just started learning Python 2 week ago because of this. Haha. I know that emonhub is in the background receiving, parsing, processing, buffering and forwarding the data to emoncms. But I am afraid the process is too complicated for me to understanding at this moment. So I can't hitch hike emonhub and get the output from there. So if someone could point me in some direction which (function/method) to focus on then it would be great.

Since I have no idea on that, I proceed to read the port myself but it seems that it is not so straight forward.

I don't know Python at all, but there are very many tutorials available that should point you in the right direction. Try searching for "python read from serial port example". It might be more helpful if you include in your search the platform that you're working on.

I've programmed in many languages and variants of languages, a few points to think about:
Are you trying to read characters one by one or complete lines?
Are you trying to interpret what comes in, or so you just want to see the ASCII code values?
What happens to your script if no data is available - will it wait forever or fail?

"So if someone could point me in some direction which (function/method) to focus on then it would be great."

What is it that you wish to do ? 

there is a small serial logging script written for debugging on the "emonTH Unreliable reading timing?" thread that may help as it is "just" a basic serial reader.

Paul

@Robert Wall

Me too! I am not well verse in Python, I only know C++ and C# well. Which is why I did search for examples and got the above script and edited a little to suits my need.

To answer your questions:

I want to read everything that is passed from the emonTx V3 to the raspberry, I believe the readlines() does that, I will filter out the unwanted stuff after I can get the correct reading. One step at a time.

I have included a Timeout at 3secs, so it will not wait forever for the data, if there is no data then my script will read nothing and still stop. I will use write another crontab ftp script to send the data into my server hosted in my laptop.

@Paul

What I wish to do now is to get everything that is passed into the raspberry. Either by

1. Reading from the serial port(which is what I am trying to do now but not really getting what I want)
2. Using emontx to do all the parsing and stuff for me and pass the Power readings from the CT into my own database. 

Both of the above option are not so great for me as I am not well verse in Python, I tried opening up emonhub to read but there are so many files I have no idea where to starts. Which function is the last step where it pass the data into emoncms. Knowing that particular function, I might be able to add in additional functions and pass that data into my own database(so I don't have to use emoncms). 

After I got the above working, I will send those data using a crontab ftp script and send those data into my own server hosted in my laptop. But one step at a time, so I need to get the above working first, before I can proceed this ftp script.

Thanks for all your help! :)

Any particular reason you want to write it in python?  Another option to consider is to just write your program (in the language of your choice) to read from stdin.   You can then use stty to configure the serial port as you want it, and when you run your program, simply tell it to read its stdin from the serial port of your choice with something like:

my_prog <  /dev/ttyAMA0

I would love to help you further but cannot fathom out what you are after, it's clear emoncms and emonhub are not doing what you want but beyond that it's really quite vague what you do want, I have read your various other threads and they all seem follow a common theme but I'm still none the wiser. 

You want to read the serial port - emonhub does that and you want to save data to a remote sql database - emoncms can do that. but apparently you do not want either of these softwares, If you want to write your own versions you should, as dBC suggests, do it in a language you do know or if you want emonhub/emoncms to "do something else" you need to explain what that is.

Paul

@dBC

Reason being I did not know I can write it in another language! Haha, I always thought raspberry pi run in linux so it can only run Python.

@Paul

For the current system I did now,the data is send from emonTx >> Raspberry(emonhub) >> Emoncms(Remote) 

For the current system that I know I can now do, the data is send from emonTx >> Raspberry(emonhub) >> Emoncms(Remote) >> my own MQSQL/MYSQL

For the system I want to develop, the data is send from emonTx >> Raspberry(Any "platform")  >> my own MQSQL/MYSQL. 

This save me a step of using emoncms(don't really like that apikey thing). And if I can get the data into my own SQL table, I can do analysis using matlab. 

Actually, what I want is rather simple. A table with 5 columns. Timestamp, value of CT1, value of CT2, value of CT3, value of CT4. As long as I can get those values in a table in my own database that's good enough for me. :)

I think I have a better way to explain myself. In emoncms, there is a function to export into csv(under Feed). What I want is exactly like that. The first column is the timestamp, the second is the value. All these values to be inserted into my own database.

I believe this is exactly what is happening behind emoncms. The timestamp and the CT values. If I got a way to get these data, it will be awesome!

"I did not know I can write it in another language! Haha, I always thought raspberry pi run in linux so it can only run Python."

You can use any language you like, provided that you have a compiler, linker and libraries that will produce executable code for the processor and operating system in your machine. That is true for any computer, not just your RPi.

See http://www.dummies.com/how-to/content/top-10-programming-languages-porte...

Do not confuse the operating system (Linux) with the language you use to program in. There is nothing to directly link the two.

If you choose to write something from scratch to do this then many of the points raised by Robert with regard reading the serial port will also need considering when forwarding the data across a network, like what happens if there is no network or target found? Timeouts and retries etc. Will you just send the data and move on to the next or will you act on a confirmation of some sort?

You could write something to actually access the remote SQL server directly or you could use a middle man, you mentioned the csv from emonCMS, that is basically a text file, How did you plan on passing the csv files to your SQL tables?

If csv is acceptable you could put together a csv/text reporter for emonhub, that way emonhub is handling the serial read and parsing, buffering the data and/or saving as a text file locally (or remotely?. There have been csv reporters written in the past, see "How to log the data into USB memory stick via raspberry pi2 instead of sending to emoncms.org?" or "Emonhub Write to local sqlite database rather than post to web-target" for a couple of examples. 

There are many many different ways you could tackle this I guess.

Paul 

@Robert Wall

Thanks! It's dumb for me not to know that. That's good knowledge I have learnt today :)

@Paul

Thanks for the link, I guess I will just use emonhub to write the csv file. Will saved me the work since most functions are already written. (Reading the links now)

As for the passing of csv into SQL table, I have done it before in the past. It uses FTP, goes into a certain directory, recursively looping through all the folders and retrieve the entire directory's CSV files. Basically, it retrieve, transfer, insert, backup, delete.