Emonhub and OEM Gateway [SOLVED]

I have Emonhub/EmonPi and did configure OEM Gateway to send data from Arduino directly to my Raspberry USB. I had 3 issues:

1) /var/log/oemgateway is deleted all the time automatically.
2) localhost url do not updates Nodes directly, but inputs.
3) node sent from Arduino is just "5", and in serial I have "5", but in URL I have 5.0 and data is sent to a wrong place.

Does I really need to do this special configuration and execute OEM Gateway or it can be done directly from EmonPi? (read Arduindo data from USB port)

Is possible to solve the node number problem?

Thanx a lot!

pb66's picture

Re: Emonhub and OEM Gateway [SOLVED]

I'm not quite sure what it is exactly you have or are trying to achieve but you should not be using OEMG and emonHub together, emonHub replaced OEMG sometime ago and is included on the emonSD images.

When you say "send data from Arduino directly to my Raspberry USB" do you mean you have an Arduino connected by USB to the Pi? if so you can search this forum for "serial-direct" to get info on connecting a emonTx to the Pi via the (GPIO) serial port and it's pretty much the same except the serial port is /dev/ttyUSB0 (assuming the arduino is the first or only usb device).

regarding the 3 issues

1) IF you are using a read-only OS which is normal for an OEM SDcard image, then this would be normal behavior.

2) I'm not sure but think you maybe referring to the nodes module in emoncms here, in which case yes, that to can be expected if you are using the emonPi SDcard image which has variants of emonHub and emonCMS written to work almost exclusively together, you can probally "get it to work" with OEMG but I have no clue as to why you would want to, you are certainly not likely to find anyone else having done it.

3) Yes, that was a bug in OEMG that fixed in emonHub depite not being a problem at the time, but it would seems to have become a problem in later versions if you are seeing an issue.

The simplest solution to most of the issues seems to be don't use OEMG,

Paul

allmac's picture

Re: Emonhub and OEM Gateway [SOLVED]

Thanx again... I did remove OEMG instalation.

Now I did Serial-direct configuration, solving error messagens using infos from forum:

http://openenergymonitor.org/emon/node/10941

However, as adding NodeID as first item, I am not able understand the error message.
I have 7 items, but it asks for 6? Emoncms.org and Local are not receiving data, and I understood in log informations that at least Emoncms.org should be receiving something.

2015-09-13 17:09:35,171 INFO emoncms remoto sending: http://emoncms.org/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[]&sentat=1442174975
2015-09-13 17:09:35,183 WARNING SerialDirect 611 RX data length: 6 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h']
2015-09-13 17:09:36,001 DEBUG emoncms remoto acknowledged receipt with 'ok' from http://emoncms.org
2015-09-13 17:09:38,016 DEBUG SerialDirect 612 NEW FRAME : 5 94 64 33 2 126 28
2015-09-13 17:09:38,023 WARNING SerialDirect 612 RX data length: 6 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h']

Like this, but I am not able to find what can be wrong, or if I should just remove NodeID infor.
http://openenergymonitor.org/emon/node/6167

[[5]]
    nodename = emonTxShield
    firmware = emonTxShield
    hardware = emonTxShield
    [[[rx]]]
        names = Node, Corrente1, Corrente2, Corrente3, Corrente4, Voltagem, Temperatura
        datacodes = h, h, h, h, h, h, h
        scales = 1, 1, 1, 1, 1, 1, 1
        units = N, W, W, W, W, V, C

I am loosing something?

Thanx!!

allmac's picture

Re: Emonhub and OEM Gateway [SOLVED]

Just to update information about Serial-Direct... now it is working.
Hope this will help somebody else.
Here is my configuration to keep Emonhub working modifying RFM2Pi configuration:

[hub]
loglevel = INFO #(default:WARNING)

[interfacers]
[[SerialDirect]]
     Type = EmonHubSerialInterfacer
      [[[init_settings]]]
           com_port = /dev/ttyACM0 #USB
           com_baud = 9600
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,
           subchannels = ToRFM12
           calibration = 110V #(UK/EU: 230V, US: 110V)
           baseid = 5 #(emonPi default:5)
           quiet = false #(default:true)

[[emoncms remoto]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = http://emoncms.org
        apikey = xxxxxxxxxxxxxxxxxxxxxxx
        senddata = 1
        sendstatus = 1

[nodes]
[[5]]
    nodename = emonTxShield
    firmware = emonTxShield
    hardware = emonTxShield
    [[[rx]]]
        names = CT1, CT2, CT3, CT4, Volt, Temp
        #datacodes = h, h, h, h, h, h #RFM2Pi
        #scales = 1, 1, 1, 1, 1, 1 #RFM2Pi
        #units = W, W, W, W, V, C #RFM2Pi

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

Hello Paul and allmec,

Not sure if anyone is out there still (last post was a month ago for this thread). I'm curious about the connection on the Arduino side- is it from Rx/Tx or the Arduino's USB port?

This begs the question: is it possible to interface with emonHub from the Arduino's USB port and how is that different from UART?

Thanks, Jake

pb66's picture

Re: Emonhub and OEM Gateway [SOLVED]

I'm not really familiar with an actual "Arduino" I have only ever used Arduino type boards, but I would expect it depends on how your sketch is written. From emonHub's side it can use the serial port via gpio or a USB connection, I have a rfm2pi on /dev/ttyAMA0 and a jeelink on /dev/ttyUSB0, I also frequently connect a emonTx or emonTH via a FTDI on USB too, 

Paul

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

Thanks for responding Paul.

The sketch uses Serial.print() just like the EmonTx Firmware- in fact it is a copy of "current only" EmonTx Shield sketch that adds in the node ID before each transmission. Viewing the serial monitor shows that data is cleanly displayed (for example, 7 69 3799 3482 3646) but I'm not sure if this data is passed through the USB connection on the Arduino or only through the Arduino's UART.

This seems like it would be an attractive solution since it would power the Arduino as well as provide transmission for someone building a custom front end. Do you think there is interest here for that? Is it worth starting a new thread or would I be better off looking elsewhere or trying another solution?

Thanks, Jake

pb66's picture

Re: Emonhub and OEM Gateway [SOLVED]

I see no reason why it shouldn't work, if I had one I would probably know the answer so perhaps someone else can chip in, 

"Viewing the serial monitor shows that data is cleanly displayed(for example, 7 69 3799 3482 3646) but I'm not sure if this data is passed through the USB connection on the Arduino or only through the Arduino's UART."

How is it connected for that result? I assume you don't have both connected at once so if it's by USB then yes it will work, using the same "/dev/tty???" and baud your serial monitor is using, or if it's a different machine try /dev/ttyUSB0 or Allmac seems to be using /dev/ttyACM0.

Paul

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

To get that result, the Arduino is connected via its USB port to a USB port on my desktop machine. When I plug the Arduino into my Pi running emonHub, I get the "SerialDirect Thread is dead" error. I am unclear on how to proceed from here- I'm using allmac's interfacer above and I know that I also need to use /dev/ttyACM0.

Any further insights/questions are welcome. I will keep googling & troubleshooting, and if I end up creating a new thread I will link it here.

Thanks, Jake

pb66's picture

Re: Emonhub and OEM Gateway [SOLVED]

"and I know that I also need to use /dev/ttyACM0."

Are you sure? have you tried ttyUSB0? is it the only or first connected usb device?

At the commandline type 

ls -la /dev/tty* 

is your Arduino found?

In emonhub.conf set loglevel = DEBUG and restart emonhub, the creation of each interfacer is logged in emonhub.log you may find some info there, are the attempts settings correct? is the creation successful? or does it "die" during creation?

Paul

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

After typing ls-la /dev/tty* at the command line, one line is

crw-rw---T 1 root dialout 166,  0 Jun 17 14:40 /dev/ttyACM0

whereas there is no result for /dev/ttyUSB0. Further, that line is absent when I unplug the Arduino, leaving me fairly confident that ACM0 is what I should use.

My loglevel was set to INFO, I have changed it to DEBUG. How does one restart emonhub? Regardless, I restarted the Pi and got the same result.

The creation of the interfacer does not fail according to emonhub.log. It reads as follows:

  • MainThread Creating EmonHubSerialInterfacer 'SerialDirect'
  • MainThread Opening serial port: /dev/ttyACM0 @ 9600 bits/s
  • MainThread Setting SerialDirect subchannels: ToRFM12
  • MainThread Interfacer: Subscribed to channel' : T
  • MainThread Interfacer: Subscribed to channel' : o
  • MainThread Interfacer: Subscribed to channel' : R
  • MainThread Interfacer: Subscribed to channel' : F
  • MainThread Interfacer: Subscribed to channel' : M
  • MainThread Interfacer: Subscribed to channel' : 1
  • MainThread Interfacer: Subscribed to channel' : 2
  • MainThread Setting SerialDirect pubchannels: ['ToEmonCMS']
  • MainThread Interfacer: Subscribed to channel' : T (then o, R, F, M, 1, 2 as above)
  • MainThread MQTT Subscribed to channel' : ToEmonCMS
  • MainThread emoncmsorg Subscribed to channel' : ToEmonCMS
  • MainThread SerialDirect thread is dead...

Therefore I don't believe that the creation of the interfacer "dies" during creation. Is it worrisome that the interfacer runs through ToRFM12 in "subscribed to channel"?

Thanks, Jake

pb66's picture

Re: Emonhub and OEM Gateway [SOLVED]

Try adding a comma to the end of the "subchannels = ToRFM12" line in [[SerialDirect]], it looks like the missing comma causes the list to be interpreted as a list of letters rather than a list containing one item.

Paul

allmac's picture

Re: Emonhub and OEM Gateway [SOLVED]

Where are you connecting your Arduinos's EMONCMS USB? Windows? Linux? Mac?

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

Paul, adding that comma changes the log to "MainThread Interfacer: Subscribed to channel' : ToRFM12" as desired, but I still get the warning "MainThread SerialDirect thread is dead." After backing up and trying to implement SerialDirect on a standard emonTx shield connected via serial (AMA0), I'm realizing that the Tx LED never comes on and I get the same results as above. I think I'd better search around more on here to solve that problem first. Any hints or suggestions on what to search for?

allmac, not entirely sure what you mean by my Arduino's "emoncms USB." In my previous attempts, I connected Arduino USB to desktop (ubuntu) USB to load the sketch and got the serial monitor results stated above (expected serial output). Then I would connect Arduino USB to Pi USB and get "MainThread SerialDirect thread is dead." Does that help?

Thanks, Jake

allmac's picture

Re: Emonhub and OEM Gateway [SOLVED]

I have in my Pi the following:

1) Arduino's software output to Serial... you can try it using Arduino's Devtool Serial Monitor.

2) Arduino USB connected to Raspberry USB (OK)!

3) Serial Monitor tool installed in my Pi: http://www.hobbytronics.co.uk/raspberry-pi-serial-port

Now Arduino is sending data do serial, and you can use Serial Monitor software to view data in your Pi.

If you can see data in your Pi's Serial Monitor, than you are already receiving all right. My configuration file here at the top can help in EMONCMS configuration reading Serial Monitor's data.

I have also Arduino's Serial Tools installed in my Pi. Than I can use Pi to modify and update Arduino software: http://inotool.org

allmac's picture

Re: Emonhub and OEM Gateway [SOLVED]

This is an working version of my Arduino's software.
I need to clean, optimize and calibrate, because I have some negative data do CT.
But it works fine, receiving Arduino's data in Pi via USB.

To view data in my Pi I use this: minicom -b 9600 -o -D /dev/ttyACM0

/*Recommended node ID allocation

------------------------------------------------------------------------------------------------------------

-ID-    -Node Type- 

0    - Special allocation in JeeLib RFM12 driver - reserved for OOK use

1-4     - Control nodes 

5-10    - Energy monitoring nodes

11-14    --Un-assigned --

15-16    - Base Station & logging nodes

17-30    - Environmental sensing nodes (temperature humidity etc.)

31    - Special allocation in JeeLib RFM12 driver - Node31 can communicate with nodes on any network group

-------------------------------------------------------------------------------------------------------------

*/

 

#define FILTERSETTLETIME 5000                                           //  Time (ms) to allow the filters to settle before sending data

 

const int CT1 = 1; 

const int CT2 = 1;                                                      // Set to 0 to disable 

const int CT3 = 1;

const int CT4 = 1;

 

#define RF_freq RF12_868MHZ                                            // Frequency of RF12B module can be RF12_433MHZ, RF12_868MHZ or RF12_

915MHZ.

const int nodeID = 5;                                                  // emonTx RFM12B node ID

const int networkGroup = 210;                                          // emonTx RFM12B wireless network group - needs to be same as emonBas

e and emonGLCD                                                 

 

const int UNO = 1;                                                     // Set to 0 if your not using the UNO bootloader (i.e using Duemilano

ve)

#include <avr/wdt.h>

 

#define RF69_COMPAT 0                                                  // set to 1 to use RFM69CW 

#include <JeeLib.h>                                                    // make sure V12 (latest) is used if using RFM69CW

ISR(WDT_vect) { Sleepy::watchdogEvent(); }                             // Attached JeeLib sleep function to Atmega328 watchdog -enables MCU 

to be put into sleep mode inbetween readings to reduce power consumption 

 

#include "EmonLib.h"

EnergyMonitor ct1,ct2,ct3, ct4;                                        // Create  instances for each CT channel

 

#include <OneWire.h>

#include <DallasTemperature.h>

#define ONE_WIRE_BUS 4                                                 // Data wire is plugged into port 2 on the Arduino

OneWire oneWire(ONE_WIRE_BUS);                                         // Setup a oneWire instance to communicate with any OneWire devices (

not just Maxim/Dallas temperature ICs)

DallasTemperature sensors(&oneWire);                                   // Pass our oneWire reference to Dallas Temperature.

 

// By using direct addressing its possible to make sure that as you add temperature sensors the temperature sensor to variable mapping will 

not change.

// DeviceAddress address_Temperatura = { 0x28, 0x57, 0x6E, 0x7F, 0x04, 0x00, 0x00, 0xAA };

 

DeviceAddress address_Temperatura = { 0x28, 0x57, 0x6E, 0x7F, 0x04, 0x00, 0x00, 0xAA };

 

typedef struct { int Corrente1, Corrente2, Corrente3, Corrente4, Voltagem, Temperatura;} PayloadTX;     // create structure - a neat way of 

packaging data for RF comms

PayloadTX emontx;                                                       

 

const int LEDpin = 9;                                                   // On-board emonTx LED 

 

boolean settled = false;

 

void setup() 

{

  Serial.begin(9600);

  digitalWrite(LEDpin, HIGH); delay(1000);

  digitalWrite(LEDpin, LOW); delay(1000);

  digitalWrite(LEDpin, HIGH); delay(1000);

  digitalWrite(LEDpin, LOW);

 

  //Serial.println(F("\n================================================"));

  //Serial.println(F("Ligado...\n"));

  //Serial.print("Node: "); 

  //Serial.println(nodeID); 

  //Serial.print("Freq: ");

  //if (RF_freq == RF12_433MHZ) Serial.println("433Mhz");

  //if (RF_freq == RF12_868MHZ) Serial.println("868Mhz");

  //if (RF_freq == RF12_915MHZ) Serial.println("915Mhz"); 

  //Serial.print("Network: "); 

  //Serial.println(networkGroup);

  //Serial.println(F("================================================\n"));

 

   

  if (CT1) ct1.current(1, 60.606);                                     // Setup emonTX CT channel (ADC input, calibration)

  if (CT2) ct2.current(2, 60.606);                                     // Calibration factor = CT ratio / burden resistance

  if (CT3) ct3.current(3, 60.606);                                     // emonTx Shield Calibration factor = (100A / 0.05A) / 33 Ohms

  if (CT4) ct4.current(4, 60.606); 

  

  if (CT1) ct1.voltage(0, 134.7, 1.7);                                // ct.voltageTX(ADC input, calibration, phase_shift) - calibration for

 AC-AC adapter  http://openenergymonitor.org/emon/modules/emontx/firmware/calibration.                                          

  if (CT2) ct2.voltage(0, 134.7, 1.7);                                // Default set for Ideal Power adapter 268.97 for the UK adapter, 260 

for the Euro and 130 for the US.

  if (CT3) ct3.voltage(0, 134.7, 1.7);

  if (CT4) ct4.voltage(0, 134.7, 1.7);

    

  rf12_initialize(nodeID, RF_freq, networkGroup);                    // initialize RFM12B

  rf12_sleep(RF12_SLEEP);

 

  pinMode(LEDpin, OUTPUT);                                           // Setup indicator LED

  digitalWrite(LEDpin, HIGH);

                                                                                     

}

 

void loop() 

{

  

  if (UNO) wdt_enable(WDTO_8S); 

  

  sensors.requestTemperatures();                                     // Send the command to get temperatures

  emontx.Temperatura = sensors.getTempC(address_Temperatura);

  

  if (CT1) {

    ct1.calcVI(20,2000);                                             // Calculate all. No.of crossings, time-out 

    emontx.Corrente1 = ct1.realPower;

    //Serial.print(emontx.Corrente1);                                         

  }

  

  if ((ct1.Vrms) < 10) { ct1.Vrms = 0;}

  emontx.Voltagem = ct1.Vrms;                                         // AC Mains rms voltage

  

  if (CT2) {

    ct2.calcVI(20,2000);                                             // Calculate all. No.of crossings, time-out 

    emontx.Corrente2 = ct2.realPower;

    //Serial.print(" "); Serial.print(emontx.Corrente2);

  } 

 

  if (CT3) {

    ct3.calcVI(20,2000);                                             // Calculate all. No.of crossings, time-out 

    emontx.Corrente3 = ct3.realPower;

    //Serial.print(" "); Serial.print(emontx.Corrente3);

  } 

  

  if (CT4) {

    ct4.calcVI(20,2000);                                             // Calculate all. No.of crossings, time-out 

    emontx.Corrente4 = ct4.realPower;

    //Serial.print(" "); Serial.print(emontx.Corrente4);

  }

  

  //Serial.print(" "); Serial.print(emontx.Voltagem);

  //Serial.print(" "); Serial.print(emontx.Temperatura);

  //Serial.println(); delay(100);

 

  if (!settled && millis() > FILTERSETTLETIME) settled = true;        // because millis() returns to zero after 50 days ! 

 

  if (settled)                                                        // send data only after filters have settled

  { 

    if (UNO) wdt_reset();

    send_rf_data();                                                       // *SEND RF DATA* - see emontx_lib

    Serial.print("5");Serial.print(" ");

    Serial.print(emontx.Corrente1);Serial.print(" ");

    Serial.print(emontx.Corrente2);Serial.print(" ");

    Serial.print(emontx.Corrente3);Serial.print(" ");

    Serial.print(emontx.Corrente4);Serial.print(" ");

    Serial.print(emontx.Voltagem);Serial.print(" ");

    Serial.print(emontx.Temperatura);Serial.println();

    digitalWrite(LEDpin, HIGH); delay(2); digitalWrite(LEDpin, LOW);      // flash LED

    delay(2000);                                                          // delay between readings in ms

  }

}

 

void send_rf_data()

{

  rf12_sleep(RF12_WAKEUP);

  // if ready to send + exit loop if it gets stuck as it seems too

  int i = 0; while (!rf12_canSend() && i<10) {rf12_recvDone(); i++;}

  rf12_sendStart(0, &emontx, sizeof emontx);

  // set the sync mode to 2 if the fuses are still the Arduino default

  // mode 3 (full powerdown) can only be used with 258 CK startup fuses

  rf12_sendWait(2);

  rf12_sleep(RF12_SLEEP);

}

Robert Wall's picture

Re: Emonhub and OEM Gateway [SOLVED]

It is not as important now to allow time for the filters to settle, since emonLib now uses a low pass filter which can be initialised to a value that is close to the final, settled, value.

johlhausen's picture

Re: Emonhub and OEM Gateway [SOLVED]

Hi friends,

Thank you for all your help and advice. After feeling frustrated and quitting over the weekend, I came back tonight with encouraging results. I will attempt to document my progress here. Background: I have an emonTx shield (v. 2.5) on a knock-off Arduino Uno. It is connected from the Arduino's USB-B type port to a USB port on a Raspberry Pi B+ running the standard emonHub image.

  1. On my desktop, I opened the Shield_CT1234 in Arduino IDE and made the following modifications:
    • Change RF_COMPAT to 1 
    • Added "Serial.print(nodeID); Serial.print(" ");" after void loop()
    • (Changed the AC RMS voltage from 240.0 to 120.0 because America)
  2. Uploaded to Arduino and verified output formatting of type "6 53 42 16 33" via serial monitor.
  3. Opened emoncms by typing in Pi's IP address in my desktop's web browser and made the following changes to the stock emonHub.conf:
    • com_port to /dev/ttyACM0
    • com_baud to 9600
    • datacode to B
    • (calibration to 110V)
    • From node 6, eliminate Vrms and its scale & units. Also change datacode to B
  4. Plug Arduino into Pi via USB and it works!

I feel pleased with this since it modifies existing OEM code as little as possible. Perhaps this will be useful to others desiring an easy serial connection. Paul, allmac, what questions can I answer to make this thread as useful to you and other visitors as possible?

My endgame with this is to create a custom Arduino front-end with an Arduino Mega to measure 16 CTs at once. With two such custom Arduinos connected to a Pi, perhaps I can build an all-in-one box to measure all 30 circuits in a full standard U.S. breaker box plus the two mains. This should cost less than $400 and have only one power cord. If I'm successful, I'll do my best to post here and make a guide for others.

Again, thanks for your help Paul and allmac. Jake

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.