Archived Forum

I'm not an expert - hopefully one will be along soon...

But I think you might need to install using the raspberry pi installation guide rather than using the SD image. Certainly worth a try. 

https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/readme.md

Simon

Was the direct serial connection working before you re-imaged? Its been a while since I last did a build using a direct serial connection between the emontx and emonpi - things have moved on quite a bit since then in terms of emoncms and emonhub version. Can you catch the point in the logs where the SerialDirect thread dies? There may be some more clues there.

Can I rename your thread to reflect the topic perhaps: "Need help, emonhub with emontx to pi direct serial connection"?

1st step would be to set emonhub's "loglevel = DEBUG" in emonhub.conf rather than WARNING which will give you more info.

The "WARNING MainThread SerialDirect thread is dead" error message can only be found in the emon-pi variant of emonhub and yet the emonhub.conf you have is for the original version of emonhub. There are no reporters in the emon-pi variant, the "EmonHubEmoncmsReporter" was replaced with a "EmonHubEmoncmsHTTPInterfacer".

There is also an absence of the routing settings (the to and from) required for the interfacers in that variant.

While you are in the conf you can get rid of the group, frequency and baseid settings as they are not used for serial direct, only needed for RFM networks, this won't be effecting the issues you have though.

Your example packet "10 513,25 510,44 -509,99 509,74 230,83" shows some comma's, is this a typo or a regional thing? emonhub will expect (depending on version) a frame in one of 2 forms, a nodeid followed by space separated decimal values eg "10 513.25 510.44 -509.99 509.74 230.83" or (less likely) a node id followed by space seperated "byte values" ie values between 0 and 255 to be reconstructed into full values using datacodes, either way the comma's could cause the packets to be discarded unless there is a "regional" adjustment made at the recieving end (emonhub) too.

The emonhub log should show you exactly whats happening there if you change the loglevel.

Also be aware that emoncms.org will probally block your address/device if you are posting at 2s intervals, the recommended rate is 10s, but you should be ok with a local install.

Paul

I am also having trouble with the "SerialDirect thread is dead" message when trying to connect via serial to the emon-pi emonHub variant 1.1.

The Arduino I am using works when connected via serial to my older emonHub, running the emonSD-13-03-15.img image that provided me with "emonHub Pre-Release Development Version (rc1.2)".

With the identical hardware setup, but running the emonSD-29Mar16 release candidate 2 image, I get the errors below. 

First, my interfacer config:

### This interfacer manages the USB port where the Moteino base is communicating serial data
[[SerialDirect]]
     Type = EmonHubSerialInterfacer
      [[[init_settings]]]
           com_port = /dev/ttyUSB0 
           com_baud = 115200
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,
           subchannels = ToRFM12,

And here is the result:

2016-04-25 22:22:07,573 INFO     MainThread EmonHub emonHub 'emon-pi' variant v1.1

2016-04-25 22:22:07,573 INFO     MainThread Opening hub...

2016-04-25 22:22:07,574 INFO     MainThread Logging level set to DEBUG

2016-04-25 22:22:07,575 INFO     MainThread Creating EmonHubSerialInterfacer 'SerialDirect' 

2016-04-25 22:22:07,579 DEBUG    MainThread Opening serial port: /dev/ttyUSB0 @ 115200 bits/s

2016-04-25 22:22:07,579 DEBUG    MainThread Setting SerialDirect subchannels: ['ToRFM12']

2016-04-25 22:22:07,580 DEBUG    MainThread Interfacer: Subscribed to channel' : ToRFM12

2016-04-25 22:22:07,581 DEBUG    MainThread Setting SerialDirect pubchannels: ['ToEmonCMS']

2016-04-25 22:22:07,582 DEBUG    MainThread Interfacer: Subscribed to channel' : ToRFM12

2016-04-25 22:22:07,583 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT' 

2016-04-25 22:22:07,585 INFO     MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi

2016-04-25 22:22:07,590 DEBUG    MainThread MQTT Subscribed to channel' : ToEmonCMS

2016-04-25 22:22:07,592 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg' 

2016-04-25 22:22:07,594 DEBUG    MainThread emoncmsorg Subscribed to channel' : ToEmonCMS

2016-04-25 22:22:07,692 INFO     MQTT       Connecting to MQTT Server

2016-04-25 22:22:07,695 INFO     MQTT       connection status: Connection successful

2016-04-25 22:22:07,697 DEBUG    MQTT       CONACK => Return code: 0

2016-04-25 22:22:07,798 INFO     MQTT       on_subscribe

2016-04-25 22:22:09,484 WARNING  MainThread SerialDirect thread is dead

2016-04-25 22:22:09,694 WARNING  MainThread SerialDirect thread is dead

2016-04-25 22:22:09,903 WARNING  MainThread SerialDirect thread is dead

2016-04-25 22:22:10,112 WARNING  MainThread SerialDirect thread is dead

Can you provide more info on the sketch and/or serial data ? 

The "SerialDirect thread is dead" is a simple check written into the main core of emonhub to avoid a situation where an interfacer thread crashes out without any notice and it's then unknown if the interfacer is "dead" or just quiet.

Basically the thread is crashing and there is no clues as to why since the error handling and logging is "a little thin" in this version.

As in the example above it could well be the incoming data just isn't perfect and it's not being handled the same way it would be in the original version. 

Paul

Here is the serial data:

OPTIONS: I18n 

Compiled on Apr 28 2012, 19:24:31.

Port /dev/ttyUSB0


Press CTRL-A Z for help on special keys


11 0

11 0

11 0

The sketch is my modification of Felix Rusu's Moteino sketch:

// BB: Modified this sketch 4/13/2016
// Use: Plug a MoteinoUSB running this sketch into the USB port of the emonHub Raspberry Pi
//     It will listen to the Moteino radios and repeat into the serial on the Pi through USB.
//     In emonhub.conf there will be a serial interfacer listening to the serial port.
// Original comments from Felix:
// Sample RFM69 receiver/gateway sketch, with ACK and optional encryption, and Automatic Transmission Control
// Passes through any wireless received messages to the serial port & responds to ACKs
// It also looks for an onboard FLASH chip, if present
// RFM69 library and sample code by Felix Rusu - http://LowPowerLab.com/contact
// Copyright Felix Rusu (2015)

#include <RFM69.h>    //get it here: https://www.github.com/lowpowerlab/rfm69
#include <RFM69_ATC.h>//get it here: https://www.github.com/lowpowerlab/rfm69
#include <SPI.h>      //comes with Arduino IDE (www.arduino.cc)
#include <SPIFlash.h> //get it here: https://www.github.com/lowpowerlab/spiflash

//*********************************************************************************************
//************ IMPORTANT SETTINGS - YOU MUST CHANGE/CONFIGURE TO FIT YOUR HARDWARE *************
//*********************************************************************************************
#define NODEID        1    //unique for each node on same network
#define NETWORKID     100  //the same on all nodes that talk to each other
//Match frequency to the hardware version of the radio on your Moteino (uncomment one):
#define FREQUENCY     RF69_433MHZ
//#define FREQUENCY     RF69_868MHZ
//#define FREQUENCY     RF69_915MHZ
//#define ENCRYPTKEY    "sampleEncryptKey" //exactly the same 16 characters/bytes on all nodes!
//#define IS_RFM69HW    //uncomment only for RFM69HW! Leave out if you have RFM69W!
#define ENABLE_ATC    //comment out this line to disable AUTO TRANSMISSION CONTROL
//*********************************************************************************************

#define SERIAL_BAUD   115200

#ifdef __AVR_ATmega1284P__
  #define LED           15 // Moteino MEGAs have LEDs on D15
  #define FLASH_SS      23 // and FLASH SS on D23
#else
  #define LED           9 // Moteinos have LEDs on D9
  #define FLASH_SS      8 // and FLASH SS on D8
#endif

#ifdef ENABLE_ATC
  RFM69_ATC radio;
#else
  RFM69 radio;
#endif

SPIFlash flash(FLASH_SS, 0xEF30); //EF30 for 4mbit  Windbond chip (W25X40CL)
bool promiscuousMode = false; //set to 'true' to sniff all packets on the same network

int TRANSMITPERIOD = 5000;

typedef struct {
  int           nodeId; //store this nodeId
  unsigned long uptime; //uptime in ms
  float         temp;   //temperature maybe?
} Payload;
Payload theData;

void setup() {
  Serial.begin(SERIAL_BAUD);
  delay(10);
  radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW
  radio.setHighPower(); //only for RFM69HW!
#endif
//  radio.encrypt(ENCRYPTKEY);
  radio.promiscuous(promiscuousMode);
  //radio.setFrequency(919000000); //set frequency to some custom frequency
  char buff[50];
  sprintf(buff, "\nListening at %d Mhz...", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915);
  Serial.println(buff);
  if (flash.initialize())
  {
    Serial.print("SPI Flash Init OK. Unique MAC = [");
    flash.readUniqueId();
    for (byte i=0;i<8;i++)
    {
      Serial.print(flash.UNIQUEID[i], HEX);
      if (i!=8) Serial.print(':');
    }
    Serial.println(']');
    
  }
  else
    Serial.println("SPI Flash MEM not found.");
    
#ifdef ENABLE_ATC
  Serial.println("RFM69_ATC Enabled (Auto Transmission Control)");
#endif
}

byte ackCount=0;
uint32_t packetCount = 0;
long lastPeriod = -1;

void loop() {
  //process any serial input
  if (Serial.available() > 0)
  {
    char input = Serial.read();
    if (input == 'r') //d=dump all register values
      radio.readAllRegs();
      
    if (input == 'p')
    {
      promiscuousMode = !promiscuousMode;
      radio.promiscuous(promiscuousMode);
      Serial.print("Promiscuous mode ");Serial.println(promiscuousMode ? "on" : "off");
    }
    
    if (input == 'd') //d=dump flash area
    {
      Serial.println("Flash content:");
      int counter = 0;

      while(counter<=256){
        Serial.print(flash.readByte(counter++), HEX);
        Serial.print('.');
      }
      while(flash.busy());
      Serial.println();
    }
    if (input == 'D')
    {
      Serial.print("Deleting Flash chip ... ");
      flash.chipErase();
      while(flash.busy());
      Serial.println("DONE");
    }
    if (input == 'i')
    {
      Serial.print("DeviceID: ");
      word jedecid = flash.readDeviceId();
      Serial.println(jedecid, HEX);
    }
    if (input == 't')
    {
      byte temperature =  radio.readTemperature(-1); // -1 = user cal factor, adjust for correct ambient
      byte fTemp = 1.8 * temperature + 32; // 9/5=1.8
      Serial.print( "Radio Temp is ");
      Serial.print(temperature);
      Serial.print("C, ");
      Serial.print(fTemp); //converting to F loses some resolution, obvious when C is on edge between 2 values (ie 26C=78F, 27C=80F)
      Serial.println('F');
    }
  }

  
  if (radio.receiveDone())
  {

      for (byte i = 0; i < radio.DATALEN; i++)
      Serial.print((char)radio.DATA[i]);
   
    
    if (radio.ACKRequested())
    {
      byte theNodeID = radio.SENDERID;
      radio.sendACK();

      if (ackCount++%3==0)
      {
        delay(3); //need this when sending right after reception .. ?
        radio.sendWithRetry(theNodeID, "ACK TEST", 8, 0);  // 0 = only 1 attempt, no retries
      }
    }
    Serial.println();
    Blink(LED,3);
  }
}

void Blink(byte PIN, int DELAY_MS)
{
  pinMode(PIN, OUTPUT);
  digitalWrite(PIN,HIGH);
  delay(DELAY_MS);
  digitalWrite(PIN,LOW);
}

My best guess is the text from the setup() isn't being caught and removed so when the interfacer tries to extract the node id it fails as there is no int present. The original emonhub's serial interfacer had basic input sanitation where as the raw data is passed without any checks in this variant.

Any attempts to edit the emonhub code could cause issues with the emonpi update script so your best course of action is a quick edit to the sketch to remove any printing of non-numeric data. Primarily in the setup() but since the loop() "if (Serial.available() > 0)" code block is not useful without proper integration within emonhub, perhaps a custom Monteino interfacer. I would probably cut that away too, replacing it with some hardcoded "options" if required.

if you want to implement something a little grander I will help with some pointers but if you plan to keep updating via the emonpi update script this is the easiest option unless the changes are made upstream.

Paul

Thanks, Paul! That was it. I hadn't bothered to clean up the mess I had made of that sketch because it was working on my old emonHub. I cleaned it up and removed all the extraneous serial printing. Now it works perfectly.

I did notice that emonhub is adding an RSSI value to that input, which I am not sending it. It sets it to zero and displays it in the inputs view. Is there some assumption in this version of emonhub that there is always an RSSI value?

Regarding implementing something a little grander, here is what I am envisioning: using my emonHub+Moteino as an internet-of-things base station. I.e uing the OpenHAB app to set a switch, which triggers an MQTT message, which Node-RED catches. Node-RED then sends a command over serial (USB) to the Moteino base station, which radios a remote Moteino to do something. I'd be interested in any pointers you might have. Like, is that even a good way to approach it? And, does Node-RED have an easy way to write serial data to the USB port of the Pi?

Thanks again!

Brandon

The other thing that appears to cause the "MainThread SerialDirect thread is dead" warning is simply connecting or disconnecting the device plugged into the USB port. And once that warning starts up, it repeats at least once every second. It would be nice, if I could suggest it, if the input sanitation from the older emonHub was brought forward to this one.

I found that simply restarting emonHub with "sudo service emonhub restart" was not enough to resolve "thread is dead" warning. I had to reboot the Pi on which it is running.

I would second the earlier motion to rename this thread: "WARNING MainThread SerialDirect thread is dead" and in my opinion we could add [SOLVED]. For now, the trick is to be very careful that you reboot your Pi running emonHub when you plug in a USB serial input, and you must make sure there is nothing printing to that serial port other than nice, clean input messages in the expected format.

Hi Brandon

emonhub should be robust enough not to fail due to incorrect serial data, ideally the sketch should identify itself so that down the line you know what sketch and version you are running. The removal of all the serial prints is not an acceptable fix IMO it is a quick and dirty workaround but if it gets you up and running that's good to know.

Yes the emon-pi variant assumes everything has an rssi, the original emonhub only passes the rssi if a valid rssi is attached to an incoming RFM packet, but this variant is very different in many ways.

If restarting emonhub doesn't resolve the "WARNING MainThread SerialDirect thread is dead" issue then there is something else in the mix causing an issue. 

Any serial port (gpio or usb) can only have 2 devices using those lines, for that reason you should not have node-red connecting to the same port/device as emonhub. This maybe the reason you need to reboot the Pi if there is anything else accessing the same serial port, restarting emonhub will not free the serial port fo emonhubs exclusive use.

Ideally you should be able to configure emonhub to listen for the openhab mqtt and act "directly" with the monteino. I am not saying you cannot use node-red, many do but, you cannot have multiple softwares connected to the one device and therefore openhab should connect directly with emonhub or of you go to node-red then node-red should go to emonhub which may be pointless.

I am not opposed to the thread name being changed at all, perhaps "WARNING MainThread SerialDirect thread is dead" would be more appropriate but since the issue is far from solved I would be opposed to the [solved] tag especially as there are many, many reasons that could cause that catch all error, it is not the sort of error that will ever be solved for everyone.

Paul

Hi all,

thanks for all the good tips.
I will give them a try now - didn't found the time to go on with it until now.

First of all I will change the setting for log level and check the arduino script for the period/commata issue. It might be have been me, who copied the period things to Notepad++ and change all periods for commas (excel import required in germany)

In the end, it seems I have to update my emonCMS / emonHub Setup on the central Pi to support the new method.
I really don't like to make a "from the scratch" setup, since everything for ssd and settings is working fine.

I'll get back with more info soon.

If you'd like to rename the thread - feel free. Serial direct / serial thread dead / emonHub update issue might be good chunks for a header.

Cheers
Christian