Archived Forum

Hi Paul, interesting question, how are these readings sampled and processed ?

Are you receiving a processed average for the last 20secs or a raw windspeed reading every 20secs.?

If the latter, I question whether emoncms will be able to give you accurate readings even if averaging was applied. For something that normally changes slowly like ambient temp 20 sec samples will do the job as we can be pretty sure of what's happening during the 20secs between readings, Wind speed measurements however, as you say can vary wildly between samples almost rendering then useless. For example if the sample rate were to coincide with gusts, imagine an exaggerated scenario where the wind speed was 85mph for 16 secs and then 10mph for 4 secs, if this cycle repeated consistently the average wind speed is 70mph but depending on where in the 20sec cycle the sample was taken a reading of of 10mph or 85mph could be expected, in fact there would be a 1 in 5 chance the result would be a 10mph average .

Would it be possible to significantly increase the sample rate and average the samples before reporting ? eg sample once per second for 20 seconds and divide the sum of the readings by 20.

If this is already being done or requires further smoothing, can the reporting period be extended to 1min or 5mins to sample a greater span of variation  or if you have you own processing stage/sketch could it be modified to report an average of the last 10 or 20 "20sec readings" every 20 seconds?

I'm not aware of a way to average in emoncms, but in this instance with something as unpredictable and as rapidly changeable as wind speed I think more samples would be needed to be of more use.

Paul

Paul, I'm using node-red to parse weather data from a nearby weather station via an API, and feeding that into emoncms, which I presume  has already had some averaging applied because it also supplies wind gust data, which is usually higher than the 'wind' data.

You have raised a few interesting points, which I need to clarify with the data provider, especially re exactly what processing has been done, and then maybe increase the sample rate and build an avenging feature into my node-red flow, before it hits emoncms.

Paul

Just found the Met Offices windspeed model;

'....Because wind is an element that varies rapidly over very short periods of time it is sampled at high
frequency (every 0.25 sec) to capture the intensity of gusts, or short-lived peaks in speed, which inflict
greatest damage in storms. The gust speed and direction are defined by the maximum three second
average wind speed occurring in any period.
A better measure of the overall wind intensity is defined by the average speed and direction over the
ten minute period leading up to the reporting time. Mean wind over other averaging periods may also
be calculated....'

Paul

Sounds reasonable, how does that compare with your providers data?

Sounds reasonable, how does that compare with your providers data?

No comparison whatsoever!
It's clear when examining the data stream that there is no averaging at all at source, and rather that try to do the sums remotely, it's easier to find another data stream..

Fortunately, I live 2 miles from an International Airport, who submit accredited METAR weather data direct to the NOAA Weather Service, who just happen to have a api!!

So by calling the api via node-red, I have successfully parsed the resultant XML, and then feed the data into emoncms for further processing, logging and visualizations.

Paul

Which data provider where you using previously?

Sometime back you were using wunderground which I took a look at yesterday afternoon, As an experiment  I created a custom interfacer for emonhub which makes a call every 3mins (to stay within the Free API call limits) I left it running over night and the results seem quite stable yes they vary as you would expect but not to the degree you experienced.

Were you using another provider up till yesterday ? 

How is the latest provider panning out ? I see only one wind speed is given is that an average or a snapshot?

I'm not finding much detail on the data values at all.

Paul 

I was accessing data from an 'experimental' weather station situated on the nearby airport's apron (via Wunderground's api). Initially, the data was good & reliable, but then the precipitation stopped recording and wind gusts became sporadic, so I began looking for a replacement, having a choice of several personal weather stations - all of which feed into Wunderground.

The new provider is the official airport weather station, which provides data for safe air travel, hence it is accurate! and yes, the windspeed is averaged over a 30 minute period (which is the METAR update frequency).

I too noticed that the 'wind gusts' was missing, but then found that they do not include irrelevant data in METAR's, so instead of repeating 'Snow-depth = 0' throughout summer, they only add it when they get snow! the same applies with wind-gusts, and I notice this morning that the xml is showing gusts - see the history towards the bottom of this page.

The disappearing 'wind_gust_kt' label did cause me some problems with the XML parsing, but eventually got around it by;

var gusts = 0;
if (typeof METARdata.wind_gust_kt != "undefined") {
gusts = METARdata.wind_gust_kt[0];
}

So if the label is missing, it returns gusts = 0.

Paul

so I began looking for a replacement, having a choice of several personal weather stations - all of which feed into Wunderground.

I take it you are now accessing the new provider directly and not via wunderground, correct ?

For snowfall inserting zero is ideal but would the gusts value be more accurate if it fell back to the wind speed value instead? if nothing else it might make graphs clearer.

It's not as straight forward as I would of expected to find the right data source.

Paul 

I now get them direct from NOAA - the American National Weather Centre who collate airport weather data globally.

That's a good idea about falling wind-gusts back to wind-speed, I'll add that tonight!

I think that you have to compromise when selecting a weather station, I would ideally have wanted rainfall to be included, but strangely, the METAR does not include it.

Paul

"I would ideally have wanted rainfall to be included, but strangely, the METAR does not include it." That's because it's not relevant. What is relevant is the rain intensity and the runway state (for braking). The current TAF for your airport says:
EGCN 291101Z 2912/2921 21018KT 9999 SCT030 TEMPO 2912/2921 21022G32KT PROB40 TEMPO 2915/2921 7000 SHRA - the last bit meaning a 40% probability between 4pm and 10pm of showers of rain ("SHRA") - but not heavy (+) nor light (-), and between 1pm and 10 pm the wind is forecast to gust to 32 knots. The ATIS (spoken, broadcast on an aeronautical frequency) may include something like "wet damp damp", meaning the runway is wet in the touchdown zone, damp in the middle and damp at the stop end.

Thanks Robert. 

This weather thing becomes an obsession, having spent the last 2 hours trying to write a August-Roche-Magnus approximation into javascript.

My lack of a basic coding knowledge always lets me down!!

Paul

If you look at the source of this page - you'll find it in the .js file listed in the header.

Good thinking! It just needed a little tweak, but apart from that it dropped straight into node-red, and now I've successfully got humidity logged in emoncms too.

Paul

Hi,

I managed to plot moving averages in multigraph. It may interest you.

Have a look here

Eric