Create a weather station
-
MY WEATHER STATION
AIM
The aim of this project is to create a weather-station from scratch.
Several sensors are integrated. The data of some will be compared to see the differences between the different types of sensor measuring the same quantity.
Sensor integration, data transmission, data processing and data presentation are addressed.
Only the eGeoffrey part of the system is described in detail.The mechanical part is not described.
Sensors
The following sensors are applied:
BME280 : pressure, humidity and temperature
TSL2561 : light level
ML8511 : UV level
ML6070 : UV level
DS18B20 : temperature
HMC5883: compass for the wind direction
Anemometer: wind speed sensor
Windsensor Rev. P : a hot wire anemometer with temperature sensor
Tipping bucket rain volume sensorThey are combined in one single mechanical construction.
They are hooked-up to one LOLIN D1 mini. This module contains a microcontroller able to transmit via incorporated WiFi capabilities. Several GPIO and communication protocols are available.
The following sensors use the I2C protocol :
BME280, TSL2561, HMC5883, ML6070The following sensor uses the "one wire" protocol:
DS18B20The following sensors provide an analog output:
ML8511The Windsensor Rev. P has two analog outputs, one for the temperature and one for the windspeed.
The anemometer has an analogue output.
The tipping bucket works with two simple on/off signals.
One sensor has a 5V output. The rest have 3.3V outputs. One requires 15 V, one 12V and the rest need 3.3V power supply.
Data-collection
Per measured quantity it is adjustable which interval between measurements is applied.
It might happen that values change suddenly. To be able to catch those rapidly changing values also, per quantity an adjustable difference between sensed values is applied to collect and transmit data. This results in a regular update of data per sensor. On a sudden change in value a sensor transmits extra data.Data-transmission
For data-transmission the MQTT protocol is used. Using the "ESP8266MQTTMesh" library the MQTT protocol is used in an automatically self-establishing radio network. All nodes together create a network using up-to four radio-links each simultaneously. As long as a node has at least one radio-link with another node, it has full transmission capability. The node with the best signal of the WiFi network establishes one single WiFi link used by all nodes. This WiFi link is used by all nodes for their individual connection with the MQTT broker. The network uses normal WiFi protection mechanisms.
Data processing and presentation
eGeoffrey is used for data processing, data presentation and sensor and module control.
DESIGN
People have a place. At that place are buildings. At that place is a garden. For the software a building and the garden both are spaces. A space has rooms. A room has modules. A module has sensors.
Messages are going out of the sensor module or are coming into the sensor module.
The data presentation is grouped into functional areas e.g.:
-
weather
-
control
-
system
Naming convention in- and out going messages
The sensor module uses a root "in-topic" and a root "out-topic". eGeoffrey uses a topic sequence to send and a separate topic sequence to receive MQTT messages.
The sensor module is leading. It's "topic-out" is "out". It's "topic-in" is "in".
For eGeoffrey it is automatically the other way around. eGeoffrey receives messages via "out" and sends messages via "in".Data display
To gain a better overview of the data presented several pages are used.
One page shows the actual weather data out of the internet for the location. Another page shows the actual values of the sensors of the weather-station. Per quantity a separate page is created. Data and widgets displaying data of the same quantity are visible on one page. There is e.g. a page bringing together actual values and an historic overview of temperature data.
To control sensors and modules a control page for each purpose is available.
To have an overview of several modules being on-line a "heartbeat" page is made.
An extra page shows data of eGeoffrey (the raspi where eGeoffrey lives).Sensor module
The weather station has several sensors. Each sensor transmits data with an operational interval. For testing purposes each sensor can transmit data with a test interval. When the quantity measured changes rapidly more than a "delta" value, extra data is transmitted to catch this kind of rapid strong variations.
Furthermore the measured value of each sensor can be modified with a positive or negative float value. The aim is to move the curve representing the historic data of the sensor up and down on the screen.With eGeoffrey the following aspects of each sensor can be modified:
-
on/off
-
ops/test
-
ops interval
-
test interval
-
adjustment value
-
delta
When an aspect of a sensor is changed, the feedback of each aspect is displayed together with the name of the sensor. Between changes the content of these widgets is blanked.
Each sensor can be interrogated to transmit the value of it's aspects.
The weather station can be switched:
-
on/off
-
from ops to test and vice versa
-
to default settings
It can be rebooted.
During testing of the hardware it is necessary to trace why a widget is not behaving as expected. There can be several causes. Therefore individual parts of a data transfer path need to be checked.
It is possible to:
-
query each sensor individually with a dummy value in return
-
query all sensors simultaneously with a dummy value for each sensor in return
-
blank all sensor data presentation simultaneously
-
query each sensor individually with a real sensor value in return
Controlling sensors
To send control parameters from within eGeoffrey a separate sensor control page is made.
On this page:-
one widget is used to select the sensor.
-
six widgets are used to switch the aspect or set the value.
-
a last widget is used to send an “acknowledgment” to the module.
The module receives the MQTT messages transmitted by eGeoffrey. As soon as the “acknowledgement” is received the data are processed and stored into EEPROM to be used from now on.
Because eGeoffrey has to address sensors of an individual module, the module ID needs to be part of the topic eGeoffrey uses to send data to that module. Receiving data the wildcard "+" is used instead of the module ID.
To be sure that the data are received correctly the sensor module sends the data back to eGeoffrey.
Six widgets are used to display the data received from the module.On this page the widgets are available to query the sensors.
Controlling the weather station
On a separate page it is possible to switch all sensors together on/off. They can also be switched from ops to test mode together. After finishing a test series it is handy to reset the module to the default values with only two actions. To have the module reboot a separate topic is used.
A dedicated widget provides feedback from the weather station.
MENU STRUCTURE
toplevel:
-
weather
-
control
-
system
below weather:
-
the town where we live
-
local
-
temperature
-
wind
-
air
-
light
-
rain
below control:
-
weather-station
-
module
-
heartbeat
below system:
- eGeoffrey.
TOPIC STRUCTURE
The "ESP8266MQTTMESH" library does not allow for topics having more than 65 characters. Therefore some topic elements have been condensed:
module-in -> in
module-out -> out
weatherstation -> wthr
sensor -> sr
temperature -> tmp
windspeed -> wspd
winddirection -> wdrtn
pressure -> prs
humidity -> hm
light -> lght
rain -> rn
anemometer -> anemo
opsinterval -> opsntrvl
testinterval -> tstntrvlTopic structure weather station data
area-ID/space-ID/module-ID → module topics
area-ID/space-ID/module-ID/sensor → sensor general topics
area-ID/space-ID/module-ID/sensor/sensor-quantity/sensor-Id → sensor specific topicsTopic structure weather station sensor data
myplace/garden/wthr/sr/tmp/ds18
myplace/garden/wthr/sr/tmp/bme
myplace/garden/wthr/sr/tmp/xp
myplace/garden/wthr/sr/wspd/anemo
myplace/garden/wthr/sr/wspd/xp
myplace/garden/wthr/sr/wdrtn/compass
myplace/garden/wthr/sr/wdrtn/txt
myplace/garden/wthr/sr/prs/bme
myplace/garden/wthr/sr/hm/bme
myplace/garden/wthr/sr/lght/tsl2561
myplace/garden/wthr/sr/uv/ml8511
myplace/garden/wthr/sr/uv/ml6070
myplace/garden/wthr/sr/rn/volumeTopic structure control data
control sensor:
myplace/garden/wthr
myplace/garden/wthr/sr
myplace/garden/wthr/sr/switch
myplace/garden/wthr/sr/opstest
myplace/garden/wthr/sr/opsntrvl
myplace/garden/wthr/sr/tstntrvl
myplace/garden/wthr/sr/adjust
myplace/garden/wthr/sr/ackcontrol module:
myplace/garden/wthr
myplace/garden/wthr/switch
myplace/garden/wthr/opstest
myplace/garden/wthr/default
myplace/garden/wthr/reboot
myplace/garden/wthr/ack
myplace/garden/wthr/inforetrieve settings:
myplace/garden/wthr/sr
myplace/garden/wthr/sr/retrieve/select
myplace/garden/wthr/sr/retrieve/ack
myplace/garden/wthr/sr/retrieve/infoTopic structure system data
myplace/garden/wthr/heartbeat
DATA EXCHANGE
Sending data
Sensor module publishes sensor data with topic:
out/myplace/garden/wthr/sr/tmp/ds18
out/myplace/garden/wthr/sr/tmp/bme
out/myplace/garden/wthr/sr/tmp/xp
data type: floatout/myplace/garden/wthr/sr/wspd/anemo
out/myplace/garden/wthr/sr/wspd/xp
data type: integer
out/myplace/garden/wthr/sr/wdrtn/compass
data type: integer
out/myplace/garden/wthr/sr/wdrtn/txt
data type: stringout/myplace/garden/wthr/sr/prs/bme
data type: float
out/myplace/garden/wthr/sr/hm/bme
data type: integerout/myplace/garden/wthr/sr/lght/tsl2561
out/myplace/garden/wthr/sr/uv/ml8511
out/myplace/garden/wthr/sr/uv/ml6070
data type: floatSensor module publishes control feedback data with topic:
out/myplace/garden/wthr/sr
data type: stringout/myplace/garden/wthr/sr/switch
out/myplace/garden/wthr/sr/opstest
out/myplace/garden/wthr/sr/opsntrvl
out/myplace/garden/wthr/sr/tstntrvl
data type: integerout/myplace/garden/wthr/sr/adjust
out/myplace/garden/wthr/sr/delta
data type: floatout/myplace/garden/wthr/sr/retrieve/info
data type: string
out/myplace/garden/wthr
data type: stringeGeoffrey
In eGeoffrey data are published by "sensors". For publishing data the "processing" aspects are not important.
eGeoffrey publishes sensor control data with topic:
in/module-ID/myplace/garden/wthr/sr
data: integer - -1.. number of sensors (12, "-1" used to blank the widget)in/module-ID/myplace/garden/wthr/sr/switch
data: integer 1 / 0in/module-ID/myplace/garden/wthr/sr/opstest
data: integer 1 / 0in/module-ID/myplace/garden/wthr/sr/opsntrvl
data: integer - 1 .. 60 (minutes)in/module-ID/myplace/garden/wthr/sr/tstntrvl
data: integer - 1 .. 15 (seconds)in/module-ID/myplace/garden/wthr/sr/adjust
data: float - -5 .. 5in/module-ID/myplace/garden/wthr/sr/delta
data: float - 0.1 .. 150in/module-ID/myplace/garden/wthr/sr/ack
data: integer 1 / 0in/module-ID/myplace/garden/wthr/sr/retrieve/select
data: integer - 0 .. 4in/module-ID/myplace/garden/wthr/sr/retrieve/ack
data: integer 1 / 0eGeoffrey publishes module control data with topic:
in/module-ID/myplace/garden/wthr
data: integer - -1 .. 1 number of modules (1, "-1" used to blank the widget)in/module-ID/myplace/garden/wthr/switch
data: integer 1 / 0in/module-ID/myplace/garden/wthr/opstest
data: integer 1 / 0in/module-ID/myplace/garden/wthr/default
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/reboot
data: integer - -1 .. number of modules (1, "-1" used to blank the widget)in/module-ID/myplace/garden/wthr/ack
data: integer 1 / 0Receiving data
In eGeoffrey data are received by "sensors". For receiving data also the "processing" aspects are of importance.
eGeoffrey receives sensor data with topic:
Processing:
Automatic aggregation – avg – min – max
Retention Policies – historyBeware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr/sr/tmp/ds18
data: floatout/+/myplace/garden/wthr/sr/tmp/bme
data: floatout/+/myplace/garden/wthr/sr/tmp/xp
data: floatout/+/myplace/garden/wthr/sr/wspd/anemo
data: integerout/+/myplace/garden/wthr/sr/wspd/xp
data: integerout/+/myplace/garden/wthr/sr/wdrtn/compass
data: integerout/+/myplace/garden/wthr/sr/wdrtn/txt
data: string
(Processing:
Automatic Aggregation - None
Retention Policies - Single Value)out/+/myplace/garden/wthr/sr/prs/bme
data: floatout/+/myplace/garden/wthr/sr/hm/bme
data: integerout/+/myplace/garden/wthr/sr/lght/tsl2561
data: floatout/+/myplace/garden/wthr/sr/uv/ml8511
data: floatout/+/myplace/garden/wthr/sr/uv/ml6070
data: floatout/+/myplace/garden/wthr/sr/rn/volume
data: integer
(Processing:
Automatic aggregation – sum
Retention Policies – history)eGeoffrey receives sensor control data with topic:
Processing:
Automatic aggregation - none
Retention Policies – single valueBeware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr/sr
data: stringout/+/myplace/garden/wthr/sr/switch
data: stringout/+/myplace/garden/wthr/sr/opstest
data: stringout/+/myplace/garden/wthr/sr/opsntrvl
data: integerout/+/myplace/garden/wthr/sr/tstntrvl
data: integerout/+/myplace/garden/wthr/sr/adjust
data: floatout/+/myplace/garden/wthr/sr/delta
data: floatout/+/myplace/garden/wthr/sr/retrieve/info
data: stringeGeoffrey receives module control data with topic:
Processing:
Automatic aggregation - none
Retention Policies – single valueBeware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr
data: stringDATA INPUT and PRESENTATION
Sensor data widgets:
Widget Type: value
For overviews more Widget Type's are available.out/myplace/garden/wthr/sr/tmp/ds18
out/myplace/garden/wthr/sr/tmp/bme
out/myplace/garden/wthr/sr/tmp/xp
out/myplace/garden/wthr/sr/wspd/anemo
out/myplace/garden/wthr/sr/wspd/xp
out/myplace/garden/wthr/sr/wdrtn/compass
out/myplace/garden/wthr/sr/wdrtn/txt
out/myplace/garden/wthr/sr/prs/bme
out/myplace/garden/wthr/sr/hm/bme
out/myplace/garden/wthr/sr/lght/tsl2561
out/myplace/garden/wthr/sr/uv/ml8511
out/myplace/garden/wthr/sr/uv/ml6070
out/myplace/garden/wthr/sr/rn/volumeSensor control widgets:
in/myplace/garden/wthr/sr
type: input boxin/myplace/garden/wthr/sr/switch
type: On/Off switchin/myplace/garden/wthr/sr/opstest
type: On/Off switchin/myplace/garden/wthr/sr/opsntrvl
type: input boxin/myplace/garden/wthr/sr/tstntrvl
type: input boxin/myplace/garden/wthr/sr/adjust
type: input boxin/myplace/garden/wthr/sr/delta
type: input boxin/myplace/garden/wthr/sr/ack
type: button
action: SET in/1D8445/myplace/garden/wthr/sr/ack 1Sensor Feedback widgets:
Widget Type: value
out/myplace/garden/wthr/sr
out/myplace/garden/wthr/sr/switch
out/myplace/garden/wthr/sr/opstest
out/myplace/garden/wthr/sr/opsntrvl
out/myplace/garden/wthr/sr/tstntrvl
out/myplace/garden/wthr/sr/adjust
out/myplace/garden/wthr/sr/delta
out/myplace/garden/wthr/sr/retrieve/infoModule control widgets:
in/myplace/garden/wthr/
type: input boxin/myplace/garden/wthr/switch
type: On/Off switchin/myplace/garden/wthr/opstest
type: On/Off switchin/myplace/garden/wthr/default
type: button
action: SET in/1D8445/myplace/garden/wthr/default 1in/myplace/garden/wthr/reboot
type: button
action: SET in/1D8445/myplace/garden/wthr/reboot 1in/myplace/garden/wthr/ack
type: button
action: SET in/1D8445/myplace/garden/wthr/ack 1Module feedback widgets:
Widget Type: value
out/myplace/garden/wthr
out/myplace/garden/wthr/sr/retrieve/infoTESTING
Test configuration
Laptop1
Webbrowser with aGeoffrey website to:
-
send control data
-
observe data going in and out of eGeoffrey
-
receive sensor data
Laptop 2
-
"Coolterm" to observe data going in and out of the module.
-
"MQTT observer" to observe data going in and out of Mosquitto.
-
Spreadsheet to register data input and results.
Each controlled sensor is tested. Each switch is put in both positions. Each quantity is tested with the minimum, middle and maximum value.
Quantity position
switch on/off on off
switch ops/test on offvalues
ops interval: 1 - 29 - 60
test interval: 1 - 8 - 5
adjustment: -4.99 - 0 - 4.99
delta: 0.1 - 9 - 149.99Every time before a test series the info widgets are cleaned.
Unexpected results are noted down.
After conclusion of testing the results are analysed and corrective actions are performed.
The tests are rehearsed until all control data are transmitted correctly, received correctly, evoked the correct action, the feedback is sent correctly and the feedback is displayed correctly.
Than the system is ready for operation.
Rules
Rules are a means to evaluate the data.
This rule supervises the heartbeat signal of a module.The simplest way is to have a recurrent rule running every minute with:
-
two variables
value = sensor_id
value_elapsed = ELAPSED sensor_id -
two constants
value_off = 0
elapsed_threshold = 1 -
And two conditions in the same block
value == value_off
value_elapsed > elapsed_threshold
After the weather station has been on-line for several weeks some adjustments to the software are done. Some of the sensors produce spikes in the data. Some sensors are delivering nothing but spikes.
To do away with the spikes, the ability to produce extra data on strong fluctuations can be switched off. When the sensor is in overload, transmission of data is suppressed.
This results in a far more intelligible presentation of the data.
All this preparation and labour resulted in a working weather station.
To give you an impression of how my weather station looks like here are a few screenshots:
Local weather data
All temperature sensors
Changing settings per sensor
Feedback from the weather station
Adjusting all sensors together
Enjoy!
-
-
Lovely How To, thanks for sharing it! Very comprehensive and detailed. Question, are you also aiming to publish the weather station data to some online weather service? I had in the past something in eGeoffrey for publishing to Weatherunderground before being acquired by IBM, just trying to understand if there is the need of something similar for other services (e.g. Windy, etc.). Thanks!
-
I do have yet another wish where you could do me a tremendous fever.
In the Netherlands we do have a nice weather system. I would like to connect to that system. There is an API in python available. Would you be so kind and have a look at it wether it could be an input for eGeoffrey also?[https://pypi.org/project/buienradar/](link url)
-
@eporocrail said in Create a weather station:
Sure let me have a look and I'll let you know.
I've also linked this fantastic guide from https://docs.egeoffrey.com/howto/