1. Home
  2. HOW TOs
  3. Create a weather station

Create a weather station

  • E

    MY WEATHER STATION

    AIM

    The aim of this project is to create a weather-station from scratch.

    Several sensors are integrated. The data of some will be compared to see the differences between the different types of sensor measuring the same quantity.

    Sensor integration, data transmission, data processing and data presentation are addressed.
    Only the eGeoffrey part of the system is described in detail.

    The mechanical part is not described.

    Sensors

    The following sensors are applied:

    BME280 : pressure, humidity and temperature
    TSL2561 : light level
    ML8511 : UV level
    ML6070 : UV level
    DS18B20 : temperature
    HMC5883: compass for the wind direction
    Anemometer: wind speed sensor
    Windsensor Rev. P : a hot wire anemometer with temperature sensor
    Tipping bucket rain volume sensor

    They are combined in one single mechanical construction.

    They are hooked-up to one LOLIN D1 mini. This module contains a microcontroller able to transmit via incorporated WiFi capabilities. Several GPIO and communication protocols are available.

    The following sensors use the I2C protocol :
    BME280, TSL2561, HMC5883, ML6070

    The following sensor uses the "one wire" protocol:
    DS18B20

    The following sensors provide an analog output:
    ML8511

    The Windsensor Rev. P has two analog outputs, one for the temperature and one for the windspeed.

    The anemometer has an analogue output.

    The tipping bucket works with two simple on/off signals.

    One sensor has a 5V output. The rest have 3.3V outputs. One requires 15 V, one 12V and the rest need 3.3V power supply.

    Data-collection

    Per measured quantity it is adjustable which interval between measurements is applied.
    It might happen that values change suddenly. To be able to catch those rapidly changing values also, per quantity an adjustable difference between sensed values is applied to collect and transmit data. This results in a regular update of data per sensor. On a sudden change in value a sensor transmits extra data.

    Data-transmission

    For data-transmission the MQTT protocol is used. Using the "ESP8266MQTTMesh" library the MQTT protocol is used in an automatically self-establishing radio network. All nodes together create a network using up-to four radio-links each simultaneously. As long as a node has at least one radio-link with another node, it has full transmission capability. The node with the best signal of the WiFi network establishes one single WiFi link used by all nodes. This WiFi link is used by all nodes for their individual connection with the MQTT broker. The network uses normal WiFi protection mechanisms.

    Data processing and presentation

    eGeoffrey is used for data processing, data presentation and sensor and module control.

    DESIGN

    People have a place. At that place are buildings. At that place is a garden. For the software a building and the garden both are spaces. A space has rooms. A room has modules. A module has sensors.

    Messages are going out of the sensor module or are coming into the sensor module.

    The data presentation is grouped into functional areas e.g.:

    • weather

    • control

    • system

    Naming convention in- and out going messages

    The sensor module uses a root "in-topic" and a root "out-topic". eGeoffrey uses a topic sequence to send and a separate topic sequence to receive MQTT messages.
    The sensor module is leading. It's "topic-out" is "out". It's "topic-in" is "in".
    For eGeoffrey it is automatically the other way around. eGeoffrey receives messages via "out" and sends messages via "in".

    Data display

    To gain a better overview of the data presented several pages are used.

    One page shows the actual weather data out of the internet for the location. Another page shows the actual values of the sensors of the weather-station. Per quantity a separate page is created. Data and widgets displaying data of the same quantity are visible on one page. There is e.g. a page bringing together actual values and an historic overview of temperature data.

    To control sensors and modules a control page for each purpose is available.
    To have an overview of several modules being on-line a "heartbeat" page is made.
    An extra page shows data of eGeoffrey (the raspi where eGeoffrey lives).

    Sensor module

    The weather station has several sensors. Each sensor transmits data with an operational interval. For testing purposes each sensor can transmit data with a test interval. When the quantity measured changes rapidly more than a "delta" value, extra data is transmitted to catch this kind of rapid strong variations.
    Furthermore the measured value of each sensor can be modified with a positive or negative float value. The aim is to move the curve representing the historic data of the sensor up and down on the screen.

    With eGeoffrey the following aspects of each sensor can be modified:

    • on/off

    • ops/test

    • ops interval

    • test interval

    • adjustment value

    • delta

    When an aspect of a sensor is changed, the feedback of each aspect is displayed together with the name of the sensor. Between changes the content of these widgets is blanked.

    Each sensor can be interrogated to transmit the value of it's aspects.

    The weather station can be switched:

    • on/off

    • from ops to test and vice versa

    • to default settings

    It can be rebooted.

    During testing of the hardware it is necessary to trace why a widget is not behaving as expected. There can be several causes. Therefore individual parts of a data transfer path need to be checked.

    It is possible to:

    • query each sensor individually with a dummy value in return

    • query all sensors simultaneously with a dummy value for each sensor in return

    • blank all sensor data presentation simultaneously

    • query each sensor individually with a real sensor value in return

    Controlling sensors

    To send control parameters from within eGeoffrey a separate sensor control page is made.
    On this page:

    • one widget is used to select the sensor.

    • six widgets are used to switch the aspect or set the value.

    • a last widget is used to send an “acknowledgment” to the module.

    The module receives the MQTT messages transmitted by eGeoffrey. As soon as the “acknowledgement” is received the data are processed and stored into EEPROM to be used from now on.

    Because eGeoffrey has to address sensors of an individual module, the module ID needs to be part of the topic eGeoffrey uses to send data to that module. Receiving data the wildcard "+" is used instead of the module ID.

    To be sure that the data are received correctly the sensor module sends the data back to eGeoffrey.
    Six widgets are used to display the data received from the module.

    On this page the widgets are available to query the sensors.

    Controlling the weather station

    On a separate page it is possible to switch all sensors together on/off. They can also be switched from ops to test mode together. After finishing a test series it is handy to reset the module to the default values with only two actions. To have the module reboot a separate topic is used.

    A dedicated widget provides feedback from the weather station.

    MENU STRUCTURE

    toplevel:

    • weather

    • control

    • system

    below weather:

    • the town where we live

    • local

    • temperature

    • wind

    • air

    • light

    • rain

    below control:

    • weather-station

    • module

    • heartbeat

    below system:

    • eGeoffrey.

    TOPIC STRUCTURE

    The "ESP8266MQTTMESH" library does not allow for topics having more than 65 characters. Therefore some topic elements have been condensed:

    module-in -> in
    module-out -> out
    weatherstation -> wthr
    sensor -> sr
    temperature -> tmp
    windspeed -> wspd
    winddirection -> wdrtn
    pressure -> prs
    humidity -> hm
    light -> lght
    rain -> rn
    anemometer -> anemo
    opsinterval -> opsntrvl
    testinterval -> tstntrvl

    Topic structure weather station data

    area-ID/space-ID/module-ID → module topics
    area-ID/space-ID/module-ID/sensor → sensor general topics
    area-ID/space-ID/module-ID/sensor/sensor-quantity/sensor-Id → sensor specific topics

    Topic structure weather station sensor data

    myplace/garden/wthr/sr/tmp/ds18
    myplace/garden/wthr/sr/tmp/bme
    myplace/garden/wthr/sr/tmp/xp
    myplace/garden/wthr/sr/wspd/anemo
    myplace/garden/wthr/sr/wspd/xp
    myplace/garden/wthr/sr/wdrtn/compass
    myplace/garden/wthr/sr/wdrtn/txt
    myplace/garden/wthr/sr/prs/bme
    myplace/garden/wthr/sr/hm/bme
    myplace/garden/wthr/sr/lght/tsl2561
    myplace/garden/wthr/sr/uv/ml8511
    myplace/garden/wthr/sr/uv/ml6070
    myplace/garden/wthr/sr/rn/volume

    Topic structure control data

    control sensor:

    myplace/garden/wthr
    myplace/garden/wthr/sr
    myplace/garden/wthr/sr/switch
    myplace/garden/wthr/sr/opstest
    myplace/garden/wthr/sr/opsntrvl
    myplace/garden/wthr/sr/tstntrvl
    myplace/garden/wthr/sr/adjust
    myplace/garden/wthr/sr/ack

    control module:

    myplace/garden/wthr
    myplace/garden/wthr/switch
    myplace/garden/wthr/opstest
    myplace/garden/wthr/default
    myplace/garden/wthr/reboot
    myplace/garden/wthr/ack
    myplace/garden/wthr/info

    retrieve settings:

    myplace/garden/wthr/sr
    myplace/garden/wthr/sr/retrieve/select
    myplace/garden/wthr/sr/retrieve/ack
    myplace/garden/wthr/sr/retrieve/info

    Topic structure system data

    myplace/garden/wthr/heartbeat

    DATA EXCHANGE

    Sending data

    Sensor module publishes sensor data with topic:

    out/myplace/garden/wthr/sr/tmp/ds18
    out/myplace/garden/wthr/sr/tmp/bme
    out/myplace/garden/wthr/sr/tmp/xp
    data type: float

    out/myplace/garden/wthr/sr/wspd/anemo
    out/myplace/garden/wthr/sr/wspd/xp
    data type: integer
    out/myplace/garden/wthr/sr/wdrtn/compass
    data type: integer
    out/myplace/garden/wthr/sr/wdrtn/txt
    data type: string

    out/myplace/garden/wthr/sr/prs/bme
    data type: float
    out/myplace/garden/wthr/sr/hm/bme
    data type: integer

    out/myplace/garden/wthr/sr/lght/tsl2561
    out/myplace/garden/wthr/sr/uv/ml8511
    out/myplace/garden/wthr/sr/uv/ml6070
    data type: float

    Sensor module publishes control feedback data with topic:

    out/myplace/garden/wthr/sr
    data type: string

    out/myplace/garden/wthr/sr/switch
    out/myplace/garden/wthr/sr/opstest
    out/myplace/garden/wthr/sr/opsntrvl
    out/myplace/garden/wthr/sr/tstntrvl
    data type: integer

    out/myplace/garden/wthr/sr/adjust
    out/myplace/garden/wthr/sr/delta
    data type: float

    out/myplace/garden/wthr/sr/retrieve/info
    data type: string
    out/myplace/garden/wthr
    data type: string

    eGeoffrey

    In eGeoffrey data are published by "sensors". For publishing data the "processing" aspects are not important.

    eGeoffrey publishes sensor control data with topic:

    in/module-ID/myplace/garden/wthr/sr
    data: integer - -1.. number of sensors (12, "-1" used to blank the widget)

    in/module-ID/myplace/garden/wthr/sr/switch
    data: integer 1 / 0

    in/module-ID/myplace/garden/wthr/sr/opstest
    data: integer 1 / 0

    in/module-ID/myplace/garden/wthr/sr/opsntrvl
    data: integer - 1 .. 60 (minutes)

    in/module-ID/myplace/garden/wthr/sr/tstntrvl
    data: integer - 1 .. 15 (seconds)

    in/module-ID/myplace/garden/wthr/sr/adjust
    data: float - -5 .. 5

    in/module-ID/myplace/garden/wthr/sr/delta
    data: float - 0.1 .. 150

    in/module-ID/myplace/garden/wthr/sr/ack
    data: integer 1 / 0

    in/module-ID/myplace/garden/wthr/sr/retrieve/select
    data: integer - 0 .. 4

    in/module-ID/myplace/garden/wthr/sr/retrieve/ack
    data: integer 1 / 0

    eGeoffrey publishes module control data with topic:

    in/module-ID/myplace/garden/wthr
    data: integer - -1 .. 1 number of modules (1, "-1" used to blank the widget)

    in/module-ID/myplace/garden/wthr/switch
    data: integer 1 / 0

    in/module-ID/myplace/garden/wthr/opstest
    data: integer 1 / 0

    in/module-ID/myplace/garden/wthr/default
    data: integer 1 / 0
    in/module-ID/myplace/garden/wthr/reboot
    data: integer - -1 .. number of modules (1, "-1" used to blank the widget)

    in/module-ID/myplace/garden/wthr/ack
    data: integer 1 / 0

    Receiving data

    In eGeoffrey data are received by "sensors". For receiving data also the "processing" aspects are of importance.

    eGeoffrey receives sensor data with topic:

    Processing:
    Automatic aggregation – avg – min – max
    Retention Policies – history

    Beware: the eGeoffrey sensor name can not contain "+". The topic can.

    out/+/myplace/garden/wthr/sr/tmp/ds18
    data: float

    out/+/myplace/garden/wthr/sr/tmp/bme
    data: float

    out/+/myplace/garden/wthr/sr/tmp/xp
    data: float

    out/+/myplace/garden/wthr/sr/wspd/anemo
    data: integer

    out/+/myplace/garden/wthr/sr/wspd/xp
    data: integer

    out/+/myplace/garden/wthr/sr/wdrtn/compass
    data: integer

    out/+/myplace/garden/wthr/sr/wdrtn/txt
    data: string
    (Processing:
    Automatic Aggregation - None
    Retention Policies - Single Value)

    out/+/myplace/garden/wthr/sr/prs/bme
    data: float

    out/+/myplace/garden/wthr/sr/hm/bme
    data: integer

    out/+/myplace/garden/wthr/sr/lght/tsl2561
    data: float

    out/+/myplace/garden/wthr/sr/uv/ml8511
    data: float

    out/+/myplace/garden/wthr/sr/uv/ml6070
    data: float

    out/+/myplace/garden/wthr/sr/rn/volume
    data: integer
    (Processing:
    Automatic aggregation – sum
    Retention Policies – history)

    eGeoffrey receives sensor control data with topic:

    Processing:
    Automatic aggregation - none
    Retention Policies – single value

    Beware: the eGeoffrey sensor name can not contain "+". The topic can.

    out/+/myplace/garden/wthr/sr
    data: string

    out/+/myplace/garden/wthr/sr/switch
    data: string

    out/+/myplace/garden/wthr/sr/opstest
    data: string

    out/+/myplace/garden/wthr/sr/opsntrvl
    data: integer

    out/+/myplace/garden/wthr/sr/tstntrvl
    data: integer

    out/+/myplace/garden/wthr/sr/adjust
    data: float

    out/+/myplace/garden/wthr/sr/delta
    data: float

    out/+/myplace/garden/wthr/sr/retrieve/info
    data: string

    eGeoffrey receives module control data with topic:

    Processing:
    Automatic aggregation - none
    Retention Policies – single value

    Beware: the eGeoffrey sensor name can not contain "+". The topic can.

    out/+/myplace/garden/wthr
    data: string

    DATA INPUT and PRESENTATION

    Sensor data widgets:

    Widget Type: value
    For overviews more Widget Type's are available.

    out/myplace/garden/wthr/sr/tmp/ds18
    out/myplace/garden/wthr/sr/tmp/bme
    out/myplace/garden/wthr/sr/tmp/xp
    out/myplace/garden/wthr/sr/wspd/anemo
    out/myplace/garden/wthr/sr/wspd/xp
    out/myplace/garden/wthr/sr/wdrtn/compass
    out/myplace/garden/wthr/sr/wdrtn/txt
    out/myplace/garden/wthr/sr/prs/bme
    out/myplace/garden/wthr/sr/hm/bme
    out/myplace/garden/wthr/sr/lght/tsl2561
    out/myplace/garden/wthr/sr/uv/ml8511
    out/myplace/garden/wthr/sr/uv/ml6070
    out/myplace/garden/wthr/sr/rn/volume

    Sensor control widgets:

    in/myplace/garden/wthr/sr
    type: input box

    in/myplace/garden/wthr/sr/switch
    type: On/Off switch

    in/myplace/garden/wthr/sr/opstest
    type: On/Off switch

    in/myplace/garden/wthr/sr/opsntrvl
    type: input box

    in/myplace/garden/wthr/sr/tstntrvl
    type: input box

    in/myplace/garden/wthr/sr/adjust
    type: input box

    in/myplace/garden/wthr/sr/delta
    type: input box

    in/myplace/garden/wthr/sr/ack
    type: button
    action: SET in/1D8445/myplace/garden/wthr/sr/ack 1

    Sensor Feedback widgets:

    Widget Type: value

    out/myplace/garden/wthr/sr
    out/myplace/garden/wthr/sr/switch
    out/myplace/garden/wthr/sr/opstest
    out/myplace/garden/wthr/sr/opsntrvl
    out/myplace/garden/wthr/sr/tstntrvl
    out/myplace/garden/wthr/sr/adjust
    out/myplace/garden/wthr/sr/delta
    out/myplace/garden/wthr/sr/retrieve/info

    Module control widgets:

    in/myplace/garden/wthr/
    type: input box

    in/myplace/garden/wthr/switch
    type: On/Off switch

    in/myplace/garden/wthr/opstest
    type: On/Off switch

    in/myplace/garden/wthr/default
    type: button
    action: SET in/1D8445/myplace/garden/wthr/default 1

    in/myplace/garden/wthr/reboot
    type: button
    action: SET in/1D8445/myplace/garden/wthr/reboot 1

    in/myplace/garden/wthr/ack
    type: button
    action: SET in/1D8445/myplace/garden/wthr/ack 1

    Module feedback widgets:

    Widget Type: value
    out/myplace/garden/wthr
    out/myplace/garden/wthr/sr/retrieve/info

    TESTING

    Test configuration

    Laptop1

    Webbrowser with aGeoffrey website to:

    • send control data

    • observe data going in and out of eGeoffrey

    • receive sensor data

    Laptop 2

    • "Coolterm" to observe data going in and out of the module.

    • "MQTT observer" to observe data going in and out of Mosquitto.

    • Spreadsheet to register data input and results.

    Each controlled sensor is tested. Each switch is put in both positions. Each quantity is tested with the minimum, middle and maximum value.

    Quantity position
    switch on/off on off
    switch ops/test on off

    values
    ops interval: 1 - 29 - 60
    test interval: 1 - 8 - 5
    adjustment: -4.99 - 0 - 4.99
    delta: 0.1 - 9 - 149.99

    Every time before a test series the info widgets are cleaned.

    Unexpected results are noted down.

    After conclusion of testing the results are analysed and corrective actions are performed.

    The tests are rehearsed until all control data are transmitted correctly, received correctly, evoked the correct action, the feedback is sent correctly and the feedback is displayed correctly.

    Than the system is ready for operation.

    Rules

    Rules are a means to evaluate the data.
    This rule supervises the heartbeat signal of a module.

    The simplest way is to have a recurrent rule running every minute with:

    • two variables
      value = sensor_id
      value_elapsed = ELAPSED sensor_id

    • two constants
      value_off = 0
      elapsed_threshold = 1

    • And two conditions in the same block
      value == value_off
      value_elapsed > elapsed_threshold

    After the weather station has been on-line for several weeks some adjustments to the software are done. Some of the sensors produce spikes in the data. Some sensors are delivering nothing but spikes.

    To do away with the spikes, the ability to produce extra data on strong fluctuations can be switched off. When the sensor is in overload, transmission of data is suppressed.

    This results in a far more intelligible presentation of the data.

    All this preparation and labour resulted in a working weather station.

    To give you an impression of how my weather station looks like here are a few screenshots:

    Local weather data

    Local.jpg

    All temperature sensors

    Overview Temp.jpg

    Changing settings per sensor

    Change.jpg

    Feedback from the weather station

    Info.jpg

    Adjusting all sensors together

    Module.jpg

    Enjoy!

    0
  • U

    Lovely How To, thanks for sharing it! Very comprehensive and detailed. Question, are you also aiming to publish the weather station data to some online weather service? I had in the past something in eGeoffrey for publishing to Weatherunderground before being acquired by IBM, just trying to understand if there is the need of something similar for other services (e.g. Windy, etc.). Thanks!

    0
  • E

    @user2684

    I do have yet another wish where you could do me a tremendous fever.
    In the Netherlands we do have a nice weather system. I would like to connect to that system. There is an API in python available. Would you be so kind and have a look at it wether it could be an input for eGeoffrey also?

    [https://pypi.org/project/buienradar/](link url)

    0
  • U

    @eporocrail said in Create a weather station:

    https://pypi.org/project/buienradar/

    Sure let me have a look and I'll let you know.
    I've also linked this fantastic guide from https://docs.egeoffrey.com/howto/

    0
Log in to reply