My general sensor system



  • Some weird things happening having nothing to do with Geoffrey but with Raspi OS.

    I gave Raspi OS in the most recent version another try.
    The Wifi manager is now OK.
    When I do NOT update/upgrade the OS and go then to eGeoffrey market place, the modules are listed.

    After update/upgrade the marketplace is not accessible anymore?????????

    Going to try some things to look if I can find a workaround.



  • @eporocrail wonder if the update includes some new curl/wget version which requires different switches than the one I am using. Would you mind detailing the steps so I can fully reproduce? (e.g. which raspbian should I start with, which upgrades, etc.) Thanks!



  • @user2684

    I am using RaspioOs
    Version: 2020-05-27-raspios-buster.armhf.img
    Release 3.3 May 2020

    Install without automatic update.
    Access to martketplace is OK

    Manual update some info from terminal:

    Unpacking rpi-eeprom (7.2-1) over (7.0-1) ...
    Preparing to unpack .../20-raspberrypi-ui-mods_1.20200603_all.deb ...
    Unpacking raspberrypi-ui-mods (1.20200603) over (1.20200514) ...
    Setting up rp-prefapps (0.22) ...
    Setting up rpi-chromium-mods (20200602) ...
    Setting up raspberrypi-kernel (1.20200601-1) ...

    After this update I have no access any more to the marketplace



  • @eporocrail ok so if I understand correctly you start from 2020-05-27-raspios-buster.armhf.img then apt-get update & apt-get upgrade and when trying to install a new eGeoffrey package you get the behaviour you described on https://forum.egeoffrey.com/topic/15/installing-new-service/3, correct? If so I'll try to reproduce the problem. Thanks



  • @user2684
    Before the update I can access the marketplace with web browser.
    After update/upgrade manually I can not access the marketplace anymore.
    The webpage keeps displaying "loading" and nothing happens.



  • @eporocrail ok so the problem is when accessing with the web browser, not with the CLI, my bad in understanding. Are you using the browser on the raspi or on a different device to access? If you experience the problem with both my guess is that the thing has nothing to do with the upgrade but when you refresh multiple times the marketplace website, since it is building the page on the fly by pulling information from github, when reloaded multiple times, github is blocking too many requests (try visiting https://api.github.com/repos/egeoffrey/egeoffrey-marketplace/contents/marketplace for confirmation). If so I need to add an explicit warning or something for this. Thanks



  • @user2684 I think you are right. I was able to install the mysensors module.
    But that would mean that Github is blocked VERY fast.

    I expect to proceed with the installation not earlier than in two days.



  • @eporocrail looks like the case, I had a similar issue a while ago but with many many times the page realoaded, but I see this happening faster now, could easily be github lowering down the limit. I've opened up an issue to fix this (both for the marketplace and the CLI - https://github.com/egeoffrey/marketplace.egeoffrey.com/issues/1), it could take a while since I need to figure out the best way to keep a local cache, hope it is not a blocking issue for now.



  • @user2684

    I did a complete new install starting with Raspi OS.
    I was able to install Geoffrey and the mysensor service. No problem with GitHub.
    This time I was not logged in as "Guest" but as "admin".

    Inserted the details of the house.

    Started inserting a new sensor.
    I filled the fields of the first tab. I switched to "Advanced editor" and back again.
    The content of the fields I just inserted, was gone.

    I inserted the data again and clicked "Save". The system did not only "save" but also "close". After a "save" one leaves the sensor configuration.

    Where do I find information about the values for the field "Sensor Icon"?

    On the tab "data" it is not clear what "format" means. I wanted to insert a simple on/off button. What "format" do I have to select?

    Where I am using MQTT I would expect to insert data for the broker somewhere.
    Under "Sensor Configuration" there is no place for these data.

    I found it under "eGeoffrey - modules"

    Later I will continue configuration and testing
    I will report in a separate post



  • @user2684
    Some more experiences of which some require your attention.

    Register a new sensor.

    Weird behaviour: as soon as the the data on tab "General" is saved the input window is closed instead of data only being saved.

    After inserting data on tab "general data" go to "Advanced Editor". All empty. Go "Back".
    Now the newly inserted data are gone.

    Unclear what data is inserted into filed "Sensor Icon".

    Do NOT save.

    Tab "Data"
    Field "Format" is not clear. I want to insert a push button as sensor. I would expect to insert something like "On/Off".

    Tab "Processing"
    Automatic aggregation. Select from pull-down menu "None".
    Retention Policies. "single_value".
    Convert acquired value before being saved. "1_to_ON".
    Convert input value before being sent to an actuator. "None".

    Tab "Service"
    Associated Service. Select from pull-down menu "mysensors_mqtt: Interactwith a Mysensors MQTT gateway"
    How to interact with the gateway. "Push – the sensor will periodically provide new data".

    Tab "Configuration"
    The unique identifier of the node. "11".
    Each node can have several sensors attached. This is the child-id that uniquely identifies one attached sensor. "5".
    Mysensors command to trigger. Select from pull-down menu "SET".
    Mysensors type. "S_BINARY".

    Configuring the "mysensors" module.

    Under "eGeoffrey" goto "Modules".

    At the bottom three versions of the "service/mysensors" are visible and running.
    For the "Ethernet" and "Serial" version click on the "arrow-down" in the "Actions" field.
    Click on "Stop module".
    Click on the "arrow-down" in the "Actions" field.
    Select "Edit Configuration". Module Configuration check the box "Disable the module".

    Now only the module "service/mysensors_mqtt" is running.

    For the "mqtt" version click on the "arrow-down" in the "Actions" field.
    Select "Edit Configuration"

    Module Configuration.

    The MQTT hostname to connect to. Insert the "hostname" or the IP-Address. "192.168.2.1" in my case.

    The port of the MQTT Broker. Is set to "1883". Leave as is.

    The username for authenticating against the MQTT broker. In my case leave empty.

    The password for authenticating against the MQTT broker. In my case leave empty.

    Topic prefix to subscribe to. "kamer/uit".

    Topic prefix to publish to. "kamer/in".

    Under "eGeoffrey" goto "Icons". Here they are. It is a list with general purpose icons. Not many I think useful for eGeoffrey.

    Looking into the logs it appeared that errors where caused by the mysensor modules which are not used.
    Reboot the Raspi.
    After reboot under "eGeoffrey" "modules" all three mysensor modules are green (running).
    Checked to see if the ethernet and serial version still where checked "Disable module". Yes.

    Stopped both modules. No effect. Both modules keep being interrogated.



  • The marketplace issues has been solved, both for https://marketplace.egeoffrey.com and when using the CLI (please upgrade to v1.0-26 with sudo egeoffrey-cli upgrade). The marketplace website now builds periodically a cache of all the items of the marketplace and both the website and the CLI get information from this cache so avoiding hitting github directly and so be blocked for the too many requests 🙂



  • Thanks once again for these precious feedbacks. Let me reply point by point:

    Weird behaviour: as soon as the the data on tab "General" is saved the input window is closed instead of data only being saved.

    This is expected, the button save and close the window. Do you think a different label on the button would work better?

    After inserting data on tab "general data" go to "Advanced Editor". All empty. Go "Back".
    Now the newly inserted data are gone.

    Still expected (or better a different behaviour would be really difficult to implement). I'd recommend avoiding using the Advanced Editor at the beginning since the configuration schema is not documented yet and there are no options by the wizard. I could take the link away if this is creating confusion.

    Unclear what data is inserted into filed "Sensor Icon".

    This has to be documented better for sure. This field (any icon-related field) is expecting the name of an icon taken from the "eGeoffrey" / "Icons" menu under Administration

    Field "Format" is not clear. I want to insert a push button as sensor. I would expect to insert something like "On/Off".

    I've added a bunch of new pages on the docs website, have a look at https://docs.egeoffrey.com/configure/sensors/ if not done yet, wonder if clear enough. Format is the format of the data supposed to be stored in this sensor (being a sensor a logical container of one or more values). For a on/off sensor you can use "integer" and store 0 or 1. If the remote sensor is instead sending or expecting the string on/off, then would be string.

    Convert acquired value before being saved. "1_to_ON".

    This entirely depends on the logic of the attached sensor. If you set integer and store a 0 or 1 and the remote relay expect 0 or 1 to trigger this is not required. When you set "1_to_ON" means every time a new value comes in (from the remote sensor or from the web interface, the 1 is converted to the string "ON" which is not necessarily what you want to use. The web interface has a on/off widget which expects a 0/1 stored in the sensor.

    Configuring the "mysensors" module.
    Under "eGeoffrey" goto "Modules".

    Yes, this is usually to be done before configuring the sensor (I tried to describe the workflow in this new page: https://docs.egeoffrey.com/configure/workflow/). Configuring the module set up the integration, the sensor leverages then the integration

    Select "Edit Configuration". Module Configuration check the box "Disable the module".
    Now only the module "service/mysensors_mqtt" is running.

    An alternative (more advanced) would be to edit the docker-compose.yml file in the eGeoffrey directory, identify the entry for the MySensors package and under EGEOFFREY_MODULES, removing those you would not need.

    After reboot under "eGeoffrey" "modules" all three mysensor modules are green (running).
    Checked to see if the ethernet and serial version still where checked "Disable module". Yes.
    Stopped both modules. No effect. Both modules keep being interrogated.

    Ok so you're saying even after disabling those modules they are still running after a reboot. I'll try to reproduce it. What about the mqtt one, is everything working there?
    Thanks!



  • @user2684 said in My general sensor system:

    Weird behaviour: as soon as the the data on tab "General" is saved the input window is closed instead of data only being saved.

    I would like to come back to this issue once again. Like in several other windows I would like to only save data when clicking the "Save" button. And I would like to close the window when I click the "Close" button. At this moment when I click the "Save" button data are saved AND the window is closed. The latter action should not be invoked by clicking only the "Save" button.

    After inserting data on tab "general data" go to "Advanced Editor". All empty. Go "Back".
    Now the newly inserted data are gone.

    I think it is a little annoying that by going to a window one step deeper down the road the data inserted into the previous window are deleted.

    "Ok so you're saying even after disabling those modules they are still running after a reboot. I'll try to reproduce it. What about the mqtt one, is everything working there?"

    Yes. Whatever I try to disable the two of the MySensors modules which I do not need, is not working. So from the user perspective all three MySensors modules are running where two of the three modules keep producing a huge amount of errors in the logs.

    I assume that I am able to disable them in the docker.compose.yml file but that is I think not the way to have it done by the less software minded user.

    The second part of your remark is for me not clear.(What about the mqtt one, is everything working there?)
    Do you mean that next to the "MySensors" modules I also have to install extra the "MQTT" module?
    I will give it a try but if this where the case then again this is not clear for the user.
    Maybe it would be more elegant to treat The MQTT module more or less as a "dependency" of the "MySensors" module and have the "MQTT" module installed automatically. (if possible, I am not familiar with docker at all)

    I am going to look into the new documentation and I will try to get some message into eGeoffrey one way or another I can think of.



  • @user2684

    I tried to have a look at the documentation. Something is wrong with the menu structure.

    When I follow the link you provide in your last post I am able to access the documentation. But as soon as I activate the link "Welcome" the access to documents is not working anymore.

    The only way now to get to any useful info is following the above mentioned link. The normal way to access the documentation is disrupted.



  • @eporocrail said in My general sensor system:

    I tried to have a look at the documentation. Something is wrong with the menu structure.

    This has to do with the browser cache I think, try a shift+F5 in the main page, you should see the menu populated correctly. Thanks



  • @user2684

    First of all you did a great job on the documentation.
    Very much has been added.
    I assume you will go through the text again to get rid of the few typos which are still there.

    The last three links (Configure rules, Upgrades and User management
    are leading to "404".
    I think that the links are not OK or just hide the links until the pages are present in the documentation.

    A different question I have is:
    How do I test that a message is entering eGeoffrey via the "MySensors" interface?

    The situation is as follows:
    I push a button. I can seen that my system sends the message in the right format to Mosquitto. The next step is check on a second machine with the terminal if there it is possible to receive that message.
    Thereafter I would like to see that that message arrives in eGeoffrey.

    To put it differently: I want to be sure that the message evoked by the button action is following the path it is supposed to do, step by step.

    By the way, after I managed to get the system with a simple push button up and running I am going to make e step-by-step description for my own documentation. That document I intend to publish on this forum also.



  • would like to come back to this issue once again. Like in several other windows I would like to only save data when clicking the "Save" button. And I would like to close the window when I click the "Close" button. At this moment when I click the "Save" button data are saved AND the window is closed. The latter action should not be invoked by clicking only the "Save" button.

    Understand, I've opened it https://github.com/egeoffrey/egeoffrey-gui/issues/42

    After inserting data on tab "general data" go to "Advanced Editor". All empty. Go "Back".
    Now the newly inserted data are gone.
    I think it is a little annoying that by going to a window one step deeper down the road the data inserted into the previous window are deleted.

    Unfortunately since the advanced editor is actually open up a new page, I have no (easy way) to stage the changes so for now I prefer removing the link (https://github.com/egeoffrey/egeoffrey-gui/issues/43) since don't expect users to use the advanced editor (which is regardless always available in the administration menu through which the entire configuration can be edited manually without accessing the filesystem)

    Yes. Whatever I try to disable the two of the MySensors modules which I do not need, is not working. So from the user perspective all three MySensors modules are running where two of the three modules keep producing a huge amount of errors in the logs.

    You are right and this could be also affect other services, I may have missed it along the way. I should have fixed it for now for the MySensors package, update it by running sudo egeoffrey-cli update egeoffrey-service-mysensors and then sudo egeoffrey-cli start which should restart the newly updated package only. You should see all the three modules running but only the MQTT one configured.

    I assume that I am able to disable them in the docker.compose.yml file but that is I think not the way to have it done by the less software minded user.

    Agree, users are not supposed to get into that file. I'll also provide an option in the future through the cli to do so (https://github.com/egeoffrey/egeoffrey-cli/issues/8) in case this will needed

    The second part of your remark is for me not clear.(What about the mqtt one, is everything working there?)
    Do you mean that next to the "MySensors" modules I also have to install extra the "MQTT" module?

    No, just wondering if you had it working as you were expecting 🙂 No need for any entra package. Only thing you need is of course a MQTT broker to make your sensors and eGeoffrey connect to. Of course it could be the one eGeoffrey is already using as a message bus, the ports are already open and eGeoffrey is using a dedicated topic so would not interfere with your sensors. Ah, I recommend to enable debug in the modules page for the mysensors mqtt module so you can get more info in the logs.



  • I assume you will go through the text again to get rid of the few typos which are still there.
    The last three links (Configure rules, Upgrades and User management
    are leading to "404".

    Yes, I tried to dump out as many information as possible to help you along the way but something is still missing and needs a good review.

    To put it differently: I want to be sure that the message evoked by the button action is following the path it is supposed to do, step by step.

    Enable debug on the mqtt mysensors module first so to be sure you can see more details in the logs. When clicking the button you should start seeing something in showing up in the logs since once configured that module subscribe to the topic you have configured. If there is no sensors mapping that specific subtopic, nothing happens. So next step si to configure the associated sensor in eGeoffrey. If the button is sending a 0 or 1 value, configured it with an integer format. When done, once you press the button you should see an additional line in the logs saying the sensor got a new value. Alternatively check the box at the top right of the notification bar and for every new value set to any sensor you will get a popup showing up in the web interface.

    By the way, after I managed to get the system with a simple push button up and running I am going to make e step-by-step description for my own documentation. That document I intend to publish on this forum also.

    Thanks a lot, really appreciated. Adding a sensor to eGeoffrey should not be difficult, if it is means there is something to be done to make things simpler. And for sure one thing is improving the documentation with real examples. Thanks again!



  • @user2684

    I completely shut down my Mac and started it again. Than I went to your website and tried to go to the documentation.

    No success. Still showing an old version of the menu. One link leads to 404.

    ![0_1592127766918_Screenshot 2020-06-14 at 11.30.48.jpg](Uploading 100%)



  • @user2684
    As you can see uploading jpeg of the screenshot does not work. In the preview window it reads that error occurred. Connection lost. ???


Log in to reply