Temperature is published but configuration (fan/light) does no update; control_server.py just hangs

[handcyclist]

Hello:
I've been trying to make this work but with only partial success.
pubsub_thermostat.py publishes the temperature to my topic02 but does not get updates from my topic01
control_server.py runs but does not update configuration and just hangs after printing "Server running. Consuming telemetry events from projects//topics/topic02
Today I reinstalled all the packages as per your instructions.
When I started on this, about a week ago, I also updated my Raspberry as per your instructions.
This is my first experience with Python.
I have setup my google/firebase project so that messages going to my google pubsub trigger a function that sends the messages to firebase pubsub. From here successfully modified an Android/Firebase sample app that reads and shows me temperature.
I can see the temp read by my BME280 on my Raspberry on my Android, but I want to be able to make "state" changes from the Cloud to my Raspberry.
I assume the problem is due to newer updates. Please let me know what information I should send you, or if you are already aware of this, what should I expect from this point.
Thank you very much for your help

[handcyclist]

In case this is related to my problem, the following instruction in the main .md file, does not work. It gives a not authorized message.
sudo gcloud components repositories add https://storage.googleapis.com/cloud-iot-gcloud/components-2.json

[mdupuy]

If you're running "sudo gcloud" and getting the error that you're not authorized it is because you probably authorized your "pi" account with "gcloud authorize" but when you're running anything with sudo, it is running as root, which you probably didn't authorize. In which case, just run "sudo gcloud init". This is explained in the paragraph just above the command, albeit in slightly abstract terms since different systems have different requirements regarding the need for using "sudo".

As for your original issue report, there should not be two different topics. The thermostat.py should publish to one topic that you pass in when you start it, the server should consume from that same topic and send back config updates. I think the confusion might be a change in terminology between the beta release of IoT Core and v1 from "events" to "topics". I've updated the front readme.md to reflect the change in any environment variables you might have set. i.e.

mysub=my-sub
events=events
mytopic=events

The variable "events" was updated to "mytopic" the example readme.md but not defined in the top level readme.md. Regardless, this might not be your problem. I'm not in front of my kit right now to check if this could be the issue but I'll try to get to it in the next 24hrs and check and respond further.

[handcyclist]

Thank you for replying back quickly, that's very much appreciated.
--about the 'repositories' issue:
I just tried "sudo gcloud init" and "sudo gcloud auth application-default login" but did not work. This last one (auth) tells me that I am not setup; the "init" one starts the initialization process but it fails to sign-in throug the browser (because of "root")
I tried the "repositories" command again with sudo and without sudo and this is the error message:

---

pi@raspberrypi:~ $ gcloud components repositories add https://storage.googleapis.com/cloud-iot-gcloud/components-2.json
ERROR: (gcloud.components.repositories.add) Account [<my.google.account>] does not have permission to install this component. Please
ensure that this account should have access or run:

$ gcloud config set account ACCOUNT

to choose another account.

HTTP Error 403: Forbidden
pi@raspberrypi:~ $ sudo gcloud components repositories add https://storage.googleapis.com/cloud-iot-gcloud/components-2.json
ERROR: (gcloud.components.repositories.add) This component requires valid credentials to install.

You do not currently have an active account selected.
Please run:

$ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

$ gcloud config set account ACCOUNT

to select an already authenticated account to use.
pi@raspberrypi:~ $

---

---about the "server not working issue"
Today I realized that "state" is not the same as "config". The second topic I mentioned (topic01), I had assigned for publishing "state" but then I realized that there is kind of a built-in "config" topic.
I also realized that the topic subscription that I was originally using was actually a "push" subscription in order to move messages from cloud-pubsub to firebase-pubsub.
I added a new "pull" subscription to my topic but the server still did not work.

[handcyclist]

I just reboot my Raspberry and the server now appeared to start doing what it was supposed to do, but the light (fan) would not turn on.
(It was turning on when I put the stuff together and tested; I'll check to make sure the light works with your test routine)

[handcyclist]

I did successfully check that the Raspberry is able to turn the LED On and Off on GPIO.
The server now runs, however the LED does not turn On while control_server.py and pubsub_thermostat.py are running.
Another interesting issue is that if I use , the server hangs, but if I do not set GOOGLE_APPLICATION_CREDENTIALS, then the server runs; it gives a warning when starting; the warning is something about limitations. The server runs for a while but then it exits; the last time gave the following message: "./runTempControl: line 1: 2405 Segmentation fault python control_server.py --project_id="...........". runTempControl is an executable that contains the running script "python control_server.py "

---

The pubsub-thermostat.py apparently can subscribe to "config" but nothing happens even when the server indicates that it is changing the Fan to On.
Here is the text when pubsub-thermostat.py starts:

pi@raspberrypi:~/Cloud-IoT-Core-Kit-Examples/pubsub-thermostat $ ./runSensor
pubsub_thermostat.py:163: RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
GPIO.setup(FAN_GPIO, GPIO.OUT)
Creating JWT using RS256 from private key file rsa_private.pem
Connection Result: 0: No error.
Publishing payload {"temperature": 28}
Subscribed: (1,)
Received message '' on topic '/devices/RaspberryPiWith16GigaPonySDcard/config' with Qos 1
Received message '' on topic '/devices/RaspberryPiWith16GigaPonySDcard/config' with Qos 1
Published message acked.

[handcyclist]

---no clue why the "bold" text above---
Thank you for your help

[mdupuy]

Your "gcloud init" process sounds unnecessarily complicated. If you did "gcloud init" from your "pi" (or other user account), you gave it your Google credential/email address. Then you gave it the project ID and it gave you a URL to paste into a browser that was logged in to Google with that same set
of credentials to get a token. Did that all work fine? If so, the same set of steps should work with "sudo gcloud init" but it will authorize your "root" account with those same Google credentials. You don't need to be running the browser as root. This is all a bit of a digression though, it sounds like you have everything set up and don't need to use the gcloud command for any further setup.

If the LED changes with the python code in the readme, it'll change in the project provided the temp thresholds are passed and you are getting the config changes. You might have to tweak the threshold temps to your ambient room temp a bit before using your fingers or breath or similar methods to change the sensor temp. (sorry if that part was obvious, just going down my mental checklist before starting a new environment from scratch to replicate)

[handcyclist]

I modified the threshold argument to on at 30 and off at 29 since when it starts at ambient is currently 27-28. But even after the server read a temp higher than 30, the LED did not turn-on.
By the way, I noticed that the server file is trying to use fan_on_thresh and fan_off_thresh but the arguments that it is parsing are fan_on and fan_off. I changed that in my server, but nothing happened.

About the "repositories" command, I had tried with sudo inside "pi" user and the shell gives some issues although it is able to open the browser login page, but then the browser goes numb, does nothing.
I tried from the "root" account but it did exaclty the same with the shell and the browser. I did it in root without sudo and with sudo and no difference.

I am going to try to attach some pics

[handcyclist]

[handcyclist]

-In the first and second picture, there was a message about "no handler could be found for logger concurrent.futures"
-Also please notice how the server is going crazy reading files from the topic, not precisely the latest files
-At some points, the server can see the higher temp, but no changes in LED
-The last pic shows the same "repository" command problem when I tried from an elevated shell in windows10

[mdupuy]

I was able to replicate part of this issue after a full system wipe and bring up.
In the move from beta to v1, tls_version=ssl.PROTOCOL_TLSv1_2 became a requirement. I've fixed that. My LED comes on and off when the temperature changes but there is some issue with the ack-ing of messages coming from the thermostat that is making the server send changes more frequently then it should. The demo works at my end but after running it, I need to empty out the queue using the gcloud command before running it again:

while gcloud pubsub subscriptions pull --auto-ack projects/$project/subscriptions/$mysub; do echo; done

I'll try to track this down so I'll leave this issue open for now.

[handcyclist]

The pubsub-thermostat example finally worked for me.

The last thing I did after which the example worked well was to create a new service account, this time with project-owner properties instead of project-editor, and adding many roles, including iot, pubsub, and a bunch of others.

Before running "gcloud auth application-default login" I made sure of the following:

1-That the shell was in the pubsub-thermostat directory

2**-That the service_account.json (from my new service) was in directory pubsub-thermostat

3**-That variable GOOGLE_APPLICATION_CREDENTIALS was setup using "export GOOGLE_APPLICATION_CREDENTIALS='/home/pi/googlekeys/api-tester1/service_account.json' " (which is the directory I chose).

**I'm not sure if both #2 & #3 are required, or if just #2, or if just #3, but that's what I did and it finally worked.

On the pubsub-thermostat "md" file, the link "Create an API key and service account named api-tester" points to something completely irrelevant to API keys or service accounts; by the way, I found the info for creating API keys at one location, and a completely different location for info on creating service accounts. The picture shows the link I mention.

[handcyclist]

---

I modified the publishing app "pubsub_thermostat.py" to run continuously instead of just a number of measurements.

When running continuously, the app runs out of memory and gets disconnected, but the app keeps running taking data from the sensor but unable to send it; trying to go around this I modified the publish data loop in the "main" method to detect if the connection was broken, and if it was to terminate the loop and start "main" again. That works apparently, but I haven't been able to leave the app running for more than a day so far. I also added some instructions to delete variables that are set with each cycle; not sure this is really helping though.

I also made a few other changes, mainly to print something so I could understand whether the app was getting to the required places

---

I figured out that the reason why the server app "control_server.py" was not acknowledging messages was because the instruction was spelled incorrectly: was "messesage.ack()" (that is "mess + es + age") instead of "message.ack()" (that is without the extra "+es").

I made some more changes trying to improve the app; one of those changes was to delete the values of variables that will be set again through the cycle iterations. I did this because yesterday, after the control_server ap finally worked, I left it running but at some point it just kept cycling through the 60 sec while loop; this morning I reboot my raspberry and the app worked again. So far it's been working fine for the last 4 hours or so.

Placed some print statements here and there, switch to pubsub_v1 (instead of pubsub) and SERVICE_NAME = 'cloudiotcore' (instead of 'cloudiot'), but works either way.

Added "try and except" statements to def _update_device_config(…; I found these somewhere else when I was trying to understand why it was not working.
Changed the arguments from fan_on & fan_off to fan_on_thresh and fan_off_thresh; I don't understand how the constants fan_on_thresh and fan_off_thresh get their values from the arguments fan_on & fan_off; (Is it because fan_on_thresh starts with fan_on and therefore fan_on_thresh kind of inherits the values from fan_on, and so on?)

---

I am going to try to send the text of my modified apps for you to review, comment, and use if desired.

Thanks for your help.

[handcyclist]

control_server.txt
pubsub_thermostat.txt