Is it cold outside? Part 2

Written By Patrick Wagner

When I left you last, we’d got most of the information for the project assembled and just needed to extract the information from the webpages we found and set up push notifications before sewing it all together - let’s get that sorted now so we can get this up and running!

I’ve used pushover for notifications before so I’ll use that - signing up for an account gives you 30days free, after that it’s $5USD for a lifetime license for an individual which I love. They charge for larger groups using their service but keep the introductory price low and giving me 10k notifications/month gives me a lot of freedom for testing/maintaining multiple projects. Checking the API page I can see I need to send 3 things to “https://api.pushover.net/1/messages.json” for the notifications to work:

  1. User key - you get this when you make an account

  2. API token - you get this when you set up a project

  3. A message - do I really need to explain this one?

After this, just log into your account on a mobile device and you’re all ready to send notifications. Try this:

url_api = 'https://api.pushover.net/1/messages.json'
user_token = <secret>
api_token = <secret>
post_body = {'token': api_token, 'user': user_token, 'message': "Hi!"}
post_request = requests.post(url_api, json=post_body)

Now I have notifications working, let’s work on how to extract the information I want out of the webpages. I quickly introduced BeautifulSoup (BS) in the last entry, but I’ll dive more into it’s use today. After parsing HTML code, BS is able to search for information by tag, class, ID, or a combination. If you want the ‘apple’ class <div> tags, you can grab them easily. If you only want the ‘apple’ class <div> tags that are inside <tr> tags, then that’s just as easy! So here’s an example of the syntax:

result = requests.get(url, headers=header_list)
soup = BeautifulSoup(result.content, 'html.parser')
data = soup.find_all('tr', class_='rowleftcolumn')[0].get_text()

This will parse the webpage at ‘url’ and then find all <tr> tags of the class “rowleftcolumn”, outputting them to a list but I’ll just get the first result by specifying ‘[0]’ - I then extract the text from the tag, rather than all the information. That means that when presented with the following:

<tr class="rowleftcolumn">
  <td headers="t1-datetime">24/09:30pm</td>
  <td headers="t1-tmp">12.1</td>
  <td headers="t1-apptmp">9.6</td>
  <td headers="t1-dewpoint">9.7</td>
  <td headers="t1-relhum">85</td>
  <td headers="t1-delta-t">1.2</td>
  <td headers="t1-wind t1-wind-dir">WSW</td>
  <td headers="t1-wind t1-wind-spd-kmh">13</td>
  <td headers="t1-wind t1-wind-gust-kmh">17</td>
  <td headers="t1-wind t1-wind-spd-kts">7</td>
  <td headers="t1-wind t1-wind-gust-kts">9</td>
  <td headers="t1-press-qnh">1031.8</td>
  <td headers="t1-press-msl">1031.8</td>
  <td headers="t1-rainsince9am">0.2</td>
</tr>

I’d get this output, which lets me pick the line that I want

24/09:30pm
12.1
9.6
9.7
85
1.2
WSW
13
17
7
9
1031.8
1031.8
0.2

So when approaching any script you should plan beforehand - let’s map out what I want to do:

  • It should check the BoM forecast to see if it’s a hot day

  • Only if it’s a hot day should it start checking my internal sensor and the current temperatures from BoM (don’t want to spam them unnecessarily)

  • The first time during a hot day that the outside temperature drops below my internal temperature, it should notify me

And now it’s time to compile everything!

import time
import requests
from bs4 import BeautifulSoup
import datetime
import Adafruit_DHT

# variable setup
sensor = Adafruit_DHT.DHT11
pin = 17
url_api = 'https://api.pushover.net/1/messages.json'
user_token = <secret>
api_token = <secret>
url_max = 'http://www.bom.gov.au/nsw/forecasts/sydney.shtml'
url_current = 'http://reg.bom.gov.au/products/IDN60901/IDN60901.94767.shtml'
header_list = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0'}
max_checked = False
notified = False
max_temperature = 0.0
temperature = 0.0

while True: # This will run forever
  ###setup###
  hour_now = int(datetime.datetime.now().strftime('%X').split(':')[0]) # extracts the 'hour' value from the current time
    if hour_now == 1 and max_checked == True: # if the local time is 1AM, it resets values for the new day
        max_temperature = 0.0                   
        max_checked = False
        notified = False

    if hour_now > 12 and max_checked == False: # after midday, it checks the forecast for the max expected temperature once
        result = requests.get(url_max, headers=header_list)
        soup = BeautifulSoup(result.content, 'html.parser')
        max_temperature = float(soup.find_all('em', class_='max')[0].get_text())
        max_checked = True
  ###checking###
    try:
        humidity, temperature = Adafruit_DHT.read_retry(sensor, pin) # it checks the current internal temperature
    except:
        pass

    if max_temperature > temperature or max_temperature > 25: # if its colder inside (indicating it's a hot day) or the forcast is >25C
        if hour_now > 15 and hour_now < 21: # between 3pm & 9pm it gets the current temperature
            result = requests.get(url_current, headers=header_list)
            soup = BeautifulSoup(result.content, 'html.parser')
            current_temperature = soup.find_all('tr', class_='rowleftcolumn')[0].get_text()
            current_temperature = float(current_temperature.split('\n')[3])
  ###notifying###
            if current_temperature < temperature and notified == False: # if it's cooler outside after a hot day and I haven't been notified already - sends a notification
                post_body = {'token': api_token, 'user': user_token, 'message': "It's time to open your windows - it's cool outside"}
                post_request = requests.post(url_api, json = post_body)
                notified = True
    else:
        continue

    time.sleep(1800) # sleeps for 30mins before restarting

I’ve just thrown a lot on the screen so let’s go through it. I start by importing all the libraries I need and declaring my variables. Some of the variables I’ve already covered, but a note on declaring the last 4 - essentially after a single run-through, each will already have a value, but on it’s first run it needs default values set so it has something to check against. All the if statements refer to these variables and if they don’t exist then the script fails before it fully starts.

I wrap everything in a ‘while True’ loop with a 30min sleep at the end - this reduces the use of my pi’s resources and reduces the strain on the BoM’s site. Now, will one machine have an impact on their website, probably not - but like I said before, it’s the polite thing to do and if enough people aren’t considerate, then BoM might start to feel an impact. As for the main body - I split it into 3 sections: setup, checking & notifying.

Setup resets the values every morning at 1am and checks the day’s maximum temperature in the afternoon. Checking gets the output of the internal sensor, decides if it’s a hot day that day, then monitors the temperature observations for a drop in temperature. Finally, notifying sends the push notification to my phone when the hot day cools enough.

And now I have a working weather-checking script! Buuuuut there’s a small snag - how do I get it to run automatically? What if there’s a blackout? An error? I’ll cover that in the next entry!

Patrick Wagner
Previous

Is it cold outside? Part 3
Next

Is it cold outside? Part 1