Usage Examples for the bmondata Library

Installation

The package can be installed through use of pip:

pip install --upgrade bmondata

The --upgrade flag ensures that the newest version will replace a bmondata version that you may already have.

Use of the Server Class

The Server class is used to initiate requests to one BMON Server. The base URL of the BMON server is the one required parameter when instatiating the object. If you want to use the Server object to store sensor readings on the server, you also need to provide the 'store_key' parameter. The 'store_key' is the secret key found in the settings.py file on the BMON server.

In [12]:
import bmondata

# Make a Server object for retrieving data only
server = bmondata.Server('https://bmon.analysisnorth.com')

# The Server object below can also be used to store sensor readings 
# on the BMON server.
server = bmondata.Server('https://bmon.analysisnorth.com', store_key='temporary-key')

Retrieve Sensor Readings

The sensor_readings() method is used to retrieve sensor readings from one or more sensors. The readings are returned in a Pandas DataFrame. To retrieve readings from one sensor, specify the Sensor ID as the first parameter. (You can determine a Sensor ID by viewing the 'Current Sensor Values' report and then hovering your cursor over the name of a sensor; the Sensor ID will be shown on the last line of the pop-up window.)

In [2]:
df = server.sensor_readings('phil_hp_pwr_10187_temp')
df.head()
Out[2]:
phil_hp_pwr_10187_temp
2018-01-20 14:53:38 26.600
2018-01-20 14:56:50 26.600
2018-01-20 14:59:42 26.582
2018-01-20 16:28:59 25.777
2018-01-20 16:32:11 25.880

The timezone used for the date/time stamps defaults to the timezone of the building that the sensor is associated with. If there are multiple buildings linked to the sensor, the most prevalent timezone is used. However, you can force the timestamps to use a particular timezone by adding a timezone parameter to the function call:

df = server.sensor_readings('phil_hp_pwr_10187_temp', timezone='US/Alaska')

The timezone parameter is a string value and must be one of the values in the list pytz.all_timezones (pytz is a pip-installable Python package).

To retrieve readings from multiple sensors, provide a list of Sensor IDs:

In [3]:
df = df = server.sensor_readings(['phil_hp_pwr_10187_temp', 'phil_hp_pwr_7470_temp'])
df.head()
Out[3]:
phil_hp_pwr_10187_temp phil_hp_pwr_7470_temp
2018-01-20 14:53:38 26.600 64.940
2018-01-20 14:56:32 NaN 64.958
2018-01-20 14:56:50 26.600 NaN
2018-01-20 14:59:42 26.582 65.120
2018-01-20 16:28:59 25.777 NaN

Readings from multiple sensors often are not synchronized in time, thus the DataFrame will include many NaN values. Time-averaging of readings is discussed later and can eliminate most of the NaN values.

You can have the DataFrame use more meaningful column names by providing a column label for one or more of the sensors:

In [4]:
sensors = [
    ('phil_hp_pwr_10187_temp', 'outdoor_temp'), 
    'phil_hp_pwr_7470_temp'
]

df = server.sensor_readings(sensors)
df.head()
Out[4]:
outdoor_temp phil_hp_pwr_7470_temp
2018-01-20 14:53:38 26.600 64.940
2018-01-20 14:56:32 NaN 64.958
2018-01-20 14:56:50 26.600 NaN
2018-01-20 14:59:42 26.582 65.120
2018-01-20 16:28:59 25.777 NaN

For each Sensor that you wish to label, use a two-tuple containing the Sensor ID and the Sensor Label instead of just supplying the Sensor ID.

To filter the readings based on date/time, use the start_ts and end_ts parameters:

In [5]:
df = server.sensor_readings(
    sensors,
    start_ts = '2019-01-15 3:00 pm',
    end_ts = '2019-01-17 10:30 am'
)
df.head()
Out[5]:
outdoor_temp phil_hp_pwr_7470_temp
2019-01-15 15:00:20 NaN 70.58
2019-01-15 15:00:21 33.04 NaN
2019-01-15 15:02:23 33.04 NaN
2019-01-15 15:02:31 NaN 70.58
2019-01-15 15:04:01 NaN 70.58

The format of start_ts and end_ts is very flexible. Any date/time that can be parsed by dateutil.parser.parse() will work. If start_ts is not provided, readings start at the earliest available; if end_ts is not provided, readings continue through the latest available.

You can request that the sensor readings be averaged into time periods such as 1 hour or 1 day. For a full list of the possible time period codes, see DateOffset Objects. Here is an example for 1 hour averaging:

In [6]:
df = server.sensor_readings(
    sensors,
    start_ts = '2019-01-15 3:00 pm',
    end_ts = '2019-01-17 10:30 am',
    averaging = '1H'
)
df.head()
Out[6]:
outdoor_temp phil_hp_pwr_7470_temp
2019-01-15 15:00:00 33.006233 70.139767
2019-01-15 16:00:00 32.436133 70.168467
2019-01-15 17:00:00 31.556200 70.931667
2019-01-15 18:00:00 31.355200 70.568900
2019-01-15 19:00:00 31.684400 70.030333

The default is to label the time period with the time at left (beginning) edge of the interval. If instead you want the timestamp to fall at a different point in the interval, you can use the label_offset parameter to shift it. Here we mark the middle of the interval by using an offset of 30 minutes:

In [7]:
df = server.sensor_readings(
    sensors,
    start_ts = '2019-01-15 3:00 pm',
    end_ts = '2019-01-17 10:30 am',
    averaging = '1H',
    label_offset = '30min'
)
df.head()
Out[7]:
outdoor_temp phil_hp_pwr_7470_temp
2019-01-15 15:30:00 33.006233 70.139767
2019-01-15 16:30:00 32.436133 70.168467
2019-01-15 17:30:00 31.556200 70.931667
2019-01-15 18:30:00 31.355200 70.568900
2019-01-15 19:30:00 31.684400 70.030333

Retrieve Sensor Metadata

Sensor titles, units and other information can be retrieved for one or more sensors by using the sensors() method. Pass a Sensor ID or a list of Sensor IDs to the method:

In [8]:
server.sensors(['phil_hp_pwr_10187_temp', 'phil_hp_pwr_7470_temp'])
Out[8]:
[{'id': 234,
  'sensor_id': 'phil_hp_pwr_10187_temp',
  'title': 'New Outdoor Wireless Temp',
  'notes': 'No sensor notes available.',
  'is_calculated': False,
  'tran_calc_function': 'val - 0.76',
  'function_parameters': '',
  'calculation_order': 0,
  'formatting_function': '',
  'other_properties': '',
  'unit': 'deg F',
  'buildings': [{'bldg_id': 5, 'sensor_group': 'Weather', 'sort_order': 40}]},
 {'id': 243,
  'sensor_id': 'phil_hp_pwr_7470_temp',
  'title': 'House Temp',
  'notes': 'No sensor notes available.',
  'is_calculated': False,
  'tran_calc_function': 'val + 0.24',
  'function_parameters': '',
  'calculation_order': 0,
  'formatting_function': '',
  'other_properties': '',
  'unit': 'deg F',
  'buildings': [{'bldg_id': 5,
    'sensor_group': 'Space Conditions, Temperature',
    'sort_order': 10}]}]

The return value is a list of dictionaries, each dictionary describing a Sensor. The keys in the dictionary are the fields associated with the Sensor model in the BMON Django application. The buildings key in the dictionary gives a list of buildings that the Sensor is associated with. Further documentation of the fields is available Here; search for the class Sensor section of the code.

If you do not provide any IDs (either no parameters, or an empty list), information for all sensors will be returned. For example, server.sensors() will return a list of all sensors.

Building and Organization Information

Methods are available to return information about Buildings and Organizations in the BMON system. Pass one or a list of Building IDs to get Building information:

In [9]:
server.buildings([6, 13])
Out[9]:
[{'id': 6,
  'title': 'Rivendell Hall',
  'report_footer': '',
  'latitude': 64.864072,
  'longitude': -147.775014,
  'floor_area': None,
  'building_type': None,
  'outdoor_temp': '',
  'electric_ids': '',
  'fuel_ids': '',
  'indoor_temps': '',
  'indoor_lights': '',
  'co2_sensors': '',
  'timezone': 'US/Alaska',
  'schedule': 'M-F: 8a-5p',
  'timeline_annotations': '',
  'other_properties': '',
  'current_mode': '',
  'fuel_rate': None,
  'electric_rate': None,
  'sensors': [{'sensor_id': 'pafa_temp',
    'sensor_group': 'Weather',
    'sort_order': 10},
   {'sensor_id': 'pafa_wind', 'sensor_group': 'Weather', 'sort_order': 20},
   {'sensor_id': 'pioneer_temp', 'sensor_group': 'Weather', 'sort_order': 30},
   {'sensor_id': 'pioneer_wind', 'sensor_group': 'Weather', 'sort_order': 40},
   {'sensor_id': 'nortech_PV_power',
    'sensor_group': 'Solar PV',
    'sort_order': 10},
   {'sensor_id': '3c0034000447343233323032_temp',
    'sensor_group': 'IAQ Monitor #1',
    'sort_order': 10},
   {'sensor_id': '3c0034000447343233323032_rh',
    'sensor_group': 'IAQ Monitor #1',
    'sort_order': 20},
   {'sensor_id': '3c0034000447343233323032_co2',
    'sensor_group': 'IAQ Monitor #1',
    'sort_order': 30},
   {'sensor_id': '3c0034000447343233323032_light',
    'sensor_group': 'IAQ Monitor #1',
    'sort_order': 40}],
  'organizations': [[3, 'Other Commercial']]},
 {'id': 13,
  'title': 'Resolution Brewing',
  'report_footer': '',
  'latitude': 61.219541,
  'longitude': -149.822507,
  'floor_area': None,
  'building_type': None,
  'outdoor_temp': '',
  'electric_ids': '',
  'fuel_ids': '',
  'indoor_temps': '',
  'indoor_lights': '',
  'co2_sensors': '',
  'timezone': 'US/Alaska',
  'schedule': '',
  'timeline_annotations': '',
  'other_properties': '',
  'current_mode': '',
  'fuel_rate': None,
  'electric_rate': None,
  'sensors': [{'sensor_id': 'rbc_28.FF1A2D021400',
    'sensor_group': 'Temperature',
    'sort_order': 10},
   {'sensor_id': 'rbc_28.FFAB18021704',
    'sensor_group': 'Temperature',
    'sort_order': 20},
   {'sensor_id': 'rbc_28.FF41F5011703',
    'sensor_group': 'Temperature',
    'sort_order': 30},
   {'sensor_id': 'rbc_28.FF77A4011703',
    'sensor_group': 'Temperature',
    'sort_order': 40},
   {'sensor_id': 'rbc_28.FF1791011703',
    'sensor_group': 'Temperature',
    'sort_order': 50},
   {'sensor_id': 'rbc_28.FFEDCC011703',
    'sensor_group': 'Temperature',
    'sort_order': 60},
   {'sensor_id': 'PAMR_temp', 'sensor_group': 'Weather', 'sort_order': 10}],
  'organizations': [[3, 'Other Commercial']]}]

The sensors item gives a list of Sensors associated with the Building. The organizations item shows the organizations that the building is associated with. The fuel_rate and electric_rate items give the fuel and electric rate schedules, if present. Seach class FuelRate and class ElectricRate Here to see documentation of the rate structure fields. Further documentation of the other fields is available Here; search for the class Building section of the code.

If you do not provide any IDs (either no parameters, or an empty list), information for all buildings will be returned.

Here is the method for retrieving information about Organizations:

In [10]:
server.organizations([1, 2])
Out[10]:
[{'id': 1,
  'title': 'Homes',
  'sort_order': 10,
  'buildings': [[16, "Chris's House"],
   [8, "Dustin's Neighborhood"],
   [4, 'Homer Strawbale'],
   [14, "Ian's House"],
   [5, 'Kaluza House'],
   [3, 'Mitchell House'],
   [7, 'Rehfeldt Home'],
   [19, "Tyler's House"]]},
 {'id': 2,
  'title': 'THRHA',
  'sort_order': 20,
  'buildings': [[18, 'THRHA Angoon Housing'],
   [17, 'THRHA Juneau Warehouse'],
   [2, 'THRHA Kake Senior Center']]}]

The buildings key gives the list of buildings associated with the organization. Again, further documentation of the fields is available Here; search for the class Organization section of the code. server.organizations() will return information on all Organizations.

Storing New Sensor Readings

The bmondata package can be used to store new sensor readings into the BMON server's sensor reading database. Readings are stored using the Server object, and a list of new readings are provided. Here is an example:

In [13]:
import time
server.store_sensor_readings([
    (time.time(), '_testing', 18.8),
    (time.time(), '_hello', 24.3),
])
/home/tabb99/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
Out[13]:
2

The value returned is the number of readings that were successfully stored.

Utility Functions

A couple of utility functions are available to help process the data in some of the fields returned to describe Buildings. Currently, there are two functions: bmondata.csnl_to_list and bmondata.split_strip. Both functions split and clean a string and will be useful for turning a field containing Sensor IDs into a Python list. Here is the code for the two functions:

def csnl_to_list(csnl_str):
    """Converts a comma-separated and new-line separated string into
    a list, stripping whitespace from both ends of each item.  Does not
    return any zero-length strings.
    """
    s = csnl_str.replace('\n', ',')
    return [it.strip() for it in s.split(',') if it.strip()]

def split_strip(s, delim=','):
    """Converts a delimited string into a list, stripping whitespace 
    from both ends of each item.  Does not return any zero-length strings.
    'delim' is the delimiter.
    """
    return [it.strip() for it in s.split(',') if it.strip()]

Used to Save this Notebook as an HTML File and Upload to AWS S3

In [14]:
!jupyter nbconvert usage_examples.ipynb --to html
import boto3
s3 = boto3.resource('s3')
s3.meta.client.upload_file(
    'usage_examples.html', 
    'web.analysisnorth.com', 
    'bmondata/usage_examples.html',
    ExtraArgs={'ContentType': 'text/html'}
)
[NbConvertApp] Converting notebook usage_examples.ipynb to html
[NbConvertApp] Writing 310248 bytes to usage_examples.html