Datalogger example using Sense Hat, InfluxDB and Grafana




Introduction

In this article we will make a simple datalogger application running on the Raspberry Pi. It is compatible with Raspberry Pi Zero W, Pi 2 and Pi 3.
A data logger records data from something; usually some form of sensors but could also be I/O ports, switches and so forth. An important feature of any datalogger is the ability to timestamp each sample or measurement. For simplicity, our datalogger example will use the Raspberry Sense Hat add-on board and the companion easy to use Sense Hat python library.
The Sense HAT features an 8×8 RGB LED matrix, a mini joystick and the following sensors :

  • Gyroscope
  • Accelerometer
  • Magnetometer
  • Temperature
  • Humidity
  • Barometric pressure
Raspberry Pi Sense Hat

We will logg data from the Sense Hat temperature, humidity and the barometric pressure sensors.

Influxdb Database:
Our simple datalogger needs some kind of place to organize and store the datalogger measurement samples. We have decided to use the Influxdb database to organize and store our data. InfluxDB is a Time Series Database built from the ground up to handle high write & query loads. InfluxDB is a custom high performance datastore written specifically for timestamped data, including DevOps monitoring, application metrics, IoT sensor data, & real-time analytics. It is possible to conserve space on your machine by configuring InfluxDB to keep data for a defined length of time, automatically expiring and deleting any unwanted data from the system. InfluxDB also offers a SQL-like query language for interacting with data. In our simple datalogger example, the data is stored on the SD card; hence make sure you use a good quality type of card as it will be written to many times. You could store the data another place as in the cloud or on an external hard drive or usb stick. That will be discussed in a future article.

Grafana; Presenting the datalogger results:
The data stored by our datalogger could also been written to a file as comma separated data and then import that file in excel and made some nice graphs and presentations. In this simple datalogger example, we want to try out the Grafana application. Grafana is an open-source, general purpose dashboard and graph composer, which runs as a web application.

What is a datalogger?

Step by step Tutorial

1. Install the Influx database on the Raspberry Pi

a. Installing the Influx database is pretty simple. Please follow the instructions in this article.

b. Do not forget to start the Influxdb service on the Raspberry Pi by running this bash command in the terminal:

sudo service influxdb start

c. With InfluxDB installed, you are ready to start doing some awesome things. We will use the influx command line interface (CLI), which is included in all InfluxDB packages and is a lightweight and simple way to interact with the database. The CLI communicates with InfluxDB directly by making requests to the InfluxDB HTTP API over port 8086 by default.
The influx command should be available via the command line. Executing influx in the Raspberry terminal will start the CLI and automatically connect to the local InfluxDB instance (assuming you have already started the server with service influxdb start or by running influxd directly). The output should look like this:

$ influx -precision rfc3339
Connected to http://localhost:8086 version 1.2.x
InfluxDB shell 1.2.x
>
The command line is now ready to take input in the form of the Influx Query Language (a.k.a InfluxQL) statements. To exit the InfluxQL shell, type exit and hit return. A fresh install of InfluxDB has no databases (apart from the system _internal), so creating one is our first task. You can create a database with the CREATE DATABASE InfluxQL statement, where is the name of the database you wish to create. Names of databases can contain any unicode character as long as the string is double-quoted. Names can also be left unquoted if they contain only ASCII letters, digits, or underscores and do not begin with a digit. Throughout this guide, we will use the database name mydb:
> CREATE DATABASE logger_db
>
Now that the logger_db database is created, we’ll use the SHOW DATABASES statement to display all existing databases:
> SHOW DATABASES
name: databases
---------------
name
_internal
logger_db

>

We have created a database and it is ready to be used in our datalogger application. Exit the Influx shell by typing the command exit.

2. Install the Grafana server on the Raspberry Pi

a. Installing the Grafana realtime graph dashboard is pretty simple. Please follow the instructions in this article.
b. Please remember to start the Grafana server:

sudo service grafana-server start
This will start the grafana-server process as the grafana user, which was created during the package installation. The default HTTP port is 3000 and default user and psw is admin.

To configure the Grafana server to start at boot time:
sudo update-rc.d grafana-server defaults

3. Prepare and Write the datalogger python program

Simple python datalogger program:
A few command arguments have beeen added to make it possible for a user to specify the database name used, to set a measurement name and a run number. The measurement name and the run tag could be used to set different sessions during testing etc. You will need to install the InfluxDB Python module maintained here on github.

Install the python package :
sudo pip install influxdb

On Debian/Ubuntu, you can install it with this command:

sudo apt-get install python-influxdb


# -*- coding: utf-8 -*-
import argparse
import time
import datetime
import sys
from influxdb import InfluxDBClient
from sense_hat import SenseHat

sense=SenseHat()
 
# Set required InfluxDB parameters.
# (this could be added to the program args instead of beeing hard coded...)
host = "localhost" #Could also set local ip address
port = 8086
user = "root"
password = "root"
 
# Sample period (s).
# How frequently we will write sensor data from SenseHat to the database.
sampling_period = 5

def get_args():
    '''This function parses and returns arguments passed in'''
    # Assign description to the help doc
    parser = argparse.ArgumentParser(description='Program writes measurements data from SenseHat to specified influx db.')
    # Add arguments
    parser.add_argument(
        '-db','--database', type=str, help='Database name', required=True)
    parser.add_argument(
        '-sn','--session', type=str, help='Session', required=True)
    now = datetime.datetime.now()
    parser.add_argument(
        '-rn','--run', type=str, help='Run number', required=False,default=now.strftime("%Y%m%d%H%M"))
    
    # Array of all arguments passed to script
    args=parser.parse_args()
    # Assign args to variables
    dbname=args.database
    runNo=args.run
    session=args.session
    return dbname, session,runNo
    
def get_data_points():
    # Get the three measurement values from the SenseHat sensors
    temperature = sense.get_temperature()
    pressure = sense.get_pressure()
    humidity = sense.get_humidity()
    # Get a local timestamp
    timestamp=datetime.datetime.utcnow().isoformat()
    print ("{0} {1} Temperature: {2}{3}C Pressure: {4}mb Humidity: {5}%" .format(session,runNo,
    round(temperature,1),u'u00b0'.encode('utf8'),
    round(pressure,3),round(humidity,1)))
    
    # Create Influxdb datapoints (using lineprotocol as of Influxdb >1.1)
    datapoints = [
            {
                "measurement": session,
                "tags": {"runNum": runNo,
                },
                "time": timestamp,
                "fields": {
                    "temperaturevalue":temperature,"pressurevalue":pressure,"humidityvalue":humidity
                    }
                }
            ]
    return datapoints

# Match return values from get_arguments()
# and assign to their respective variables
dbname, session, runNo =get_args()   
print "Session: ", session
print "Run No: ", runNo
print "DB name: ", dbname

# Initialize the Influxdb client
client = InfluxDBClient(host, port, user, password, dbname)
        
try:
     while True:
        # Write datapoints to InfluxDB
        datapoints=get_data_points()
        bResult=client.write_points(datapoints)
        print("Write points {0} Bresult:{1}".format(datapoints,bResult))
        
        sense.show_message("OK")
            
        # Wait for next sample
        time.sleep(sampling_period)
        
        # Run until keyboard ctrl-c
except KeyboardInterrupt:
    print ("Program stopped by keyboard interrupt [CTRL_C] by user. ")
    

Save the script as ‘dataloggersensehat.py’ and run using python dataloggersensehat.py. To run the script in the background and indefinately as a ssh user use nohup python dataloggersensehat.py &.

Let us understand key points of the code listing above:
We import the InfluxDBClient class at the start of the program code.
We define two constants, one for the InfluxDB Host, which is the Public IP of your Compute Engine instance. Remember to replace it in the code.
The next statement is important. We initialize the InfluxDBClient with the hostname, port (8086 for the API) and provide it with the database name.
Now that we have the client object, we can then either query or insert database records.
To write data to InfluxDB, we use the `write_points` method that takes in a list of measurement points that we want to write to the database. Each measurement point, and which you should be familiar with now, specifies the measure, one or more tags (test1 and runNo), and the three field values. Also notice that we are passing in the timestamp so that we record the timestamp from the board that it was recorded at. We use the `datetime` package in python to help us get the Unix timestamp that we need to send to InfluxDB.
The result of the `write_points` method is of boolean type. If the operation is successful, it returns back a `true`.

4. Start the datalogger program

For simplicity we will run it in the python shell:

python dataloggersensehat.py -db=databasename -sn=test1

If you have forgotten to setup a valid Influxdb database you will get an error.

5. Setup Grafana dashboard and graphs for our datalogger

Unlike InfluxDB, Grafana doesn’t enable it’s service, so do this to enable at boot and start the service now:

sudo systemctl enable grafana-server
sudo systemctl start grafana-server

Navigating your web browser to port 3000, you should be presented with the Grafana web GUI.
http://192.168.1.101:3000 or http://localhost:3000
Grafana login html image
Grafana has plenty of powerful querying tools to make lots of pretty and informative graphs. If you’re just starting out, we recommend taking a look at Grafana well made documentation for Basic Concept and Getting Started guides.

The first thing you will need to do is add the ‘logger’ database as a Datasource. Navigate to Datasource->Add New and fill in as below:

Grafana data source setup
Fill in the database type (Influxdb) and a convinient name, then the name of the database we created “logger_db”. User and psw: “root” and “root”. Click the “Add” button when finished. Grafana will start testing the database and flag error or success on connection.

Next step is to setup a dashboard for our datalogger:
1
Grafana create new dashboard

2.
Grafana edit new graph

3. Enter the name you specified for the datalogger_db and click on add query.

Grafana add database query

4. Set up the measurements metrics as below: The graph will be drawn as soon as the metrics is edited.

Grafana mertics configuration

5.

Here is the final result of the datalogger dashboard:

Sense Hat datalogger grafana dashboard

Join the Discussion

Your email address will not be published. Required fields are marked *

arrow