Deep Dive Into Scriptr.io-Powered IoT Dashboards and Visualizations

Scriptr.io has an expansive data visualization module powered by Google Charts. It makes producing visualizations incredibly convenient, and, like everything else on scriptr.io, all of your work is instantly deployed and ready to use as soon as its made. I’m going to walk you through how I made this demo of scriptr.io charts.

I used traffic data collected by smart sensors in Aarhus, Denmark that monitored the number of vehicles that came by every 5 minutes and their average speed to make the visualizations. All five charts in the demo use the exact same data. This is one of the big advantages of the editor – many different chart types can be made from a single data set, with little effort involved.

Formatting the Data

The first step is to parse the data into a format usable by the Charts module. The raw data was in the form of a csv, and spanned multiple months of collecting by the sensors. The data was too much to be handled in a single http request, so I grabbed just a day’s worth of data (click here to see it) and hosted it on scriptr.io using the Files module. Simply click on the file button in the add new script dropdown in the bottom left to host whatever you need!

 

The Google Charts module, for the most part, takes data in the form of a two dimensional array. This means an array containing arrays. It is best thought of as a table containing rows where each array is a row, which is very similar to a csv, where the different rows are separated by newlines. This makes formatting the data to make Google Charts simple. I started by using the csv parsing function from this Stack Overflow post to convert the raw data to a form usable by Javascript. All the regular expressions are a bit overkill compared to a simple .split(‘,’), but it helps to have all the edge cases covered. The function makes the csv into a two dimensional array, like the below:

As you can see, the first array contains the header for each column, and the rest of the arrays are data. This is the exact format the Google Chart data needs to be in, but there’s too much extraneous data here for the charts I want to make. In order to trim it down, I’ll iterate through the array and populate a new one with the values I want, as below:

var data = parseCSV(res.body);

var formattedData = [[
  'Time',
  'Average\nSpeed',
  '# of\nvehicles'
]];

var labels;
data.forEach(function(row, ind){
  if(row.length < 9){
    //avoid incomplete rows
    return;
  }
  if (ind === 0){
    labels = row;
  } else {
    formattedData.push([
      dateFormatter(row[labels.indexOf('TIMESTAMP')]),
      Number(row[labels.indexOf('avgSpeed')]),
      Number(row[labels.indexOf('vehicleCount')])
    ]);
  }
});

function dateFormatter(date) {
  var time = date.split('T')[1];
  var parts = time.split(':');
  return parts[0] + ':' + parts[1];
}

As you can see in the above, I’m able to grab the right data value from each row by checking the index of the value I want in the first row. I also include an array with the labels I want the chart to have as the first element of the new list I’m populating. It’s important to make sure the right data goes in each column, otherwise it will get mixed up when creating the visualization.

The data (click here to see it) is ready to be used to create a chart. For example, it could produce the following column chart:

 

The chart is looking good, but there’s too many data points. In order to reduce the amount of noise and produce clearer columns, we can average together some of the data. This can be done using the native reduce function as follows:

var chartData = [[
  'Time',
  'Average\nSpeed',
  '# of\nvehicles'
]];

var counter = 0;

uncompressedData.reduce(function(a,b){
  if(counter % 3){
    a[0] = b[0];
  }
  a[1] = a[1] + b[1];
  a[2] = a[2] + b[2];
  if(counter % 5 === 0 && counter !== 0){
    a[1] = a[1]/5;
    chartData.push(a);
    counter++;
    return b;
  } else {
    counter++;
    return a;
  }
});

There’s a lot going on in the above function, so let’s break it down. The goal is to combine every five data points into a single datum. Reduce will iterate through the list and call the function given to it on each element with two parameters: the return value of the previous function call (a above), and the current element of the list (b above). In order to stay aware of which position in the list I’m at, I created a counter variable that gets incremented in the function.

No matter where in the list the iterator is, the two data sets need to be combined, so I add the values from b to a outside of all the if blocks. Most of the time that’s all that needs to be done; the catch-all else simply increments the counter and returns a. Every five elements, however, the processor needs to store the current average and start working on the next set. Therefore, everytime the counter is divisible by 5, the data gets pushed to the list, and b is returned instead of a to reset. You’ll also notice that a[1] gets averaged, while a[2] does not. This is because a[1] refers to the speed of the cars, which can be meaningfully averaged, while a[2] is the number of cars, which will make sense if simply added together. If we averaged a[2], then it would represent the average number of cars every five minutes, which doesn’t mean much!

The times also need to be combined, but this is a bit more difficult because they are strings. Fortunately, the times are all 5 minute intervals, which means that if they are averaged they’ll always equal the middle value. Therefore, whenever the counter is divisible by 3, the timestamp from b gets written to a.

After compressing the data, our column chart looks like this:

Much clearer than before! Now that we’ve covered how the data needs to be formatted, let’s go into using the scriptr.io charts editor.

Using The Chart Editor

The editor is very simple to use; most of the work is just getting the data into the right format. To begin, click on the dropdown on the bottom left, and select Google Chart. A new tab in the IDE will open, and you’ll see the following:

A chart with demo data gets instantly loaded on the right. This pane will provide a preview of the chart as you develop it, and will react to changes made in the editor. On the top left is a dropdown that allows you to select one of your scripts, at which point the chart will be generated using the script’s return value. Once I select my data script, I see this:

Boom! A chart with my data is instantly created. This chart looks decent, but I’d like to change the colors and remove the default titles on the left and top. To do this, I just click on the customize tab on the top left, where I’ll see this view:

The section on the left has several options for styling the chart, including changing the color, font, and axis labels. These options are great for making the data look good once you’ve settled on a visualization. In order to select different visualization types, click the Charts tab on the top left, where you’ll see this:

All the different types of visualizations are available for selection on the left. to change the chart type, just click on a different visualization, and it will be instantly rendered! The format of the data I made in this tutorial is valid for line, area, column, bar and scatter charts. The other charts take different inputs; for example, the pie chart requires a two dimensional array with only two columns. To see samples of data inputs for different chart types, click on the samples tab on the right side of the editor.

Using the Charts

Once the charts are created, embedding them on your website is very simple. Just click on the tab on the right that says embed URL, and you’ll be provided with an iframe ready to be copy-pasted, as well as a standalone link if you want to do the embedding work yourself. You can also click on the HTML tab on the right to get an HTML document with all the chart code contained in it.

You can even use scriptr.io to host your website; I used it to host the demo I linked to above (click here to see the HTML). Simply click on the file button in the add new script dropdown, and select HTML. Then you can paste in your HTML and it will be instantly ready to serve!

That’s it! As you can see scriptr.io is very versatile; I was able to host my data, format it, generate a visualization and serve it up in an HTML document all in the same place!