Friday, April 11, 2014

Getting started with the DigiX (Part 1 of 2)

This is the first part of my project to integrate the Digistump DigiX board with the BeagleBone. In this post, I'll provide a review of the DigiX and some of the good and less good points about working with it. The next section will describe in more detail the software and process I used to build the project.

First, for your multimedia viewing pleasure,  here is a quick video tour of this fairly simple project:


Introducing the DigiX

The DigiX is produced by DigiStump which was founded in 2012 by Mr. Erik Kettenburg in the Seattle area. I first became aware of them when I backed his successful Kickstarter for the DigiSpark which is a super-small $9 Arduino development board  - I will include them in a later part of my monitoring system. A few months after the DigiSpark shipped, I saw the Kickstarter come up for the DigiX and decided to go for it.

The specs on the DigiX are, frankly over the top! This is the Monster Killer Arduino of all Time (at least for now) - check it out:


In form factor, it looks a bit similar to the Arduino Due, but it has a double row of IO pins down the one side whereas the Due only has a single row. Here is an image of the full pinouts:

This gives you some idea of what a beast this thing is!! As well it has BOTH on-board wifi and mesh networking via the popular, low-cost nRF24L01+ wireless module and and a micro SD card slot. All this for only $69.95 USD!

 That being said, given that the BeagleBone Black is only $44.95 and the RaspBerry Pi is only $39.95 (prices from SparkFun) and both of them run full up Linux. Is it worth the extra money to have the DigiX? Here are a few thoughts on pros and cons...

Pros

  1. Rather than having to wrestle with the complexity of managing GPIO in the Linux environment, the DigiX supports the usual Arduino IDE and it's familiar interface. If you are used to stock Arduinos, then there is no ramp up.
  2. The onboard wifi work as advertised and is a breeze to configure according to the DigiX Wiki article (check it out here).
  3. There are more IO pins than any reasonable person could ever use and there is loads of program memory space (524,288 bytes).
  4. There is a good range of sample programs that exercise the functionality of the board and they all seem to compile and work right away. I was amazed that I had a simple web server up and running in about 15 minutes!
  5. I haven't tried the mesh networking yet, but I am assuming it works and it's awesome to have it on the same board.
  6. This board is a 3.3 VDC board (like the other newer Arduinos), but Digistump sells a level shifting shield so you can reuse older 5 VDC shield, which is very useful.
  7. There is a reasonable amount of documentation and active forums for help.

Cons


The main draw back is is it not quite a stock Arduino and a couple of compromises have been made to keep the board affordable. Let's look at two things I have found...

Wifi

The wifi module on the DigiX is not the same as on the Arduino Yun. The Yun uses the AR9331 while the DigiX uses an Atheros Silicon based embedded UART/WiFi module(source here). This means the DigiFi wifi library is mostly compatible with the Arduino Ethernet library, but not 100%.

Consequently, for the project I just completed, I wanted to use Socket.io or websockets to exchange information between my Beaglebone and the DigiX. I tried various sockets libraries and just could find a way to make them work with the DigiX! I posted a thread to the DigiX forums, but, since they are somewhat less active than the Arduino boards, I so far haven't found a solution. You will see in my upcoming project that the solution I found works, but is definitely "suboptimal" from a design standpoint.


USB Serial and programming

Mr. Kettenburg provides a good technical description of what happens with USB serial on the DigiX and Due boards here. You can read the detailed explanation, but what it boils down to is that the Arduino Due has two USB ports - one for programming and one for native USB - while the DigiX has only one USB port which has to work for programming and USB. Unfortunately, on the DigiX it seems to sometimes be a coin toss as to which is going to be available when you boot up the board, which caused me some headaches at first.

When I was first using the board, it would do something like this when doing programming:
  1. The Arduino IDE is set on COM16 and you hit the upload button in the IDE.
  2. The program compiles and if there are no errors, then it goes to upload and says "The COM port isn't available".
  3. Restart the DigiX and if you are lucky it comes up with COM15.
  4. While on COM15, hit the upload button and it goes through and programs the board.
  5. However when you go to the IDE to select the serial port for the serial monitor, it is back to COM16!
This could be a huge hassle sometimes requiring multiple restarts of the board and plugging and unplugging the USB until the right port came up. As well, the IDE serial monitor would sometimes not fire up and that required more restarts.

At the moment, I seem to have solved this and it is working smoothly. What I did was follow this advice from the DigiX wiki:

If the COM port isn't showing in the Arduino IDE - unplug and replug the board. If that doesn't work - while plugged in, hold down the erase button on the board for a moment and then unplug and replug - you may then have to select it from the com port menu as it may be on a different port - but it is a sure way to get it to respond even if your sketch crashed the USB stack. 

At the same time as doing this, I moved it to another USB port on my system and Windows reloaded the drivers. It works normally now, but it leaves me concerned it will stop working again in the future. I guess I shouldn't complain because it is a bit of an RTFM situation, but perhaps future versions of the board could have the two port solution like the Due or maybe a jumper block to put the USB port into one mode or the other to avoid this confusion?

Summary


Despite the minor hassles I would still say the DigiX is a great board and worth the money. It gives you loads of room to grow and build very sophisticated networked projects. If you just want something relatively simple to take some digital/analog inputs and then do some digital.analog outputs, then this board may be too much and you should look at some of the simpler Arduinos on the market.

As the "Internet of Things" movement develops, it will be de rigueur to be able to to hook your embedded project to the Internet and to other devices via either mesh or Bluetooth Low Energy. The DigiX has you covered with wifi and mesh on board and while it is more expensive than some boards, you save by having all the connectivity integrated and not having to get separate shields and so on.

Nest up, the software the drives this project!








Wednesday, March 26, 2014

BeagleBone Node.js project updated with Socket.io

After I finished my last project, I realized I wasn't entirely satisfied with it. While it basically functioned, relying on it doing a whole page refresh every so often just didn't seem "elegant". As well, when you changed the settings in one browser, they didn't change in other browsers, which, again, didn't seem very slick.

I figured I had to learn to use the Socket.io library anyway to eventually communicated with my Arduino DigiX, so why not update the BeagleBone project to "socketize" it? Naturally, this took a bit of a learning curve, but I like the results.

The project hardware setup is exactly the same as my previous post - using the BeagleBone and the BlueLine Innovations PowerCost Monitor. I have also gotten smart and finally signed up for GitHub and here is a link to the complete source code:


Finally, I have done a YouTube overview of the functionality of the project:



In case it isn't clear in the video, here is what the web page looks like:



The software uses all the libraries from the previous version (Request, Flot, jQuery etc.). The only addition is Socket.io, however, I ended up practically rewriting the application from scratch to accommodate the change!

Server side - App.js

 

Some of the elements of App.js from the last version of this project are the same, so I won't go into detail on them such as using the Request library to do the screen scrape and then popping that into the TingoDB/Mongoose database. However, it seems like Socket.io is only "semi compatible" with Express and you need to make several changes from the default Express setup.

Again, there are many, many good tutorials out there by folks who understand this at a deeper level. Below is just my description of how I got this to work. Here are a couple of good Socket.io tutorials I found along the way that may be helpful:



Here is the section of App.js setting up Socket.io, which seems to have to come later in the App.js file than the usual "require" statements at the beginning:

var io = require('socket.io').listen(server);

app.set('port', process.env.PORT || 3008);

io.set('log level', 1); 


In my last version of this project the "app.get" and "app.post" functions did most of the heavy lifting of rendering information over to web page. Now with Socket.io, the "app.post" section is completely gone since the button pushing is now handled by Socket.io rather than the traditional POST method.

The main action is now all within this one big Socket.io loop:

io.sockets.on('connection', function(socket) {
    console.log('A new user connected!');
    Readings.find({}, {}, {
        sort: {
            'time': -1
        },
        limit: process.env.SAMPLES
    }, function(err, readings) {
        socket.broadcast.emit('readingsData', readings);
        socket.broadcast.emit('sampleSetting', process.env.SAMPLES);
        socket.broadcast.emit('refreshSetting', process.env.REFRESHINT);
        console.log('Initial data over to browser.');
    });
    socket.on('sampleInput', function(sampleInputSetting) {
        console.log('setting data = ' + sampleInputSetting);
        process.env.SAMPLES = sampleInputSetting;
        socket.broadcast.emit('sampleSetting', sampleInputSetting);
        console.log('Sending sample rate back out');
    });
    socket.on('refreshInput', function(refreshInputSetting) {
        console.log('setting data = ' + refreshInputSetting);
        process.env.REFRESHINT = refreshInputSetting;
        socket.broadcast.emit('refreshSetting', refreshInputSetting);
        console.log('Sending refresh rate back out');
        Readings.find({}, {}, {
            sort: {
                'time': -1
            },
            limit: process.env.SAMPLES
        }, function(err, readings) {
            socket.broadcast.emit('readingsData', readings);
            socket.emit('readingsData', readings);
        });
    });
    setInterval(function() {
        Readings.find({}, {}, {
            sort: {
                'time': -1
            },
            limit: process.env.SAMPLES
        }, function(err, readings) {
            socket.broadcast.emit('readingsData', readings);
            console.log(process.env.SAMPLES + ' readings sent over');
        });
    }, (process.env.REFRESHINT * 1000));
});


At it's heart, when a browser connects and establishes a socket, this sends over an initial chunk of data and then kicks off the setInterval loop. This broadcasts out a chunk of database data to all connected browsers according to the refresh interval environment variable (process.env.REFRESHINT).

When the "submit" button is hit on the website, this opens the the refreshInput and sampleInput sockets and they in turn send over the revised values to the other browsers connected to the server.

Overall, this works fine, but really looks ugly to me and I am sure I'm doing something wrong! I have repeated the database query three times. I think this should be put into a separate function, but then this isn't the way Node.js works. I looked at implementing the Promises framework (more here), but I got lost in it and I decided to just close off where I am. Promises will need to wait until later.

Client side - Main.js


Since the client-side JavaScript had gotten rather complicated, I decided to put it off into a  separate file which I called "main.js" which lives in the public/javascripts folder of the project. This is referenced in the "Layout.jade" file (more about Jade in a minute).

At the top is this to load the Socket.io:

var socket = io.connect();

 Then we have the jQuery function started up and the logic for the Settings button:

$(document).ready(function() {
    console.log('Doc loaded');
    $('#submitBtn').click(function() {
        var sampleInput = document.getElementById("dataSampleInput").value;
        console.log(sampleInput);
        var refreshInput = document.getElementById("refreshTimeInput").value;
        console.log(refreshInput);
        socket.emit('sampleInput', sampleInput);
        socket.emit('refreshInput', refreshInput);
    });


This is some jQuery that activates when the button is pushed, and picks up the values entered into the two fields with Javascript and then the pushes the values back to the server with socket.emit. You can see the "sampleInput" and "refreshInput" values here that end up over at the server side.

Then there are two further sockets that take the settings value back from the server when they are rebroadcast out:

    socket.on('sampleSetting', function(serverSampleSetting) {
        console.log('Received new sample rate ' + serverSampleSetting);
        $('#dataSampleInput').val(serverSampleSetting);
    });
    socket.on('refreshSetting', function(serverRefreshSetting) {
        console.log('Received new refresh rate ' + serverRefreshSetting);
        $('#refreshTimeInput').val(serverRefreshSetting);
    });


This uses jQuery to write these values back to the fields so all browsers show the same settings.

The socket.on 'readingsData' is what receives the data object over from the server when it is sent over according to the refresh interval. There is then basically the same logic as I used in the last project to break down the data object into two separate arrays and then feed that into Flot to generate the graph. Just check out my previous post on how that works.

The "Server connected" and "Server disconnected" status line is an idea I got from James Doyle and this great tutorial on YouTube (GitHub here). It is very simple and works like this. When the main readingsData socket starts up a couple of lines of jQuery write to a DIV called "logger":

        $('#logger').text('Server connected.');
        $('#logger').css('color', 'green');


When the socket disconnects, it changes the message:

        socket.on('disconnect', function() {
            // visually disconnect
            $('#logger').text('Server disconnected.');
            $('#logger').css('color', 'red');

Mr. Doyle used plain JavaScript for his, but I found for whatever reason this didn't work across all browsers, but jQuery does. The only limitation with this is that it will go to "disconnected" if the server is offline and then back to "connected" if it comes back on fairly quickly (let's say 15 min or so), but if the server is down for too long it won't auto-reconnect and you need to reload the web page.  It also won't go to a "disconnect" status if the Ethernet cable is pulled out of the BeagleBone, for instance.

Jade files - Layout.jade and Index.jade

If you check out these files on GitHub, you will see they are much simpler than they were the first time around. I moved the bulk of the scripting into "Main.js" and the original form I used for submission of the settings has been much simplified.

Hopefully this is interesting to all and sundry! Next up I promise to get on with working with the DigiX and maybe I will even manage a flashing LED using Socket.io!!

Saturday, March 1, 2014

Creating a Node.js application on the BeagleBone


Well, as usual, it has taken a log time to get to another post! Hopefully it was worth the wait.

WARNING 1 - I am very much of a beginner with Node.js! Please Google before asking me any questions about the programming on this project. The code in this project more or less works and demonstrates basic concepts, but totally lacks the kind of error handling and security that production code should always incorporate.

WARNING 2 - I am also NOT an expert on the BeagleBone and this project really does not use any of the GPIO features of the board (in fact, the Node.js could be run on Windows as easily as the BB). Anyone wanting to understand the board should first look at Derek Molloy's fantastic blog and YouTube videos!

Motivation and background

 

I first pre-ordered my BeagleBone way back in 2011 when the product was first announced. At the time I was interested in the BeagleBoard and the RaspBerry Pi wasn't yet released and it seemed to have good specs for the cost. When I got it, I booted it up and played around with it a bit, but there wasn't a ton of documentation available yet and I was very busy with work. Lately I've been coming back around to my embedded electronics work and when I pulled it out I found that now there was tons of new material around on the device. Just to be clear, mine is the original "white" BeagleBone not the newer "Black" model.

After updating the Angstrom image and booting it up, I saw that the BB provides it's own on-board IDE which is a locally version of normally cloud-based Cloud9 (more on Cloud9) and runs this new and mysterious thing called "Node.js". I had a small amount of JavaScript knowledge, so I bought a text on Node.js (Sams Tech Yourself Node.js) so I could start puzzling it out.

Rather than plunge into the depths of doing GPIO with the BB right away and introducing too many variables, I decided to basically use the BB as a web server and to come up with a simple goal so I could  try to learn some Node.js and prepare for something more substantial later on. I decided something that would be "simple" to do would be figue out how to screen scrape my BlueLine Innovations Wifi PowerCost Monitor (more details here) to provide a visual indication of my power usage and the outside temp. Since I am already using Plot

Gear and setup

 

No elaborate wiring diagrams or anything here. The equipmentused is:
  • BeagleBone (White, running Angstrom Linux image)
  • BlueLine Innovations PowerCost Monitor™ WiFi Bridge
  • BlueLine Innovations Energy Monitor outside meter sensor

Here are the BeagleBone and the PowerCost Monitor:


Here is the outdoor portion strapped onto my utility power meter. Seems a bit funky, but it has been out there for a few years now.


In this project, the Outdoor monitor communicates with the Wifi gateway over a proprietary protocol and then the Gateway serves up a small web page which looks like this:


The Node.js running on the BeagleBone captures the webpage at regular intervals, stores the temperature and usage data in a Tingo database (more on that later) and then serves up a web page that shows a table of recent results with a graph. Here is the layout:


The Web Page 


This is the  resulting power & temp usage monitor web page. It shows:
  • a graph at the top of power use and temp over time
  • a listing of the data points with their timestamps 
  • a settings area where the number of data samples and the interval between samples can be set.



The code

 

This project makes use of many different Node.js and JavaScript library. Specifically:

  • Node.js - asynchronous, non-blocking, server-side JavaScript
  • Express - web framework for Node.js
  • Jade - HTML templating engine for Node.js
  • TingoDB - This is a "noSQL" database alternative to MongoDB. (I couldn't get Mongo to run on the BeagleBoard Angstrom release, but TingoDB works fine - probably because it is written entirely in JavaScript.)
  • Tungus - This is a driver to allow use of the Mongoose implements mongoose.js driver API.
  • Mongoose - object modelling tool for Mongo (in this case being used with TingoDB via Tungus - say that five times fast!)
  • Request - A simple HTTP client for Node.js
  • Flot - A charting package for jQuery
  • jQuery - JavaScript library for HTML document manipulation
Getting all these elements to work together was a  fair challenge given my complete ignorance at the start of this project!

I won't bother with a complete step-by-step build process here. There are many number of great tutorials on the web that walk through creation of Node.js/Express/Jade websites. Christopher Buecheler has a great one over here, for instance. Suffice it to say, first install Node.js, then use npm to install all the dependencies. If you get error messages with the npm installation, Google them and you will pretty much always find an answer. After that, I used Express to create the basic shell of the website.

This is a comparatively simple, single page Node.js project. Besides the libraries above, there are 4 critical files:
  • app.js
  • layout.jade
  • index.jade
  • style.css

The complete source code files are at the end of the article and I will highlight critical snippets of code to help understand how it all hangs together. Other code in the files (especially app.js) are set up by Express when you create the website.

Once I had the basic Express/Jade site built and included all the dependencies, I ended up with a directory structure that looked like:


App.js notes:

 

App.js is the server code that runs the whole thing. It also initializes the database, sets environment variables, uses the Request library to extract data from the Wifi PowerCost Monitor website and store that in the TingoDB database. Finally, when the gets a "GET" request from a browser, it renders the data on to the Jade engine.

Here are the sections that manage the database. This code links to the dependencies:

var tungus = require('tungus');
var Engine = require('tingodb');
var mongoose = require('mongoose');

This code initializes Mongoose with Tingo and names the database:

var db = mongoose.connect('tingodb://readingsdb');


Then Mongoose connects and prints to the console:

mongoose.connect('tingodb://readingsdb', function (err){
    if (!err) {
        console.log('connected to databse');
    } else {
        throw err;
    }
});


Then we set up the schema for the database:

var Schema = mongoose.Schema;
var ObjectId = Schema.ObjectId;
var ReadingsSchema = new Schema({
  time: { type : Date, default: Date.now },
    usage2: Number,
    temp2: Number
    });
var Readings = mongoose.model('Readings', ReadingsSchema);


I decided to set the refresh rate (how often the monitor browser reloads) and the interval time (how long between data samples) via environment variables, which I do here:


process.env['SAMPLES'] = '15';

process.env['REFRESHINT'] = '120';


The screen scrape and data extraction is done by the getData function which uses the Request library to load the PowerCost Monitor webpage into a variable, then uses simple JavaScript string commands to isolate the two values we want. It loads the values into the database created above and used a JavaScript setInterval function to repeat the process however often has been set for the refresh rate.

One minor note is I find I have to subract 2 from the temperature I get from the PowerCost Wifi Monitor to get the correct temperature. It just seems to be a slight calibration issue.


function getData (){
    request({uri: "http://192.168.1.33/pcmconfig.htm"}, function (error, response, body){
        if (!error && response.statusCode == 200) {
            var n = body.search("Present Demand");
           usage = body.substr((n+44),5);
           usage = usage.trim();
           console.log(usage);
           var n2 = body.search("Sensor Temp");
           temp = body.substr((n2+40),3);
           temp = temp.trim();
           temp = temp - 2;
           console.log(temp);
           var readingInfo = new Readings({
               temp2: temp,
               usage2: usage
            });
        readingInfo.save(function(err, readingInfo){
              if (err) return console.error(err);
            console.dir(readingInfo);
            });
        setTimeout(getData,(process.env.REFRESHINT * 1000));
    }});
}
//run the above function
getData();



Now we have our data and stored it into a database so we need to render it over to Jade. When a GET request is received from a browser, this function is run:

app.get('/', function(req, res){
  Readings.find({}, {}, { sort: { 'time' : -1}, limit: process.env.SAMPLES }, function(err, readings) {
    if (err) return console.error(err);
      res.render('index', 
        { title: 'Power Usage and Temp from PowerCost Monitor',
          refreshRate: process.env.REFRESHINT,
          sampleNum: process.env.SAMPLES,
          readings: readings
              });
           }
          );
       }
    );


This renders the title, the environment variables and an object with the actual temp & power readings over to Jade via the "res.render" function.


If the user wants to change the sample interval or the refresh rate, they enter them in the the "Settings" fields and when you hit "Submit" it triggers the app.post function, whichwill set the refresh rate and sample number environment variables and then re-render the page:

app.post('/', function(req, res){
    process.env['SAMPLES'] = req.param("samples");
    process.env['REFRESHINT'] = req.param("interval");
    console.log('Samples is set to:  ' + process.env.SAMPLES);
    console.log('Refresh is set to:  ' + process.env.REFRESHINT);    
      Readings.find({}, {}, { sort: { 'time' : -1}, limit: process.env.SAMPLES }, function(err, readings) {
    if (err) return console.error(err);
      res.render('index', 
        { title: 'Power Usage and Temp from PowerCost Monitor',
          refreshRate: process.env.REFRESHINT,
          sampleNum: process.env.SAMPLES,
          readings: readings
              });
           }
          );
       });

Finally, app.js contains the code to actually run the webserver, which is just as Express generated it:

http.createServer(app).listen(app.get('port'), function(){
  console.log('Express server listening on port ' + app.get('port'));
});


Layout.jade

 

Over the course of development, I went back and forth over whether to include the bulk of the client-side JavaScript in the Layout.jade file (essentially the headers of the webpage) or at the end. After reading a few articles (such as this one) , I decided to put the main client-side JS at the end of the page (in this case, the Index.jade file) and it does seem to perform better.

Not really too much to the Layout.jade file, but note this line:

      meta(http-equiv="refresh" content="#{refreshRate}" ) 


It sets the refresh rate based on the environment variable mentioned earlier. I tried to go with having the table and the Flot chart in DIVs and just refreshing the DIVs, but this seemed to not work easily so I will tackle this in the future.

Index.jade

 

This is the other major working part of this project and comes in two parts. First is the Jade portion:

It took me a while to learn how to get the variables to come across properly and to get used to how finicky Jade is about spaces! The coding and hierarchy for the resulting HTML page rely entirely on how you have used spaces to indent, so you have to be very careful! Also note that I have set up the size of the placeholder DIV for the Flot chart. It seems like you need to do this to ensure the chart generates properly.

extends layout

block content    
    h1= title
    
    h2 Graph over time
    br
    #placeholder(style='width:700px;height:300px')
    
    h2 Table of results
    #tablediv
      table
       thead
         tr 
           td 
             b Timestamp
           td
             b Temperature
           td
             b Power usage
         tbody
             each reading in readings
               tr
                 td #{reading.time}
                 td #{reading.temp2}
                 td #{reading.usage2}



The section below is the form for doing the settings. The thing to notice here is that this form also picks up the current values of the environment variables for refresh and interval. The defaults are 15 samples and an interval of 180 seconds.

    h2 Settings
    form#dataSamples(name="dataSamples",method="post",action="/")
        label(for "samples") Data samples to graph:
        input#setSamplesInput(type="text", placeholder="samples", name="samples", value="#{sampleNum}")
        br
        label(for "interval") Time between refreshes (seconds):       
        input#setIntervalInput(type="text",placeholder="interval", name="interval", value="#{refreshRate}")
        br
        button#btnSubmit(type="submit") submit

Then at the bottom of the Index.jade file is the client-side JavaScript which is really what makes everything happen. This points to the jQuery and Flot libraries (which are in the /public folder in the directory tree).

When the page finishes loading, this translates the data object of the database readings from the app.js res.render statement into two separate arrays (one for temp and one for power usage) using the JSON.stringify command so they are ready for plotting. It uses the $.plot function in the Flot library to create the plot. It also sets the labeling and axis formatting as well. (Note that to get the time-based X Axis, you need to include the Flot Time library as well as Flot itself.

    script(src='jquery/dist/cdn/jquery-2.1.0.js')
    script(src='flot/jquery.flot.js')
    script(src='flot/jquery.flot.time.js')
    script.
        window.onload = function(){
           if (window.console)console.log("Executing script");
           var temp_data = [];
           var usage_data = [];
           var reading_data = !{JSON.stringify(readings)};
           for (var i = 0; i < #{sampleNum}; i++)
             temp_data.push([new Date(reading_data[i].time), reading_data[i].temp2]);
           for (var i = 0; i < #{sampleNum}; i++)
             usage_data.push([new Date(reading_data[i].time), reading_data[i].usage2]);
           $.plot($("#placeholder"), [{
               label: "Temperature (C)",
               data: temp_data, 
               },{
               label: "Power Usage (KW)",
               data: usage_data,
               yaxis: 2
              }], {
               xaxes: [{
                 mode: "time",
                 timeformat: "%H:%M",
                 timezone: "browser"
               }],
               yaxes: [{},{position: "right"}]
              });
            };

I suspect there is a simpler way to do this that allows Flot to pick up the data directly from the rendered data object without having to use JSON.stringify array process, but I couldn't find it!

Style.css


Not really much to note here. Just some basic formatting for the few elements I am using. The complete listing is at the end.

Conclusion and next steps


So, that's it for now. As I said at the beginning, this code is certainly not complete! It needs a lot more error checking and testing in failure modes. For instance, what if the network is down or the PowerCost Wifi Monitor is not responding?

Astute observers will also note that the way I have done the "Settings" feature only really works with one browser window at a time. When I have tested changing settings with multiple browsers I have found that it sometimes doesn't pick up the environment variables properly and the refresh rate gets into an "undefined" state and then it just continually loops without any pause - obviously bad!

My next steps will be to learn the Socket.io library in Node.js and then to start connecting the BeagleBone to my DigiX board which I got through Kickstarter a few months ago! Check it out here:



Complete source code

 

app.js

 


This is the complete source code for the app.js file discussed above.
/**
 * Module dependencies.
 */


//Load Dependencies
var express = require('express');
var routes = require('./routes');
var user = require('./routes/user');
var http = require('http');
var path = require('path');
var tungus = require('tungus');
var Engine = require('tingodb');
var mongoose = require('mongoose');
var request = require("request");

//Initialize Mongoose with Tingo
var db = mongoose.connect('tingodb://readingsdb');

//Set up some global variables
var usage;
var temp;
var readings;

//Set environment variables for defaults
process.env['SAMPLES'] = '15';
process.env['REFRESHINT'] = '120';


//Mongoose connects to database
mongoose.connect('tingodb://readingsdb', function (err){
    if (!err) {
        console.log('connected to databse');
    } else {
        throw err;
    }
});

//Set up schema for database
var Schema = mongoose.Schema;
var ObjectId = Schema.ObjectId;

var ReadingsSchema = new Schema({
  time: { type : Date, default: Date.now },
    usage2: Number,
    temp2: Number
    });

var Readings = mongoose.model('Readings', ReadingsSchema);

//Do the simple screen scrape and pop the results into the database.
//Rinse and repeat according to the environment variable for refresh rate
function getData (){
    request({uri: "http://192.168.1.33/pcmconfig.htm"}, function (error, response, body){
        if (!error && response.statusCode == 200) {
            var n = body.search("Present Demand");
           usage = body.substr((n+44),5);
           usage = usage.trim();
           console.log(usage);
           var n2 = body.search("Sensor Temp");
           temp = body.substr((n2+40),3);
           temp = temp.trim();
           temp = temp - 2;
           console.log(temp);
           var readingInfo = new Readings({
               temp2: temp,
               usage2: usage
            });
        readingInfo.save(function(err, readingInfo){
              if (err) return console.error(err);
            console.dir(readingInfo);
            });
        setTimeout(getData,(process.env.REFRESHINT * 1000));
    }});
}

//run the above function
getData();

//Below are all set by default in Express
var app = express();

// all environments
app.set('port', process.env.PORT || 3008);
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'jade');
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.json());
app.use(express.urlencoded());
app.use(express.methodOverride());
app.use(app.router);
app.use(express.static(path.join(__dirname, 'public')));

// development only
if ('development' == app.get('env')) {
  app.use(express.errorHandler());
}

//When the Submit button is pressed, it runs this function and re-renders the page
app.post('/', function(req, res){
    process.env['SAMPLES'] = req.param("samples");
    process.env['REFRESHINT'] = req.param("interval");
    console.log('Samples is set to:  ' + process.env.SAMPLES);
    console.log('Refresh is set to:  ' + process.env.REFRESHINT);    
      Readings.find({}, {}, { sort: { 'time' : -1}, limit: process.env.SAMPLES }, function(err, readings) {
    if (err) return console.error(err);
      res.render('index', 
        { title: 'Power Usage and Temp from PowerCost Monitor',
          refreshRate: process.env.REFRESHINT,
          sampleNum: process.env.SAMPLES,
          readings: readings
              });
           }
          );
       });

//This serves up the page and renders the variables into the Jade template engine.
app.get('/', function(req, res){
  Readings.find({}, {}, { sort: { 'time' : -1}, limit: process.env.SAMPLES }, function(err, readings) {
    if (err) return console.error(err);
      res.render('index', 
        { title: 'Power Usage and Temp from PowerCost Monitor',
          refreshRate: process.env.REFRESHINT,
          sampleNum: process.env.SAMPLES,
          readings: readings
              });
           }
          );
       }
    );

//Start up the server!
http.createServer(app).listen(app.get('port'), function(){
  console.log('Express server listening on port ' + app.get('port'));
});


Layout.jade



doctype html
html
  head
      title= title
      link(rel='stylesheet', href='/stylesheets/style.css')
      meta(http-equiv="refresh" content="#{refreshRate}" ) 

  body
  
    block content

Index.jade


extends layout

block content    
    h1= title
    
    h2 Graph over time
    br
    #placeholder(style='width:700px;height:300px')
    
    h2 Table of results
    #tablediv
      table
       thead
         tr 
           td 
             b Timestamp
           td
             b Temperature
           td
             b Power usage
         tbody
             each reading in readings
               tr
                 td #{reading.time}
                 td #{reading.temp2}
                 td #{reading.usage2}

    h2 Settings
    form#dataSamples(name="dataSamples",method="post",action="/")
        label(for "samples") Data samples to graph:
        input#setSamplesInput(type="text", placeholder="samples", name="samples", value="#{sampleNum}")
        br
        label(for "interval") Time between refreshes (seconds):       
        input#setIntervalInput(type="text",placeholder="interval", name="interval", value="#{refreshRate}")
        br
        button#btnSubmit(type="submit") submit
        
    script(src='jquery/dist/cdn/jquery-2.1.0.js')
    script(src='flot/jquery.flot.js')
    script(src='flot/jquery.flot.time.js')
    script.
        window.onload = function(){
           if (window.console)console.log("Executing script");
           var temp_data = [];
           var usage_data = [];
           var reading_data = !{JSON.stringify(readings)};
           for (var i = 0; i < #{sampleNum}; i++)
             temp_data.push([new Date(reading_data[i].time), reading_data[i].temp2]);
           for (var i = 0; i < #{sampleNum}; i++)
             usage_data.push([new Date(reading_data[i].time), reading_data[i].usage2]);
           $.plot($("#placeholder"), [{
               label: "Temperature (C)",
               data: temp_data, 
               },{
               label: "Power Usage (KW)",
               data: usage_data,
               yaxis: 2
              }], {
               xaxes: [{
                 mode: "time",
                 timeformat: "%H:%M",
                 timezone: "browser"
               }],
               yaxes: [{},{position: "right"}]
              });
            };

Style.css


body {
  padding: 50px;
  font: 14px "Lucida Grande", Helvetica, Arial, sans-serif;
}

table
{
border-collapse:collapse;
}
table,th, td
{
border: 1px solid black;
}

td
{
padding:6px;
}


Wednesday, October 23, 2013

Resurecting the Arduino/Xbee/Gumstix sensor network

About four years ago, I built a simple sensor network that used a couple of Arduinos, a Gumstix Linux mini computer and Xbee wireless that acted as an indoor-outdoor temperature display. A couple of years back, the outdoor part of the system stopped working. I poked it and prodded it and finally just decided the Arduino clone I was using had died.

Finally, after it had sat on my shelf for a couple of years, I figured I should start to look at salvaging some of the parts for something new. Just as a last chance, I once again unplugged the processor and juggled the wires and suddenly, it works like a charm!! As with most thing, the threat of imminent destruction got it moving!

Anyway, in honour of it's miraculous recovery, I have fixed up my blog posts and reinserted the graphics Google lost for me. The posts can be found at:

  1. Overview of the sensor network
  2. Indoor Outdoor temp display
  3. Xbee -Gumstix hardware
  4. Gumstix-Arduino software

This project is pretty old now and obviously could be done much more easily with newer models of Arduino like the Yun, but it does show some good basics on getting Zigbee and PHP and Arduino all working together. Apparently, too, it is capable of great longevity!

Sunday, October 6, 2013

Apologies and fixes!

Hello all,

It has been a very, very, very long time since I have updated this blog, but I am still around!

I just wanted to take a moment to apologize to the few of you who might stumble onto this blog and find that the graphics are missing. It seems a while ago Google changed the way it stored embedded graphics in Blogger and it messed up all the links in most of my posts! This happened a while ago and I hadn't visited in a while, but I finally got around to fixing up at least most of the posts.

I know a few people have gotten to my blog from Arduino-related searches, so I have focused on trying to fix up my Arduino related posts - although most of these are on rather antiquated hardware.

Hopefully soon I will get to some projects with the Digistump DigiSpark which is a small form factor Arduino compatible board. You can read more about it here.

Thanks!

Saturday, October 1, 2011

Test Driving the Nissan Leaf

Well, I suppose once a year or so I should update this blog! Sorry for the long absence.

Anyway, I had a chance today to drive the new, all-electric Nissan Leaf as part of their promotional tour of Canada with the new car.  I'm not much of a car review or automotive expert, but I figured it is still worth recording my impressions since this is such a unique new vehicle. These are just my overall impressions of a quick tour & test drive. You can find loads of more detailed reviews by searching online.

Exterior

Definitely a nice look to the vehicle. It doesn't scream oddball, or high-tech. If anything, it looks like a slightly bigger version of the Versa.


 The main exterior difference? No tailpipe at all!!


The engine compartment is actually kind of funny since they have mocked it up to look as if it is an internal combustion engine. The guy at Nissan said that under the aluminium cover that looks like a valve cover is just a printed circuit board! On the front of this photo you can see the connector for the charging station. The battery you see there is obviously not the battery that drives the engine! That is just for the accessories like the a/c, lights & heating. You can get an optional solar panel on the spoiler that charges this battery - but not the main big batteries, which are in the back.



Interior

Inside the car has a very "deluxe" feel. Nice leather on the steering wheel, pleasant textures everywhere an nice quality plastic on the controls. I wonder how long it would take me to get all that nice pale colored surfaces all grungy! Over time, I think a darker interior might show the wear a bit less, but on a new car it just looks spiffy!


This also gives you a sense of the control layout, which is very sensible and easy to use. My only quibble is the little shift controller knob which you can see in the lower right of the picture above. You flick it to the left and down to go into "drive" do that twice and you are in "eco mode" and flick it left and up to go into reverse. Cute, but why would it have been hard to just put a good old shift lever labelled "D", "D-E" and "R"? It isn't hard to get used to, but it is one of the few things that doesn't seem ergonomic.

Interface

I did get a very short test drive, so I didn't get too much time to play with the displays & interface. The main display shows your speed & power level in a sensible way:


 The main touchscreen panel provides well organized controls - with an obvious focus on power consumption and range.


Again, I didn't get a chance to play a lot with this system, but my only concern was it seemed rather busy and I wonder about "driver workload" or distraction with so many options.

The Drive

What was it like to actually drive? I did get a quick drive of about 10 minutes or so which took me on some city streets and a quick jaunt on the highway and I have to say it is very impressive indeed. You start it with the power button you can see on the lower left of the above photo (maybe "boot it up" is more appropriate) and then pop it into drive with the selector and the funny thing is it just sits there dead silent! It took me a minute to realize that, of course, it's not like the engine would idle or anything - the motor does nothing until you press on the accelerator!

Once you do hit the accelerator, the driving experience is really very normal and pleasant. It is certainly the absolute quietest vehicle I have ever driven and the weight of the batteries really gives it a solid feel. I was reminded of the old saying about the Rolls Royce that all you could hear was the noise of the clock ticking on the dashboard! The pickup was very brisk since the electric motor delivers huge torque immediately, but all you hear is just a little whine as it runs up. What impressed me the most was just how comfortable and natural it felt to drive - there was really no feeling of having to transition or re-learn how to drive. I did try the "Eco" mode and that felt like it just put a wet blanket over all the power and acceleration, but then, that is what it is supposed to do - preserve your battery over performance.

Will I buy one?

Well, sadly, probably not soon. For once thing, there likely won't be real volumes of the vehicles available in Canada until later next year, but at the end of the day the range thing just doesn't quite work for me - although it is pretty close.

It is rated at 160 kms on a charge and the Nissan people told me that some testing showed under the best conditions it could do almost 190 km, but I really wonder what the real-world range will be in the extremes of the Canadian winter or our hot summers. I live in a smaller city and I often drive to surrounding communities that are 50 or 60 km away, which would start to push the range. That being said, I work from home so the total amount that I drive in a month is pretty limited, so why spend the extra to get an EV which won't be driven when I could get a very efficient gasoline vehicle for less? However, if I was commuting, say 25 kms or less a day, the range would really be quite acceptable and the extra cost justified when gas starts to creep up to $1.50 a liter. As well, the Nissan folks make a very good point that the car is really very simple mechanically and will need little by way of regular servicing compared to an internal combustion engine.

Ultimately, I don't see this as my next car (I plan to buy in the next six months or so), but if the next model gets up to, say, 220 km, I would definitely think hard about it. Driving away in my manual transmission Honda (which I like a lot) felt rather like climbing into a Model T.

Wednesday, August 11, 2010

PRK - 11 weeks post surgery

Well, this is probably my last post on this topic.

I went to my eye doctor today and he is very pleased with the progress. My left eye is basically perfect, but my right eye has a very slight -0.025 diopter nearsightedness, but this is starting from over -8 in both eyes. He said the right eye is still healing and this is just the tail end of the process. I am now down to only doing the FML steroid drops once a day.

My observation is that my vision - especially at a distance - is spectacular! Not having minor optical distortions or smudges or scratches from glasses is great and having back all of my peripheral vision is also fantastic. One of the best items of recent progress was that I found I didn't need reading glasses to work on the computer and I could get all my screens back to their regular fonts sizes!!

There are still some downside. I do have haloing at night which is irritating while driving but should clear up over the next few months. I do have to wear reading glasses now for reading in bed or playing games on my iPhone. Having to remember my reading glasses so I can read menus at restaurants and so on is irritating. Hopefully this will get slightly better, but I expect this is something I may be stuck with at my age. I do need to keep using artificial tears regularly, but my eyes don't seem to feel dry or uncomfortable except at the end of a long day.

When I was researching the procudure I found several useful diary blogs like this through Google. So, for those who might come upon this blog from the interwebs looking for info on PRK surgery, here are links to all the entries in sequential order:
It's been quite a journey!