Thursday, August 5, 2010

Arduino-Twilio Dialer Application

Well, it has been a while since I posted an Honest-to-God Arduino project here! What with getting my eye surgery done and one thing and another, this has taken me quite a while. Besides, there were more than a few challenges in getting it all to work along the way!

This project is a concept for an information kiosk or unit that could be placed in a public area where passers-by could request information. A user enters their mobile phone number into the keypad and they are then immediately called by the Twilio cloud-based telephony system (check out Twilio here) and presented with a simple phone menu that allows them to:
  1. Talk to an operator
  2. Leave a voice message
  3. Receive an SMS message
Here is the hardware in it’s not very pretty form:








































Here is the obligatory video showing how it works:




Hardware

The hardware consists of:
  • Arduino Duemilanove
  • Arduino Ethernetshield
  • Sparkfun serial enabled 20 x 4 LCD display (LCD-09568)
  • Sparkfun 12 button keypad (COM-08653)
Here is the circuit diagram:


The hookup of the keypad is based on this article by amando96 on Instructables. You will have to sign up for membership to see the whole article or you can just follow the above diagram. One odd thing I noticed was that the pull down resistors turned out to not be required. Originally I had them included and everything worked fine, then at a certain point in the development the keypresses started looping and I removed the resistors and everything worked fine!

This just uses the standard “keypad.h” library found on the Arduino website, but slightly modified so the pins do not interfere with the Ethernetshield. There is also an Instructable post on multiplexing the button input to use fewer digital pins, but since I didn’t need any extra digital pins I figured this was simpler.

The serial LCD is quite straightforward to hookup, but just note that I am using one of the A1 Analog pin as the Tx for the serial since I am not using any analog inputs anyway.

Software

As with the previous Twilio-Arduino project, this one requires:
  • An account with Twilio 
  • A web-server with PHP
  •  An Ethernetshield
The Arduino communicates with one PHP script running on my web-server, this then triggers Twilio to set up a call and Twilio then goes to my web-server and checks for another PHP file on what options to present to the user. It is a reasonably complex mesh of files that need to be setup, but actually the Twilio XML programming was easily the simplest part of the setup.

This diagram shows the overall flow:





The first part is the Arduino software:





/*
*  Twilio Arduino Information Service Dialer
* When used in conjunction with "dialer_twilio.php PHP script and the Twilio cloud-based
* telephony environment (www.twilio.com) this allows a user to enter a phone number into a keypad
* on an Arduino and have a call placed to their cell phone with various options for information.
*
* Uncomment the //Serial lines for troubleshooting/debug info.
*
* This code is in the public domain. Please provide credit if it is used
* in another project.
* 
* written by Chris Armour, August 4th, 2010
*
* Full description psted at http://opensource-torchris.blogspot.com/
*
*/

//=====================Libraries=============/
#include 
#include 
#include  
#include 

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //MAC address for Arduino
byte ip[] = { 192,168,0,34 }; // IP address you wish to assign to Arduino
byte server[] = { 192, 168, 0, 171 }; // IP address of your PHP server
int ServerPort = 55455;
const byte ROWS = 4; //four rows
const byte COLS = 3; //three columns
char keys[ROWS][COLS] = {
  {'1','2','3'},
  {'4','5','6'},
  {'7','8','9'},
  {'*','0','#'}
};
byte rowPins[ROWS] = {5, 4, 3, 2}; //connect to the row pinouts of the keypad
byte colPins[COLS] = {8, 7, 6}; //connect to the column pinouts of the keypad
boolean ConnectedState = false;
char DialNum[12]={"xxxxxxxxxxx"};
int a;
NewSoftSerial mySerial =  NewSoftSerial(14,15); //osft serial running off of Analog Pin 1

Keypad keypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS );
Client client(server, ServerPort); 
TextFinder finder( client);

void setup(){
  Ethernet.begin(mac, ip);
// Serial.begin(9600);
  mySerial.begin(9600); 
  mySerial.print(254, BYTE);
  delay(100);
  mySerial.print(1, BYTE);
  delay(100);
  mySerial.print("Started - Press any key to conect to the server."); //Display initial message on LCD.
}
  
void loop(){
    
  char key = keypad.getKey();

  if (key != NO_KEY){
    if (ConnectedState == false){
       if (client.connect()){      
         if(finder.find("results:") == true){
           char c = client.read();
         // Serial.print("Value of the connect code: ");
         // Serial.println(c);
           mySerial.print(254, BYTE);
           delay(100);
           mySerial.print(1, BYTE);
           delay(100);
           mySerial.print("Connected. ");
           delay(2000);
           WelcomeMsg();    
           //If connection successful, then print Welcome Msg and flip the connected state variable.
           ConnectedState = true;
         // Serial.println(DialNum);
           }
       }
       else {
       ConnectedState == false;
       ConnectFailMsg(); //If connection fails, then print an error.
       delay(2000);
       }
    }
    if (key == '*') {
    //When it picks up the "*" then it starts gathering the digits.
    // Serial.println("Got the *");
      EnterNumMsg();       
      do {
        key = keypad.getKey();
        if (key != NO_KEY){
          delay(150);
        // Serial.println(key);
          mySerial.print(key); //Prints the keypad input to the LCD for user feedback.
          DialNum[a] = key; //Puts the key info entered into an array.
          a++;
          }

        } while (key != '#');
        
      }
      if (key == '#'){
        //When the "#" is pressed, the info is sent over to the PHP script.
         delay(250);
         mySerial.print(DialNum);
         client.println(DialNum); //Dialnum array sent to PHP.
         if(finder.find("results:") == true){//PHP script evaluates number & sends back response.
             char c = client.read();
           // Serial.print("Value of the return code: ");
           // Serial.println(c);
             if (c == 'Y'){//Y back from the script means it is a valid number & success displayed.               
               NumAcceptedMsg();
               }
             if (c == 'N'){//If it is not a valid number, then N returned & error displayed.
               NumNotAcceptedMsg();
//               client.println(DialNum);
               delay(3000);
               WelcomeMsg();//Go back to Welcome message.
               }
         delay(200);
         if(finder.find("results:") == true){
             char c = client.read();
           // Serial.print("Value of the return code: ");
           // Serial.println(c);
             if (c == 'O'){//If the call was successully placed display success then go back to Welcome.
              NumCalledMsg();
              delay(4000);
              WelcomeMsg();
         }
          if (c == 'X'){//If the call couldn't be made then error message & return to welcome.
              NumNotCalledMsg();
              delay(3000);
              WelcomeMsg();
         }
        } 
   
  // Serial.println(DialNum);
  // Serial.println(a);
    a = 0;
      }
    }
  }
}

void WelcomeMsg(){
       mySerial.print(254, BYTE);
       delay(100);
       mySerial.print(1, BYTE);
       delay(100);
       mySerial.print("Welcome to the      Information service.Press the * key to  start.");
  }
    
void EnterNumMsg(){
       mySerial.print(254, BYTE);
       delay(100);
       mySerial.print(1, BYTE);
       delay(100);
       mySerial.print("Enter your 10 digit phone number        followed by #:      ");
}  

void ConnectFailMsg(){
       mySerial.print(254, BYTE);
       delay(100);
       mySerial.print(1, BYTE);
       delay(100);
       mySerial.print("Connection Failed.  Press any key to    reconnect.");
}

void NumAcceptedMsg(){
      // Serial.println("Number accepted by server");
         mySerial.print(254, BYTE);
         delay(100);
         mySerial.print(1, BYTE);
         delay(100);
         mySerial.print("Number Accepted. ");
}

void NumNotAcceptedMsg(){
        // Serial.println("Number NOT accepted by server");
           mySerial.print(254, BYTE);
         delay(100);
         mySerial.print(1, BYTE);
         delay(100);
         mySerial.print("Number not valid.");
}

void NumCalledMsg(){
            // Serial.println("Number called successfully.");
               mySerial.print(254, BYTE);
               delay(200);
               mySerial.print(1, BYTE);
               delay(200);
               mySerial.print("Number called       successfully.");
}

void NumNotCalledMsg(){
// Serial.println("Number cannot be called.");
  mySerial.print(254, BYTE);
               delay(100);
               mySerial.print(1, BYTE);
               delay(100);
               mySerial.print("Number cannot be    called.");
}


This uses the Arduino Ethernet library to set up a client application, the Keypad library to get the keypresses, NewSoftSerial to send information to the LCD and finally the TextFinder library to parse the result codes from the server. I have put all the LCD messages into functions just to clean up the code since each message needs to have a screen clear send and some delay. I’m sure I could further clean that up to just have a function that did the screen clear, but this works.

The Arduino client in term communicates with the PHP server program (dialer_twilio.php):


request("/$ApiVersion/Accounts/$AccountSid/Calls", 
     "POST", array(
     "Caller" => "XXXXXXXXXXXXXXX",
     "Called" => $dial_num,
     "Url" => "http://xxxxx.xxxxx.xxx/twi_test.php"
    
    ));
    if($response->IsError){
     echo "Error: {$response->ErrorMessage}\n";
   socket_write($spawn, "results:X\n");
   echo "Sent over X\n";
  }  else {
     echo "Started call: {$response->ResponseXml->Call->Sid}\n";  
   socket_write($spawn, "results:O\n");
   echo "Sent over O\n";
   }
} else {
  echo ("Invalid number\n");
  socket_write($spawn, "results:N\n");
  echo "Sent over N\n";
 }

   usleep(5000);
}
}while(true);

?>




Anything marked “XXXX” above is info on the account and phone numbers to be used.

This script does a quick and dirty check if the number is valid (just making sure there are no stray “x” or “#” characters and it is the right length) then uses theTwilio REST API to  set up the call. Twilio will go to the “twi_test.php” script for info on how to handle the call.

So, when Twilio goes to twi_test.php it sees:




    header("content-type: text/xml");
    echo "\n";
?>



    Hello. Welcome to the Arduino Information Service.
    Press 1 to be connected to an operator.
    Press 2 to leave a voicemail.
    Press 3 to receive an SMS message.
    Press 4 to repeat this menu.
    Press 5 to hangup.


This sets up the top level menu of the simple IVR and then directs to go to “twi_action.php” once a button is pushed. The XML such as and are the Twilio “verbs” that defines actions on the Twilio cloud system.

So here is “twi_action.php";


$stringData = $_REQUEST['Digits'];

switch($stringData){
case 5:
    echo "";
    echo " Thank you, Good bye.";
    echo "";
    echo "";
    break;
case 4:
    echo "";
    echo "";
    echo "http://xxxxx.xxxxx.xxx/twi_test.php";
    echo "";
    echo "";
    break;
 case 3:
    echo "";
        echo "You are being sent an SMS message.";
        echo "You are receiving this message from the Twilio Arduino dialer application.";
    echo "";
    break; 
case 2:
    echo "";
    echo "Leave your message after the tone and press # when you are done.";
    echo "";
    echo "";   
    break;
case 1:
    echo "";
        echo "XXXXXXX";
        echo "Goodbye";
    echo "";       
    break;
}

?>


This parses the digit pressed from the POST command and then takes action on the digits, which is either:



  1. Connect to an operator (in this case it dials through to my Skype account).
  2. Leave a voicemail. Currently this records a sound file which you can download from the Twilio website, but it could also have the sound file sent to an email address or transcribe the voice into text and have that sent along.
  3. Receive and SMS message. This sends a canned message that could be some general information to the cell phone that made the call.
  4. Repeat the menu.
  5. Hangup.
The final small script is the one that handles the voicemail recording (voicerecorder.php):



    header("content-type: text/xml");
    echo "\n";
?>

    Thanks for the message.  Here is what you recorded.
    
    Goodbye.


This one is very simple. It just plays back the message that was just recorded. That’s all there is to it! :-)

Conclusion

This project could have some practical uses as a low cost way to place a simple information service in some public place. It does, however, have the obvious limitation that there is no validation that the number entered really does belong to the user’s cell phone and not to some little old lady in Detroit, so in it’s current form it could just be a nuisance generator. This could probably be fixed with an exchange of a PIN via SMS.

What other things could this kind of project be used for? Here are some ideas:


  • Make a super secure lock by combining this project with an RFID reader lock so that once the card is swiped the user also has to enter a PIN and a code received via SMS at a cell phone that has been pre-registered with the system. This way an intruder would have to steal not only the users RFID badge, but also their phone and their PIN number.
  • This could also be adapted to trigger phone calls or SMS messages for various conditions driven by digital or analog inputs - such as temperature readings or intruder alarms. Conversely, as discussed in my Twilio LED project, a call could be used to control switches, motors or valves remotely using a telephone.
  • If it used Power Over Ethernet it would eliminate the extra power supply requirement. Even better would be using wi-fi or GSM cellular data to eliminate the need to have it wired to a router.
  • Clever use of the PROGMEM would probably allow all the response strings to be stored on the Arduino so it might be possible to dispense with using an outside webserver altogether!
This was a pretty challenging project and I hope it provides inspiration for some further cool projects!


6 comments:

  1. How did you get the array to reject the "#" sign as an input to the array, causing an overflow

    ReplyDelete
  2. Lines 101 to 105 of the Arduino sketch:

    } while (key != '#');

    }
    if (key == '#'){

    If the # is pressed, then the sketch goes on to the process of sending the number over to the PHP script.

    ReplyDelete
  3. Hey man I am very new to this PHP things. Never head of it before.
    I have only been working with the arduino UNO for 6 months now and created some really cool projects.
    However I've never created that requires phone communication via arduino and would really like to do ur project so I can enhance my learning.


    I have a few questions.
    1. You said A web-server with PHP will be required. I want to be on the same track as you, doing this project. So what web-server with PHP did you use.

    2. I doubt you're using any PC to run this project. It seems your using only an ethernet shield that is connected to the router. Am I right?
    I am asking this question because I'm planning on building a device which uses the arduino to send emails and make phone calls without the use of a PC, however I've never come across any project that uses just the Ethernet shield and no pc. so this really possible?

    3. I was highly impressed in your video when you connected to skype. I did not even know that was possible with the arduino.

    4. Could you please send me all the files you used for this project. I really want to learn this stuff. it looks incredible.

    ReplyDelete
    Replies
    1. Hello Varun,

      I think you need to review my previous(and simpler) Arduino-Twilio project at (http://opensource-torchris.blogspot.ca/2010/05/arduino-phone-control-via-twilio.html). Answers to you qeustions:

      1. I believe I was just running PHP on my either my Windows desktop or on my Linux Gumstix mini ssytem.

      2. Yes, there is a PC that was running my web server. Perhaps newer Arduinos have enough horsepower to do all the web hosting themselves. This project is a few years old now and uses a fairly old Arduino. Raspberry PI or Beaglebone + an Arduino could definitely do the job, for instance.

      3. Bear in mind that the Arduino is NOT doing any telephony processing here. This is entirely handled by the Twilio cloud service. As far as I know, the Ardunio would not have the horsepower to do all the media processing required for true telephony, but it would be an interesting challenge to build a SIP phone around one, for instance.

      4. All the files that are needed are in my blog - between this project and my previous Twilio project.

      Definitely also check out www.twilio.com for more info on their service & API.

      Delete
  4. Hello,
    Good stuff everything you're sharing, Thanks for that!.

    I'm developing a project that requires (if possible) updating 4 String variables from the Arduino Sketch from the Internet. This will help me ensure that data-logging is correct. I now have to kill the power, reload the sketch with the new variables and restart the Arduino.

    I'm not nearly as capable as you are with this coding (Specially with PHP, that I'm truly a beginner). I was wondering if you knew whether what I'm suggesting is possible? If so, could you help me achieve this, perhaps an URL, a piece of code, or something I can work on?

    Thanks in advance!

    ReplyDelete
  5. Hi Gustavo,

    I should think that is possible, but I really couldn't tell you how. I wrote this more than 2 years ago and I've forgotten a lot! You really just need to look into how to populate variables over the serial-to-ethernet link, which is not too hard. Sorry I can't be more help.

    Best of luck!

    ReplyDelete