Thursday, September 3, 2009

Arduino POP3 Email Checker

Since it turns out people actually occasionally READ this blog, I decided it was time to go back to do an Arduino project and try something different. Since I hadn't done any Arduino work with the Internet before, I decided to try something I thought would be relatively simple - having the Arduino check my email and give me some sort of visual indication of how many emails I have. I started out with:
Both were bought from the good people at Robotshop.ca.




These are just combined with a high intensity LED - which really should have a resistor and will have one eventually!



Here is a quick video - sorry it is a bit murky and the LED is a bit bright!



This was actually probably the hardest Arduino code I ever wrote! For one thing, I hadn't done anything substantial with the Arduino for a while and I found I had forgotten much of what I thought I knew. Also, getting the timing right and getting the number of emails out of the return string actually proved pretty difficult. I made things more difficult for myself by trying to use the String library (formerly TextString), which for some reason didn't return consistent results and was generally not documented and finicky. Then I found the usual LED 13 wouldn't work properly - perhaps because the Ethernet Shield was interfering with it. All-in-all, this small program must have taken me three weeks to write!

I did have a bit of help with the code from Digger450 on the Arduino forum in this exchange, which I am very grateful for!

However, now that it is done, this is a nice little demo of Ardunio on the Internet that does do something at least semi-useful. My next extension may be to hook it up to a servo so that it shows my emails on a physical chart or something. I could also use my SparkFun SerialLCD unit to display the subject lines or something.

Here is the source code:

/*=================================================
Ethernet POP3 Mail Checker & indicator

Checks how many messages are waiting on the POP 3 server
and flashed LED on Pin 9 to indicate number of messages.

It will handle up to 99 messages in the POP3 mailbox.

Uncomment the serial lines for troubleshooting.

Copyright by Chris Armour
3 September 2009
http://opensourceprojects-torchris.blogspot.com/

===================================================*/

#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192,168,0,167 }; // IP address you wish to assign to Arduino
byte server[] = { XXX, XXX, XXX, XXX }; // IP address of your POP3 server
char inString[165]; // Number of characters to be collected
int i = 0;
int mailNum1 = 0; // First digit of the email number
int mailNum2 = 0; // Second digit
int mailTotal = 0; // Total # of messsage
char d;
int ledPin = 9;

Client client(server, 110); //The default POP port is 110

long updateTimer;
boolean clientConnected = false;

void setup()
{
//  Serial.begin(9600);
pinMode(ledPin, OUTPUT);
digitalWrite(ledPin, LOW);
}

void loop()
{
   
updateClient();
d = checkAvail();
if (d >= 10){
getMailNum();
}

}

/*========================================
        Functions
=========================================*/

void updateClient() //This function contacts the POP3 server
{
if ((millis() - updateTimer) > 5000)
{
 Ethernet.begin(mac, ip);
//    Serial.println("connecting...");
 delay(1000);
 if (client.connect())
 {
//    Serial.println("connected");
 client.println("user Your.Name"); //Insert your usual email login name
 client.println("pass PassWord"); //And your password here
 client.println("quit");
 client.println();
 clientConnected = true;
 }
 else
 {
//    Serial.println("connection failed");
 }
 updateTimer = millis();
}
}

char checkAvail() //This checks if there is data available and returns a char
{
if (clientConnected)
{
 if (client.available())
 {
 char c = client.read();
     return(c);
 }
 if (!client.connected())
 {
//    Serial.println();
//    Serial.println("disconnecting.");
 client.stop();
 clientConnected = false;
 }
}
}

int getMailNum() //This actually loads the char returned by checkAvail() and puts in into an array
{
inString[i] = d;
i++;
if (i == 165){
   i = 0;
       client.flush();
       mailNum1 = inString[106] - 48; //Array position 106 contains the first digit
       mailNum2 = inString[107] - 48; //Array position 107 contains the 2nd digit if it is available
       if ((mailNum2 >= 0) && (mailNum2 <= 9)){ //If mailNum2 is present, then it is a two digit mail count
         mailTotal = (mailNum1 * 10) + mailNum2; //when 2 digits are present, multiply the 1st by 10 then add to mailTotal
//            Serial.print("Total emails:  ");
//            Serial.println(mailTotal);
          blinkLED(); //Run the blink function as many times as there are emails
       }
       else {
         if ((mailNum1 >= 0) && (mailNum1 <= 9)){//if there is only one digit, then that is mailTotal
         mailTotal = mailNum1;
//        Serial.print("Total emails:  ");
//        Serial.println(mailTotal);
         blinkLED(); //Blink the LED
         }
         }
     }
 }

void blinkLED(){ //Blinks the LED for as many times as indicated by mailTotal
         for(int x = mailTotal; x >= 1; x--){
           digitalWrite(ledPin, HIGH);
           delay(200);             
           digitalWrite(ledPin, LOW);
           delay(200);
         }
}

29 comments:

  1. Outstanding! Since my ethernet shield arrived courtesy of the good folks at Tinker.it on Wednesday, this looks like an ideal first project for it. Thanks Torchris!

    ReplyDelete
  2. Do i need a atmega 328, i have a 168 diec arduino and would like to get a net shield but just making sure it would be compatible. thank you.

    ReplyDelete
  3. I just ran a quick check with my older Diecimilia (ATMega 168) and it runs just fine. The compiler reports: "5244 bytes (of a 14336 byte maximum)" so there is lots of memory for this program.

    ReplyDelete
  4. Very cool, I have been looking to make something like this for a long time! Thanks so much.

    ReplyDelete
  5. Great! That's why I write this is to help others save some time.

    I actually have a very small modificaton I will post shortly so it signals if it loses Ethernet connectivity.

    ReplyDelete
  6. Great project! I just ordered an Arduino clone and waiting for it's arrival.

    www.Nerdful.com

    ReplyDelete
  7. So what I don't get is where to plug the ethernet shield into? DO I have to plug it into my computer? Or do I plug it into a router? If I have an ethernet switch can I just run the ethernet cable from the shield to the switch? And how do I setup a pop3 email?

    ReplyDelete
  8. Oh, and what mac address do I use and what do I use for my ip address?

    ReplyDelete
  9. Ok. I configured a pop3 server through gmail. But now how do I know what the IP address of the pop3 server is?

    ReplyDelete
  10. You need to run an ethernet cable from the Arduino to a free port on your router. You may be able to plug it into a switch as long as it is going to a router. I am not sure about connecting it to a PC directly. It might work with Windows Internet COnnection Sharing, but I doubt it.

    ReplyDelete
  11. The MAC address can be anything that doesn't conflict with anything else on your network - in other words - make one up! :-) For the IP, again, it needs to be something that won't conflict with other IPs on your home network. Just look at your router and check for an available IP - probably something outside the range of IPs being used for DHCP on you network.

    ReplyDelete
  12. This is an important point - you CANNOT use Gmail! Gmail is SSL encrypted and the Arduino can't do SSL - at least as far as I know.

    Once you have found a non-SSL POP3 provider, all you need to do is ping the server by it's name and ping will return the IP address. I use Rogers in Canada and when I ping pop.broadband.rogers.com I get:

    Pinging pop-rog.mail.fy2.b.yahoo.com [206.190.36.17] with 32 bytes of data:

    so I know the IP is 206.190.36.17.

    ReplyDelete
  13. Ok. Well, I am living in a dorm in college and this is how my internet is set up. We have wireless, but right now I have an ethernet cable running form the wall port to a switch. Then from the switch it branches out to my 2 computers, ps3, etc. I have a linksys router I can use, so can I do this: Plug the arduino board into the router port, then run an ethernet cable from another port on the router to the switch? And one more question about ip address and mac address. For the ip, since my router's ip is 192.168.1.1, could I use say 192.168.1.101 ?? And for my mac address could I use 00,11,22,33,44,55 ?? And thanks for all the help. What I eventually want the program to do is to read a specific text in the pop3 mail and then if the text is valid, turn a servo that opens my dorm room door. As of right now, my door has an rfid access.

    ReplyDelete
  14. That _should_ work, but a lot would depend on how your university has set up its network. Sorry, can't help you there.

    On the MAC, you could just copy the MAC from my program above - 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED. Highly unlikely it will clash with anything else on your network.

    As to the reading text in an email, that is a graet project idea and I built something like that off of the PC parallel port many years ago. The thing to watch out for is the Arduino has very restricted sizes for how much text you can load into RAM.

    I am working on a follow-on project to read just the subject & from parts of an email and you need to read at least the first 2,000 or so characters of the email return (via the RETR command in POP3). Unfortuantely, you can't make a text array hold more than about 800 characters!!

    To get around this I am using the "flash.h" library found here -> http://arduiniana.org/libraries/flash/. Unfortunately, I haven't figured out how to use this yet so I can't be much help!! :-(

    ReplyDelete
  15. Alright, sounds good. Thanks again for all the info.

    ReplyDelete
  16. I still cannot get it to work. I can't seem to find a pop server to store my mail on. Would you be able to recommend a free one to me. I live in Washington State. US

    ReplyDelete
  17. Sorry, I live in Canada so I wouldn't know. I think local ISPs may be more likely to have unencrypted POP service. Failing that, maybe the university IT people could help.

    Good luck with your project!

    ReplyDelete
  18. ok. So I got it hooked up to the computer and the shield can act as a webserver. So I loaded your code and did everything I was supposed to do including putting my email account and password. Then I uncommented the serial so that I could troubleshoot. And it says connecting...connected, and then it says disconnecting. And then it repeats that over and over. It does not print the number of emails in the serial. So I don't know where to go from here. It says connected, so doesn't that mean that it is connecting to the pop server?

    ReplyDelete
  19. Please have a look at my September 28th posting that deals with some of the troubleshooting issues ->

    http://opensourceprojects-torchris.blogspot.com/2009/09/further-on-pop3-email-checker.html

    Specifically, you should try the simpler version of the software that I have in that posting which will just print out the ouput from your POP3 server.

    My code relies on the _exact position_ of the email count in the return stream from the POP3 server. Your POP server may not be returning the exact same string - or maybe is returning nothing!

    Another test is to try connecting to your POP3 server with telnet and checking if you can manually get the data. Here is a very, very, very old article on how to do this!

    http://smanage.tripod.com/tel.html

    ReplyDelete
  20. Ok. Using your debugging program short script I was able to return the server information as well as the number of messages in the pop server. The output looked something like this:

    Ok+ Pop3 Server online <12321.212312@tamar.safe-mail.net
    Ok+ PASS REQUIRED
    OK+ 4 messages in inbox
    Ok+ Shutting down server


    Now, for the number of characters that I set and the number of the message character in the array. How do I count how many characters there are? Do I count spaces? And do I start counting before the Ok+ ? Or do I just count the characters following the Ok+ on each line of the statements?

    ReplyDelete
  21. That's great progress!!

    Yes, just cut & paste that into Word or Notepad and count from ZERO. The program loads the text into an array and the 1st item in an array is always 0. You will likely have to try a few different combinations to get it right.

    Feel free to write me directly - torchris4 AT gmail DOT com.

    ReplyDelete
  22. I had to ad the gateway IP and netmask of my ADSL modem to get it connected to the pop server.
    I also had to add a extra command "stat" (after the password) to get a message how many emails I have.

    Before client.flush() line 106 I added Serial.print(i); and Serial.print(inString[i]);
    this way it was easier to discover the right position of the email number.

    ReplyDelete
  23. I have a problem with the script. When it says Client client(server, 110);, it says Client has been renamed EthernetClient. How do I fix the script?

    ReplyDelete
    Replies
    1. Bear in mind this was written 3 years ago. I believe the library has changed in that time. Please refer to the library documentation at http://arduino.cc/en/Reference/Ethernet.

      Delete
  24. Hey! I seem to be stuck where it says " if (!client.connected())" instead being at " if (client.available())". I am connecting to my email by IMAP through icloud. It connected correctly but doesnt want to pull emails.

    ReplyDelete
    Replies
    1. I have only ever used this with POP3 so I have no idea how it might work with IMAP.

      Delete
  25. It seems like it may work, but I have to get an email without SSL encryption. Do you know any free email services without SSL /STL ecryption?

    ReplyDelete
    Replies
    1. No, I do not. Like I said, this was done a few years ago. At that time, my home ISP still had old-fashioned non-SSL POP email, but I believe now they have gone over to SSL as well. I am not sure if there are any Arduino libraries that support SSL, but that would be a good challenge for you to find and improve on this. Really, this is just a pretty basic demo-type application anyway.

      Delete