Monthly Archives: December 2014

Raspberry Pi and Unison

What is Unison

Unison is a small file synchronisation tool, very similar to Dropbox actually. Only difference is that Dropbox is a cloud service, while Unison is able to perform syncing between two computers on a local network.

Install Unison on MAC

I have been downloading Unison (version 2.40.69) for MAC OSX. You can find it here.
The interface looks like the screenshot below:

Install Unison on Raspberry Pi

pi@raspberrypi ~/TestUnison $ sudo apt-get install unison
pi@raspberrypi ~/TestUnison $ unison -version
unison version 2.40.65

Configuring Unison

We should configure Unison to sync correctly. To do this

WAPPTASTIC: $ cd /Users/wapptastic/Library/Application Support/Unison
WAPPTASTIC: $ ls -al
total 64
drwx------  10 wauterw  staff   340 Dec 25 18:13 .
drwx------  53 wauterw  staff  1802 Dec 25 18:18 ..
-rw-------   1 wauterw  staff   658 Nov 19 21:24 ar09d2ec108d12d6b57ec99b7a6b117f6e
-rw-------   1 wauterw  staff   779 Dec 25 18:13 ar4d95c69eb4e432a71f26ae4240c5733d
-rw-------   1 wauterw  staff   917 Aug 22 14:55 ar7537e7a45b78e53777318b7340628532
-rw-r--r--   1 wauterw  staff  3329 Dec 25 18:11 default.prf
-rw-------   1 wauterw  staff   172 Nov 19 21:23 fp09d2ec108d12d6b57ec99b7a6b117f6e
-rw-------   1 wauterw  staff    34 Dec 25 18:18 fp4d95c69eb4e432a71f26ae4240c5733d
-rw-------   1 wauterw  staff    34 Aug 22 14:55 fp7537e7a45b78e53777318b7340628532
-rwxr-xr-x   1 wauterw  staff   356 Oct 21 19:46 start.py

You see there is a default.prf file. This file is used by the MAC version of Unison when the application starts. Using this profile, Unison knows ‘from’ where ‘to’ where you would like to sync.

# Unison preferences file
root = /Users/wapptastic/TestUnison
root = ssh://pi@192.168.10.1//home/pi/TestUnison

# Some regexps specifying names and paths to ignore
#ignore = Path stats    ## ignores /var/www/stats
#ignore = Path stats/*  ## ignores /var/www/stats/*
#ignore = Path */stats  ## ignores /var/www/somedir/stats, but not /var/www/a/b/c/stats
#ignore = Name *stats   ## ignores all files/directories that end with "stats"
#ignore = Name stats*   ## ignores all files/directories that begin with "stats"
#ignore = Name *.tmp    ## ignores all files with the extension .tmp

#          When set to true, this flag causes the user interface to skip
#          asking for confirmations on non-conflicting changes. (More
#          precisely, when the user interface is done setting the
#          propagation direction for one entry and is about to move to the
#          next, it will skip over all non-conflicting entries and go
#          directly to the next conflict.)
auto=true

#          When this is set to true, the user interface will ask no
#          questions at all. Non-conflicting changes will be propagated;
#          conflicts will be skipped.
batch=true

#          !When this is set to true, Unison will request an extra
#          confirmation if it appears that the entire replica has been
#          deleted, before propagating the change. If the batch flag is
#          also set, synchronization will be aborted. When the path
#          preference is used, the same confirmation will be requested for
#          top-level paths. (At the moment, this flag only affects the
#          text user interface.) See also the mountpoint preference.
confirmbigdel=true

#          When this preference is set to true, Unison will use the
#          modification time and length of a file as a `pseudo inode
#          number' when scanning replicas for updates, instead of reading
#          the full contents of every file. Under Windows, this may cause
#          Unison to miss propagating an update if the modification time
#          and length of the file are both unchanged by the update.
#          However, Unison will never overwrite such an update with a
#          change from the other replica, since it always does a safe
#          check for updates just before propagating a change. Thus, it is
#          reasonable to use this switch under Windows most of the time
#          and occasionally run Unison once with fastcheck set to false,
#          if you are worried that Unison may have overlooked an update.
#          The default value of the preference is auto, which causes
#          Unison to use fast checking on Unix replicas (where it is safe)
#          and slow checking on Windows replicas. For backward
#          compatibility, yes, no, and default can be used in place of
#          true, false, and auto. See the section "Fast Checking" for more
#          information.
fastcheck=true

#          When this preference is set to true, the textual user interface
#          will print nothing at all, except in the case of errors.
#          Setting silent to true automatically sets the batch preference
#          to true.
silent=true

#          When this flag is set to true, file modification times (but not
#          directory modtimes) are propagated.
times=true

The most important pieces are:

The source file is /Users/wapptastic/TestUnison
The destination sync folder is /home/pi/TestUnison and can be found ssh://pi@192.168.10.1
No questions are asked by the Unison program (like confirmation messages etc….)

Putting it to test

Before we continue testing, it is important to note that we must align the major version numbers, in our case we both have 2.40 so that should work.

When the default.prf profile is correctly configured, launch Unison on your MAC. When Unison starts, you will get a screenshot similar to the one above. Then hit the ‘rescan’ button. This will scan the source folder for new content. Below screenshot is taking a view on the source folder (Unison on MAC) as well as the destination folder on the Raspberry Pi. Obviously, both folders are completely empty.

Create some folders and files in the source directory on your MAC:

and then rescan again using Unison:

Obviously, you see that Unison has noticed the changes in the source directly, but as we did not sync yet with the destination folder, that one remains empty.

Then hit ‘Go’ in the Unison app on your MAC. You will see that Unison will then start to sync and that eventually the filers and folders end up in the destination folder. Great, so all content between our MAC folder and Raspberry Pi folder are now in sync.

To show that the sync really works bidirectionally, let’s remove some folders from the destination directory on our Raspbery Pi. In below screenshot, you can see we removed the folders ‘test2’ and ‘test3’, but did not perform the sync yet.

Eventually, after performing the sync, we end up only with the folder ‘test 1’ in our MAC. It’s clear from this that Unison is syncing bidirectionally.

Have fun with this great tool!!

Getting Started with PubNub and Sendgrid

Inspired by my first experience with PubNub I took it a small step further. While publishing some messages, I also wanted to send an email along using the Sendgrid service. Below is the code for this:

from Pubnub import Pubnub
import sendgrid 
import json

publish_key = ''
subscribe_key = ''
channel = 'wapptastic'
message = 'Hello Wapptastic'

def publish( channel, message ):
    # Email List
    recipients = [
        [ "info@wapptastic.com", "Wapptastic" ],
        [ "test@example.com", "Example mail" ]
    ]

    # Info Callback
    def callback(info): 
        print(info)

    # Connection to SendGrid
    emailer = sendgrid.SendGridClient( '', '')
    pubnub = Pubnub( publish_key=publish_key, subscribe_key=subscribe_key)

    # PubNub Publish
    pubnub.publish( channel, message, callback=callback, error=callback )

    # Email Message Payload
    email = sendgrid.Mail()
    email.set_from("Wim Wauters ")
    email.set_subject("PubNub Message")
    email.set_html(json.dumps(message))
    email.set_text(json.dumps(message))

    ## Add Email Recipients
    for recipient in recipients:
        email.add_to("%s <%s>" % (recipient[1], recipient[0]))

    ## Send Email
    emailer.send(email)
publish('wapptastic', 'Email Test')

And a subscriber, similar to the one from the previous blog post on PubNub:

import datetime
from Pubnub import Pubnub

publish_key = ''
subscribe_key = ''
channel = 'wapptastic'

pubnub = Pubnub(publish_key=publish_key, subscribe_key=subscribe_key)

def callback(message, channel):
    print(message)


def error(message):
    print("ERROR : " + str(message))


def connect(message):
    print("CONNECTED")


def reconnect(message):
    print("RECONNECTED")


def disconnect(message):
    print("DISCONNECTED")


pubnub.subscribe(channel, callback=callback, error=callback,
                 connect=connect, reconnect=reconnect, disconnect=disconnect)

When both the publisher.py and subscriber.py are run using Python, you will see that the subscriber gets all messages directly from the publisher service, but also an email is sent along to the email recipients.

Raspberry Pi: Streaming from camera

I have been trying to stream video from my PiCam directly to an iPhone or Android phone. I would prefer to do this based on HTTP Live Streaming but so far this has not worked out successfully. So I decided to use VLC for the time being.

To start, first install VLC:

pi@raspberrypi ~/Programs/ $ sudo apt-get install vlc

Then run the following command:

pi@raspberrypi ~/Programs/UploadS3 $ raspivid -o - -t 0 -w 800 -h 400 -fps 24 |cvlc -vvv stream:///dev/stdin --sout '#standard{access=http,mux=ts,dst=:8160}' :demux=h264

Probably this needs some clarification:

First of all, raspivid is the built in Raspberry Pi tool to capture the video. The -o parameter causes the output to be written to stdout and -t 0 means there is no timeout. The width and height of the video is set to 800 and 400 respectively with the -w 800 and -h 400 parameter.

Then, the cvlc is the console VLC player that is responsible for streaming the video from the PiCam. The -vvv parameter specifies to get the stream from Stdin while the –sout parameter informs cvlc where to stream it to, in our case we stream to HTTP port 8160.

For viewing the stream, open a VLC player running on a computer in the same network and go to http://ip_raspberry_pi:8160/ and you will see the live images. Pretty neat, isn’t it?

In case you wonder what the picture is: it’s a screenshot from the video that is streamed from my PiCamera. The rainbox cable is the GPIO cable I’m using to connect the RPi to my breadboard.

Raspberry Pi: Upload images to AWS S3

I recently purchased a camera for my Raspberry Pi. Have been playing around a bit with it. As my readers know by now, I typically start with some easy stuff to get used to it. So I decided to take some photo’s. Obviously I then wanted to see them. As I usually connect through SSH, I could have decided to install an FTP server on my RPi and get them of the device that way. Instead I wanted to try something more fance, so I decided to upload the photo’s to Amazon S3. Please see the source code below.

Note: this is a very quick and dirty approach, I didn’t spend too much time writing good code, just wanted to have the photos quickly.

Enjoy it!

#!/usr/bin/python
import RPi.GPIO as GPIO
import sys, os, glob, time
from boto.s3.connection import S3Connection
from boto.s3.key import Key


pin = 7
GPIO.setmode(GPIO.BCM)
GPIO.setup(pin, GPIO.OUT)

AWS_ACCESS = 
AWS_SECRET = 

conn = S3Connection(AWS_ACCESS,AWS_SECRET)
bucket = conn.get_bucket('be.wapptastic')
directory = '/home/pi/Programs/'

def percent_cb(complete, total):
    sys.stdout.write('.')
    sys.stdout.flush()

def getFiles(dir):
	return [os.path.basename(x) for x in glob.glob(str(dir) + '*.jpg')]

def setPinHigh():
	GPIO.output(7, GPIO.HIGH)	

def setPinLow():
	GPIO.output(7, GPIO.LOW)

def upload_S3(dir, file):
	k = Key(bucket)
	k.key = f
        setPinHigh()
	k.set_contents_from_filename(dir + f, cb=percent_cb, num_cb=10)
	setPinLow()

def removeLocal(dir, file):
	os.remove(dir + file)


filenames = getFiles(directory)
print filenames

for f in filenames:
        print 'rnUploading %s to Amazon S3 bucket %s' % (f, bucket)
	upload_S3(directory, f)
        removeLocal(directory, f)

Let’s try the program. Take a photo using the following command:

pi@raspberrypi ~/Programs $ raspistill -o test.jpg

Then invoke our little script and upload it to the requested bucked on S3:

pi@raspberrypi ~/Programs/UploadS3 $ sudo ./upload_s3.py 
./upload_s3.py:10: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  GPIO.setup(pin, GPIO.OUT)
['test.jpg']

Uploading test.jpg to Amazon S3 bucket 
..........
pi@raspberrypi ~/Programs/UploadS3 $

We see now that the photo has been uploaded successfully and can verify it through the Amazon website:

Raspberry Pi: Blink LED when IP Connectivity

The article on blinking LEDs (cfr was a very easy one. Let’s go for another easy one to get used to the Raspberry Pi. The idea is to have a LED burning (turned on) when the RPi has internet connectivity and to turn it off when there is no internet connectivity.

To achieve that, we will send a single PIN command to a known website or IP address. We will then parse the response and when it notices a reply has been received, the LED will be turned on. In all other cases, the LED http://blog.wimwauters.com/wp-admin/post.php?post=542&action=edit&message=1#visibilitywill remain off.

See below for the full code:

#!/usr/bin/python
import requests
import time
import RPi.GPIO as GPIO
import os

pin = 7
GPIO.setmode(GPIO.BCM)
GPIO.setup(pin, GPIO.OUT)
ipaddress = '74.125.230.120'

while True:
        # Perform the ping
        pingFile = os.popen('ping -c 1 %s' % (ipaddress))
        pingData = pingFile.readlines()
        print pingData
        pingFile.close()
        # Check response length
        if len(pingData) < 2:
                # Failed to find a DNS resolution or route
                GPIO.output(pin, GPIO.LOW)
                print 'Ping not successfull'
        else:
                index = pingData[1].find('64 bytes from %s' % ipaddress)
                if index == -1:
                # Ping failed
                        GPIO.output(pin, GPIO.LOW)
                        print 'Ping not successfull'
                else:
                # Ping Ok
                        GPIO.output(pin, GPIO.HIGH)
                        print 'Ping successfull'
        time.sleep(1)

Getting started with PubNub

[keen]

Inspired by this blogpost from my good friend Antonio Mancuso, I decided to experiment also a little bit with PubSub. I know the technology from previous experiences but I have never been able to really take a deep dive into it. As Antonio gave an example based on Ruby, I decided to create an equivalent using Python. Just for fun of course! I believe the below picture from PubNub really explains it very nice. We will implement the ‘broadcast’ variant:

In below snippet, we create the publisher, which is the entity that is pushing messages to the subscribers that are listening to it. Obviously, I’m assuming you have created an account on http://www.pubnub.com.

import datetime
from Pubnub import Pubnub

publish_key = ''
subscribe_key = ''
channel = 'wapptastic'
message = 'Hello Wapptastic'

pubnub = Pubnub(publish_key=publish_key, subscribe_key=subscribe_key)

def callback(message):
   
    print(message)

pubnub.publish(channel, message, callback=callback, error=callback)

Below we create a subsriber which listens to the channel used by the publisher:

import datetime
from Pubnub import Pubnub


publish_key = ''
subscribe_key = ''
channel = 'wapptastic'

pubnub = Pubnub(publish_key=publish_key, subscribe_key=subscribe_key)

def callback(message, channel):
    print(message)


def error(message):
    print("ERROR : " + str(message))


def connect(message):
    print("CONNECTED")


def reconnect(message):
    print("RECONNECTED")


def disconnect(message):
    print("DISCONNECTED")


pubnub.subscribe(channel, callback=callback, error=callback,
                 connect=connect, reconnect=reconnect, disconnect=disconnect)

The results can be seen in below screenshot. Pretty neat service I have to admit.