Skip to main content

Live Edge Detection using Pygame and Numpy

So I thought I would start playing with image processing a bit more. I haven't used pygame before, but a mechanical engineering friend of mine was talking about it so I'm taking a look. I have also been recently following this blog, where the author made a webcam module for pygame for a Google Summer of Code project. So I svn checked out the latest pygame and fired up a few basic examples. I am impressed it's really easy to use! Loading images and making a simple animation, or event driven game is straight forward. I followed a particularly bad tutorial on showmedo, making a simple box that was keyboard event driven around a screen.
Well this is all good and well, a black box I can steer around a white screen. Boy that is cool! :-P
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.




Well sure so not exactly enthralling stuff - it's interesting to see just how easy it was to get to this point. So anyhow the next thing I did was fire up the old ipython terminal and try to get an image from my webcam displayed....



Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:

>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()

So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:

>>> from pygame import surfarray
>>> numpyArray = surfarray.array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)

 Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.

How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.

I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv

This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)


As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...

So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.

Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.








Popular posts from this blog

Python and Gmail with IMAP

Today I had to automatically access my Gmail inbox from Python. I needed the ability to get an unread email count, the subjects of those unread emails and then download them. I found a Gmail.py library on sourceforge, but it actually opened the normal gmail webpage and site scraped the info. I wanted something much faster, luckily gmail can now be accessed with both pop and imap. After a tiny amount of research I decided imap was the better albiet slightly more difficult protocol. Enabling imap in gmail is straight forward, it was under labs. The address for gmail's imap server is: imap.gmail.com:993 Python has a library module called imaplib , we will make heavy use of that to access our emails. I'm going to assume that we have already defined two globals - username and password. To connect and login to the gmail server and select the inbox we can do: import imaplib imap_server = imaplib . IMAP4_SSL ( "imap.gmail.com" , 993 ) imap_server . login ( use...
Read more

Bluetooth with Python 3.3

Since about version 3.3 Python supports Bluetooth sockets natively. To put this to the test I got hold of an iRacer from sparkfun . To send to New Zealand the cost was $60. The toy has an on-board Bluetooth radio that supports the RFCOMM transport protocol. The drive  protocol is dead easy, you send single byte instructions when a direction or speed change is required. The bytes are broken into two nibbles:  0xXY  where X is the direction and Y is the speed. For example the byte 0x16 means forwards at mid-speed. I was surprised to note the car continues carrying out the last given demand! I let pairing get dealt with by the operating system. The code to create a  Car object that is drivable over Bluetooth is very straight forward in pure Python: import socket import time class BluetoothCar : def __init__ ( self , mac_address = "00:12:05:09:98:36" ): self . socket = socket . socket ( socket . AF_BLUETO...
Read more

Matplotlib in Django

The official django tutorial is very good, it stops short of displaying data with matplotlib - which could be very handy for dsp or automated testing. This is an extension to the tutorial. So first you must do the official tutorial! Complete the tutorial (as of writing this up to part 4). Adding an image to a view To start with we will take a static image from the hard drive and display it on the polls index page. Usually if it really is a static image this would be managed by the webserver eg apache. For introduction purposes we will get django to serve the static image. To do this we first need to change the template. Change the template At the moment poll_list.html probably looks something like this: <h1>Django test app - Polls</h1> {% if object_list %} <ul> {% for object in object_list %} <li><a href="/polls/{{object.id}}">{{ object.question }}</a></li> {% endfor %} </ul> {% else %} <p>No polls...
Read more