Skip to main content

Live Edge Detection using Pygame and Numpy

So I thought I would start playing with image processing a bit more. I haven't used pygame before, but a mechanical engineering friend of mine was talking about it so I'm taking a look. I have also been recently following this blog, where the author made a webcam module for pygame for a Google Summer of Code project. So I svn checked out the latest pygame and fired up a few basic examples. I am impressed it's really easy to use! Loading images and making a simple animation, or event driven game is straight forward. I followed a particularly bad tutorial on showmedo, making a simple box that was keyboard event driven around a screen.
Well this is all good and well, a black box I can steer around a white screen. Boy that is cool! :-P
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.




Well sure so not exactly enthralling stuff - it's interesting to see just how easy it was to get to this point. So anyhow the next thing I did was fire up the old ipython terminal and try to get an image from my webcam displayed....



Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:

>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()

So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:

>>> from pygame import surfarray
>>> numpyArray = surfarray.
array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)

Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.

How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.

I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv

This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)


As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...

So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.

Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.








Popular posts from this blog

My setup for downloading & streaming movies and tv

I recently signed up for Netflix and am retiring my headless home media pc. This blog will have to serve as its obituary. The box spent about half of its life running FreeNAS, and half running Archlinux. I’ll briefly talk about my experience with FreeNAS, the migration, and then I’ll get to the robust setup I ended up with.

The machine itself cost around $1000 in 2014. Powered by an AMD A4-7300 3.8GHz cpu with 8GB of memory. A SilverStone DS380 case is both functional, quiet and looks great. The hard drives have been updated over the last two years until it had a full compliment of 6 WD Green 4TiB drives - all spinning bits of metal though.

Initially I had the BSD based FreeNAS operating system installed. I had a single hard drive in its own ZFS pool for TV and Movies, and a second ZFS pool comprised of 5 hard drives for documents and photos.

FreeNAS is straight forward to use and setup, provided you only want to do things supported out of the box or by plugins. Each plugin is install…

Matplotlib in Django

The official django tutorial is very good, it stops short of displaying
data with matplotlib - which could be very handy for dsp or automated
testing. This is an extension to the tutorial. So first you must do the
official tutorial!
Complete the tutorial (as of writing this up to part 4).

Adding an image to a view

To start with we will take a static image from the hard drive and
display it on the polls index page.
Usually if it really is a static image this would be managed by the
webserver eg apache. For introduction purposes we will get django to
serve the static image. To do this we first need to change the
template.



Change the template
At the moment poll_list.html probably looks something like this:


<h1>Django test app - Polls</h1> {% if object_list %} <ul> {% for object in object_list %} <li><a href="/polls/{{object.id}}">{{ object.question }}</a></li> {% endfor %} </ul> {% else %} <p>No polls are available.</p> …

Python and Gmail with IMAP

Today I had to automatically access my Gmail inbox from Python. I needed the ability to get an unread email count, the subjects of those unread emails and then download them. I found a Gmail.py library on sourceforge, but it actually opened the normal gmail webpage and site scraped the info. I wanted something much faster, luckily gmail can now be accessed with both pop and imap.

After a tiny amount of research I decided imap was the better albiet slightly more difficult protocol. Enabling imap in gmail is straight forward, it was under labs.

The address for gmail's imap server is:

imap.gmail.com:993

Python has a library module called imaplib, we will make heavy use of that to access our emails. I'm going to assume that we have already defined two globals - username and password. To connect and login to the gmail server and select the inbox we can do:

importimaplibimap_server=imaplib.IMAP4_SSL("imap.gmail.com",993)imap_server.login(username,password)imap_server.select(…