Skip to main content

Live Edge Detection using Pygame and Numpy

So I thought I would start playing with image processing a bit more. I haven't used pygame before, but a mechanical engineering friend of mine was talking about it so I'm taking a look. I have also been recently following this blog, where the author made a webcam module for pygame for a Google Summer of Code project. So I svn checked out the latest pygame and fired up a few basic examples. I am impressed it's really easy to use! Loading images and making a simple animation, or event driven game is straight forward. I followed a particularly bad tutorial on showmedo, making a simple box that was keyboard event driven around a screen.
Well this is all good and well, a black box I can steer around a white screen. Boy that is cool! :-P
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.




Well sure so not exactly enthralling stuff - it's interesting to see just how easy it was to get to this point. So anyhow the next thing I did was fire up the old ipython terminal and try to get an image from my webcam displayed....



Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:

>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()

So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:

>>> from pygame import surfarray
>>> numpyArray = surfarray.array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)

 Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.

How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.

I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv

This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)


As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...

So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.

Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.








Popular posts from this blog

Driveby contribution to Python Cryptography

While at PyConAU 2016 I attended the Monday sprints and spent some time looking at a proposed feature I hoped would soon be part of cryptography . As most readers of this blog will know, cryptography is a very respected project within the Python ecosystem and it was an interesting experience to see how such a prominent open source project handles contributions and reviews. The feature in question is the Diffie-Hellman Key Exchange algorithm used in many cryptography applications. Diffie-Helman Key Exchange is a way of generating a shared secret between two parties where the secret can't be determined by an eavesdropper observing the communication. DHE is extremely common - it is one of the primary methods used to provide "perfect forward secrecy" every time you initiate a TLS connection to an HTTPS website. Mathematically it is extremely elegant and the inventors were the recipients of the 2015 Turing award . I wanted to write about this particular contribution becau...
Read more

Matplotlib in Django

The official django tutorial is very good, it stops short of displaying data with matplotlib - which could be very handy for dsp or automated testing. This is an extension to the tutorial. So first you must do the official tutorial! Complete the tutorial (as of writing this up to part 4). Adding an image to a view To start with we will take a static image from the hard drive and display it on the polls index page. Usually if it really is a static image this would be managed by the webserver eg apache. For introduction purposes we will get django to serve the static image. To do this we first need to change the template. Change the template At the moment poll_list.html probably looks something like this: <h1>Django test app - Polls</h1> {% if object_list %} <ul> {% for object in object_list %} <li><a href="/polls/{{object.id}}">{{ object.question }}</a></li> {% endfor %} </ul> {% else %} <p>No polls...
Read more

Python and Gmail with IMAP

Today I had to automatically access my Gmail inbox from Python. I needed the ability to get an unread email count, the subjects of those unread emails and then download them. I found a Gmail.py library on sourceforge, but it actually opened the normal gmail webpage and site scraped the info. I wanted something much faster, luckily gmail can now be accessed with both pop and imap. After a tiny amount of research I decided imap was the better albiet slightly more difficult protocol. Enabling imap in gmail is straight forward, it was under labs. The address for gmail's imap server is: imap.gmail.com:993 Python has a library module called imaplib , we will make heavy use of that to access our emails. I'm going to assume that we have already defined two globals - username and password. To connect and login to the gmail server and select the inbox we can do: import imaplib imap_server = imaplib . IMAP4_SSL ( "imap.gmail.com" , 993 ) imap_server . login ( use...
Read more