Skip to main content

Live Edge Detection using Pygame and Numpy

So I thought I would start playing with image processing a bit more. I haven't used pygame before, but a mechanical engineering friend of mine was talking about it so I'm taking a look. I have also been recently following this blog, where the author made a webcam module for pygame for a Google Summer of Code project. So I svn checked out the latest pygame and fired up a few basic examples. I am impressed it's really easy to use! Loading images and making a simple animation, or event driven game is straight forward. I followed a particularly bad tutorial on showmedo, making a simple box that was keyboard event driven around a screen.
Well this is all good and well, a black box I can steer around a white screen. Boy that is cool! :-P
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.




Well sure so not exactly enthralling stuff - it's interesting to see just how easy it was to get to this point. So anyhow the next thing I did was fire up the old ipython terminal and try to get an image from my webcam displayed....



Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:

>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()

So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:

>>> from pygame import surfarray
>>> numpyArray = surfarray.
array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)

Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.

How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.

I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv

This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)


As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...

So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.

Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.








Comments

  1. Hehe thanks! I am thinking I need to have another play with pygame + webcam - maybe less math and more game next time tho!

    ReplyDelete

Post a Comment

Popular posts from this blog

Matplotlib in Django

The official django tutorial is very good, it stops short of displaying
data with matplotlib - which could be very handy for dsp or automated
testing. This is an extension to the tutorial. So first you must do the
official tutorial!
Complete the tutorial (as of writing this up to part 4).

Adding an image to a view

To start with we will take a static image from the hard drive and
display it on the polls index page.
Usually if it really is a static image this would be managed by the
webserver eg apache. For introduction purposes we will get django to
serve the static image. To do this we first need to change the
template.



Change the template
At the moment poll_list.html probably looks something like this:


<h1>Django test app - Polls</h1> {% if object_list %} <ul> {% for object in object_list %} <li><a href="/polls/{{object.id}}">{{ object.question }}</a></li> {% endfor %} </ul> {% else %} <p>No polls are available.</p> …

Homomorphic encryption using RSA

I recently had cause to briefly look into Homomorphic Encryption, the process of carrying out computations on encrypted data. This technique allows for privacy preserving computation. Fully homomorphic encryption (FHE) allows both addition and multiplication, but is (currently) impractically slow.

Partially homomorphic encryption just has to meet one of these criteria and can be much more efficient.
An unintended, but well-known, malleability in the common RSA algorithm means that the multiplication of ciphertexts is equal to the multiplication of the original messages. So unpadded RSA is a partially homomorphic encryption system.

RSA is beautiful in how simple it is. See wikipedia to see how to generate the public (e, m) and private keys (d, m).

Given a message x it is encrypted with the public keys it to get the ciphertext C(x)with:

C(x)=xemodm
To decrypt a ciphertext

Bluetooth with Python 3.3

Since about version 3.3 Python supports Bluetooth sockets natively. To put this to the test I got hold of an iRacer from sparkfun. To send to New Zealand the cost was $60. The toy has an on-board Bluetooth radio that supports the RFCOMM transport protocol.



The drive protocol is dead easy, you send single byte instructions when a direction or speed change is required. The bytes are broken into two nibbles: 0xXY where X is the direction and Y is the speed. For example the byte 0x16 means forwards at mid-speed. I was surprised to note the car continues carrying out the last given demand!

I let pairing get dealt with by the operating system. The code to create a Car object that is drivable over Bluetooth is very straight forward in pure Python:

importsocketimporttimeclassBluetoothCar:def__init__(self,mac_address="00:12:05:09:98:36"):self.socket=socket.socket(socket.AF_BLUETOOTH,socket.SOCK_STREAM,socket.BTPROTO_RFCOMM)self.socket.connect((mac_address,1))def_write(self,data_byte):…