So I thought I would start playing with image processing a bit more. I haven't used pygame before, but a mechanical engineering friend of mine was talking about it so I'm taking a look. I have also been recently following this blog, where the author made a webcam module for pygame for a Google Summer of Code project. So I svn checked out the latest pygame and fired up a few basic examples. I am impressed it's really easy to use! Loading images and making a simple animation, or event driven game is straight forward. I followed a particularly bad tutorial on showmedo, making a simple box that was keyboard event driven around a screen. 
Well this is all good and well, a black box I can steer around a white screen. Boy that is cool! :-P
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.
Don't despair, there is more to come, after this dismal tutorial I went to pygame's site - funnily enough they have introductory tutorials on using pygame and they were not quite so boring! So program 2 was loading an image of a ball and letting it bounce around the screen as shown.
Well sure so not exactly enthralling stuff - it's interesting to see just how easy it was to get to this point. So anyhow the next thing I did was fire up the old ipython terminal and try to get an image from my webcam displayed....
Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:
>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()
So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:
>>> from pygame import surfarray
>>> numpyArray = surfarray.array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)
Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.
How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.
I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv
This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)
As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...
So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.
Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.
Pretty impressed that 14 lines of code can get an image of me typing the 12th line displayed on my screen! The very observant will realize that line 12 of my screen-shot produces an error! It should have been
snapshot = camera.get_image()
Then the last two lines for completeness are:
>>> display.blit(snapshot,(0,0,)) <rect(0, 0, 640, 480)> >>> pygame.display.flip()
So now to use some other tools with this! So first up I wanted a numpy array of this image. Turns out pygame itself solves this one:
>>> from pygame import surfarray
>>> numpyArray = surfarray.array3d(snapshot)
>>> type(numpyArray) <type 'numpy.ndarray'> >>> numpyArray.shape (640, 480, 3)
Well now that was easy! So now I can do image processing using scipy's signal processing toolbox on data recieved via pygame's camera module.
How about doing that live? First I found in the pygame examples folder a camera.py file. This simply displays the live video feed in pygame and outputs the frame rate to the terminal. When I run that on my pc at hitlab I get an average of 73 frames per second.
I used that as my base and added in an optional edge detection mode. When enabled this converts the image from a pygame surface to a numpy array as above. Then it calls this function:
def edgeDetect1(imageArray): laplacian = numpy.array([[0,1,0],[1,-4,1],[0,1,0]]) deriv = signal.convolve2d( \ imageArray[:,:,0],laplacian,mode="same",boundary="symm") return deriv
This carries out edge detection on only ONE of the RGB pixel arrays (I assume red...?)
As you can see I was excited about this! I felt this was a very good start!
It wasn't all rosy however, the FPS went down. This was expected but alarming as to how much... The new result was 3.3fps. Now this was a bit hit so I did some re con into what was slowing this down, and the result surprised me. It wasn't the convolve line. It was the converting to a numpy array and back...
So instead of using surfarray, I had a nosy around the pygame docs - sure enough there is a pygame.transform.laplacian.
So using that directly on the surface captured, gave a way cooler live feeling cause it was not too laggy at 15 fps.
Hmm I should get back to my work report now... I would like to see a fast way of getting the data into numpy tho.