Exploring Vision Options

As part of the Pokey refit for the next Firefighting contest, better flame detection is a must. Imaging seems the most reliable solution.  The NXTcam was probably a large part of Physignathus' victory in the first Fort Collins Robot Firefighting Contest.  The CMUcam is popular as well.  But both are too expensive for me.

Cheap Camera Options
Pokey is supposed to be a budget/DIY robot.  A couple weeks ago, I was considering a DIY vision system built around a cheap, poorly documented CMOS camera, but gave it up as too ambitious.

Three remaining options best fit within budget and complexity constraints. Parallax has a 1d vision sensor (picture at right from http://www.parallax.com/) that captures a line at a time. By scanning across an area, one can reconstruct the complete image.  Cost is considerably less than a 2d sensor at around $50 (and already I have spare servos).

One question arises: do I need to add an encoder like the one available from Acroname or this one from Zero One Mechatronics or make my own? Is a single servo step a small enough angle to reconstruct a complete image or is gear reduction needed for smaller steps?

AVRcam picture at http://www.jrobot.net/

The second option is to buy a $100 AVRcam kit which is simply a matter of assembling and then using.  No reinventing the wheel, but not a lot of learning about computer vision, either.  It may be worth the cost to save time.  The vision system is totally open source, so future tinkering is entirely possible.

The third option is to use a black and white Game Boy camera.  I've got one on order (Scratch that) One just arrived from eBay! There are several articles floating around about using this sensor. One in particular discusses interfaces to an AVR and external ADC. The camera can remain fixed on the robot and the robot can crudely scan the room as it is moving.  However, implementation will be time consuming and complex.

Vision Processing Power
Video takes a lot of memory and processor speed.  The Game Boy camera is based on a Mitsubishi M64282FP CMOS image sensor that, much like a human retina, has built in edge detection which is an amazing feature that offloads some intense image processing.  It's only a 16KP -- that's right kilo-pixel -- camera: 128x123 pixels.

Even so, a maximum 30fps frame rate still puts a lot of demand on a mere MCU.  The sensor is a serial device, outputting an analog value one pixel at a time.  Maximum frame rate requires a 500KSPS (thousand samples per second) which is far beyond what most AVRs can provide with their built-in ADCs.

How much is really needed?  For now, at least, simply taking a couple of still pictures might be enough to detect the candle flame, with a low frame rate to point the robot at the flame and drive to it accurately.  So maybe 10fps is enough.  Or less?

If one is to process just two entire frames in memory, one needs around 32K of RAM. There's probably little reason to process more than this yet.  And maybe I can come up with some memory-saving tricks, like doing feature detection on the fly without storing the entire image.

Which MCU?
So what processor to use?  Again, think low budget.  Otherwise I should just get a CMUcam and be done with it.  The ATmega8515, ATmega32, 64, and 128 can be hooked to as much as 64K of external SRAM. The AVR32 chips support 32K and 64K of internal ram. Sticking with AVR would save time. No new development environment, no new language. They're cheap, very few components to get one going, and no new serial programmer hardware to buy.

An external ADC could sample much faster than the AVR.  I've got a couple of candidate ADCs I want to look at, one serial, one parallel. I'm leaning towards parallel as I think it'll be simpler to interface from a timing standpoint.

I've never looked at PIC processors before but there may be a couple of options there. For example, the PIC32XX3XX/4XX family runs at 80MHz, has up to 32K RAM and 1000kHz ADC sample rate.  But, it would have the disadvantage of a new chip, new IDE, new flavor of C, etc.  And I'd need to get a big TQFP breakout board.

Another option is to use a Parallax Propeller which runs at 80MHz, has 32K RAM and a little research suggests it may support fast ADC rates at lower resolutions (EDIT: the Propeller doesn't include an on-board ADC hardware peripheral, however it is possible to do 1-bit sigma-delta conversion). And it has parallel processing. It comes in a through-hole version as well as TQFP but requires several support components, particularly a serial EEPROM and a special usb-to-serial programmer. The unusual chip and its entirely new language would be very unfamiliar territory. Figure $40 ($23 as of Nov 15, 2011) for a Schmartboard development board, and try to hack one of my two usb-to-serial programmers for use with the Propeller. But, it runs about 160MIPS and does true, deterministic, real time parallel processing with 8 cores. That's a powerful argument.

Software
I'm not quite sure what the heck to do about software so more learning and experimenting is required there. This is one of those time vs money trade-offs -- with more investment I would save myself all the time of building circuits and software.  But I wouldn't learn as much, either.

At any rate, the current plan is to prototype some algorithms on a PC or Mac using simulated candle images: pictures of a candle in various situations, re-sized to 128x123 pixels to see how feasible this really is.  More on that in a later article.