Embedded Eye

Give your gizmo the gift of sight

HI: I have ordered 2 Stonymen and am now analyzing what I need to make a 3D pair. I have taken downsampling of some pictures and the full resolution is a must. Below that, there is too much of a jump between sections of the picture. So I want about 3 frames a second and all pixels. How fast can I read a pixel, assuming I get great A/D's and 40+ megainstructions per second? The limit may be the camera. Or will I have to use a gate array?

Views: 1197

Reply to This

Replies to This Discussion

OK: I am replying to myself. It seems I can handle the speed with a Microchip PIC18F87K22 computer and an  MB85R1001A nonvolatile Ferromagnetic Parallel Ram (128Kx8) and an AD7825 4 channel 2Meg/sec A/D converter. Total cost about $32 from Digikey.

The computer runs an instruction every 64nsec, so it can pulse the optics with no problem. The A/D converter samples can go directly to RAM, but better to keep a reference level in the RAM and use it immediately to adjust the levels. There is enough RAM for 7 3D snapshots, but the fun will be processing the data in real time. The FRAM can handle 10 to the 10th power read/writes, about 100 years at one scan every 1/3 second. May have to get it on a socket. 

My first application will be to sense 1 light and set a trip distance level set by a potentiometer.

My second application will be lip reading and sending out letters as a best guess as a person talks. The 3D will be useful for puckering and mouthing. I will have to speed up processing by scanning only the mouth after a first capture of where it is. Probably just a 1/4 face view in the camera.

I should probably have some sort of output to show the picture the cameras are using. Doesn't have to be real time. I will work on this, maybe serial port to USB/serial dongle to VB or C program on my laptop.

I will start the PCB when I get the cameras.

Ron

OK: I am still replying to myself. I found this great 128K x 8 nonvolatile RAM made of magnetorestrictive cells. It can be read and written-to forever. It can keep the white level reference permanently for the two eyes and have enough room for storing a few snapshots. I have now ordered all the pieces I need.

But I did buy a $20 LCD graphic display with 160x100 pixels. The interface is I2C, so few pins to worry about. The pixels are 16-gray level, so I will have to adjust averages etc. It looks easy enough to do about 2 screens a second.., something to do with LCD response speed. And it is transflective (it has a backlight but won't need it since it is also reflective)!

Digikey Display #NHD-C160100DIZ-FSW-FBW-ND CONNECTOR #WM1526-ND
Digikey MRAM #819-1027-ND

 

There are two limitations I can think of for acquiring pixels. One is the Stonyman chip itself which reads out at about 1 Mpixel per second (1uS to read out a single pixel) but that value can go up or down depending on various factors. Second is the ADC itself. PICs (I believe) tend to have faster ADCs than Atmels. You can also use an external ADC- recommended to get the fastest speed if using an ATmega type processor or non-ARM Arduino.

The other factor of course is CPU throughput to do the image processing you need for the 2 Stonyman chips. Are you doing stereo?

Thanks. I have finished my artwork but will not send it out til I get the vision breakouts. I am assuming the pins are 0.1 inch apart. The application will be stereo, but directed stereo by identifying points in the optics which can be followed dynamically. My problem is the spacing between the stereo cameras... To get resolution in 3d, I need over 7 inches between the cameras to get about .1 inch depth perception at 8 inches distance.

Parts are:

PIC micro running 16bit code at 16 MegaInstructions/sec
AD7825 4chan A/D @450ns parallel
MRAM magnetorestrictive memory, 128Kbytes parallel
USB serial port dongle interface
160x100 pixel I2C LCD, cheap, slow but good for diagnostics
(later, I2C connection to my robot)
utility pushbuttons, 7 seg LED, red and green led

Thanks for the clarification. Do you mean you will be tracking, say, bright points?

Yes, the headers have a 0.1" pitch.

HI: I intend to identify objects: simple. I really have to evaluate the image data and it's variations with multiple reads. I will start off by tracking 20 or so highlights/lowlights of the picture of one camera, using initial peak points which are larger/smaller than 'x', then spiralling around them to create working areas. This will start off with gradients, but then the fun of looking for shapes. In the lip reading app, I will target the corners of the mouth and the brightness of the nose for registration.  Just have to see how the images look. My little LCD can be used to put squares around points of interest as they are processed.

If the images are lousy, I will project dots on the face with ifr sidelooking laser diodes. They are really cheap and have a 2 degree beam in a TO92 plastic package.

Once done, the two images will be mass-compared to find background and foreground highlights. IE two images will be adjusted for best gross match by modifying the x-y axis in one of the images, then picking out the furthest mismatches to show depth. In that way, gray and unfocussed areas will be accomodated.

Lots to do. Will probably simulate images and software processes on my laptop using bitmaps adjusted to scale 100x100.

Thanks for the pitch info. I can order my boards now.

 

 

HI: Replying to myself again. Is the attached bmp reduced to 3 bit intensity a typical image? Remember that I will take a whitescale image at first development to subtract from the image. Would I have to do this for whitescale and darkscale? Probably also need a bit of side-to-side bit normalizing.

 

ANYWAY: is this an image I can use for simulation?

HI: I wonder if I should keep adding to this? My thoughts this morning...

So. The eye sensors are not fooled by flickering lamps. In fact they will see the 60hz especially from LED room lighting. I think my two forward looking LED's will have to be superbright oranges for illumination... and I will use a power FET drive with current limit directly driven from the raw power supply. This will handle the lip reading app.

As for apps that can only see room lights, I will have to sense the 120Hz and sample the lamps only for 1msec per half cycle at peak light. This slows things down a bit.

Say 2us per pixel for a full dumb scan of 100x100 pixels = 20 msec total time bringing a scan to 10 cycles of the 60 per second line frequency. 1/6 second! People move faster than that.

My first test program will see how well the eye can be 'corrected' for light levels outside the peak of AC driven lamps. The computer can scan one pixel for 16.67msec and create a light profile surrounding the peaks (can test this on a photosensor and my oscilloscope). Then apply this to the 'whitescale' program compensation which I have to use anyway.

Down to the basement now.

So... interesting. LED non-dimmables show the full AC waveform..full on to full off. That kills any attempt at out-of-peak compensation!

Compact Fluorescents have a flattened 60 Hz, but with a non-sinusoidal output which stays within 10% of the peak. And there is significant 50Khz on the CF signal giving 20usec doubt. But the real winner is a buzzing fluorescent 4 footer. It's waveform is like a mountain range. But there is still a peak stability of 1 msec every 8msec, but it is not at the AC line peak.

Of course incandescents do 60Hz but never to zero like the LED nondimmable.

Conclusions:

1) 1 msec per 8.33 msec is stable enough but has to be sampled in real time to determine the location of the peak.

2) 1% stability in the electronic eye's view will be impossible. 8 bit a/d's are probably overdoing it for on-the-fly processing.

3) Household LED lighting will eventually destroy the pureness of 60 cycles.

Suggestions: Fill the house/factory with non-dimmable DC driven LED lighting from a DC power supply... or work outside.

In the meantime, my program will synch to AC light and stay locked onto it. I should probably have a single sensor connected to my computer to sense the ambient light waveform and a/d it at the same time as the eyes. Yes. I will do that, but it should have a 20 degree beam with no translucent filter and aim where the cameras aim. This way, it will see average light on the scene which is less likely to physically change over 1/120 of a second. So now I will have 3 a/d samples every 2 usec, eye1, eye2 and average. The blanking time for the non-1msec periods can be used to apply the white correction, the average number and also to sense the waveform to keep in synch. I love assembler code... I can run a routine to exactly time everything during the scan without the need of interrupts. Interrupts can only be used as a) the time synched program or b) emergencies. With many threads, uncontrolled interrupts become a nightmare in realtime apps. I have written many programs that only run in interrupts... but don't ask me to change them. Like when the lighting changes locale and the synch period alters... no timed interrupt will handle that easily.

I should probably get a faster a/d converter. The present 4 channel one is 450nsec. Just enough.. barely.

 

This is one aspect of the "direct stare" nature of the Stonyman pixels- when you sample them you get the pixel value at that exact time instance. Linear pixels integrate light over a fixed interval so they are somewhat immune to that e.g. they integrate over one or more 120Hz cycles so the variation gets for the most part eliminated. Of course the frame rate slows as a result which affects some applications.

If you are focusing on using gradients however then this effect can be reduced- the ambient light levels change very little between the acquisition of two pixels, especially if they are horizontally adjacent. One thing about logarithmic pixels- If two adjacent pixels are "10 quantization units" apart at one ambient light level, then they will be about that far apart at a different ambient light level.

That looks like a typical image you can use for simulation. The contrast coming off the Stonyman chips, when displayed in a pure raw form, can appear washed out due to the logarithmic pixel nature. To help with the display we often rescale the pixels so that the darkest is "black" and the brightest "white".

Hi, ur project really interesting! Pls keep us update of your project progress and status, I would really keen to follow your project.

BTW, where do u find cheap 2 degree beam laser diode as you mentioned?

HI: the laser diode is About $6. But the ones available are 6 degrees, not 2. Seven years ago I was designing a tubidity probe. The laser beam was too thin believe it or not.. Not enough particles to reflect back!
Optek OPV382. Digikey 365-1146-ND.

RSS

© 2022   Created by Geoffrey L. Barrows.   Powered by

Badges  |  Report an Issue  |  Terms of Service