Harvest Imaging Blog

It has been a while since the last post in the series “How To Measure … ?”  Unfortunately the writing-up of this material has not the highest priority I have to admit.  Another problem has to do with the need for new/additional measurement data, but to generate the data I do need a set-up, and to use the set-up I do need a lab.  At this moment I am busy with the installation of the lab in my new office space, some new equipment has arrived already, including a nice light-tight measurement box that will be used to do the measurements.  So in principle I can start with new measurements on a short notice (if time allows me).  But in many cases after the measurement data is available, some data processing is needed to present the results in an understandable way. 

What can you expect to be the next ?  Most probably the next chapter will deal the measurement of the linearity and/or full-well capacity of the sensors/cameras.  Looking forward to it, because it is funny to realize how much one (still) can learn by doing these evaluations.

Albert. 28-01-2013.

Together with my colleague Edoardo Charbon, we developed a new course on Imaging Sensors.  The course is intended for students of the first year of our international EE MSc program.  We had 28 students which chose to follow the class.  Quite a nice success.

Why do I put this message in my blog ?  Well to tell you about our examination procedure.  Instead of asking the students to learn all kind of definitions by heart, we gave them a list of about 50 technical publications, material that came mainly from IEEE and IISW.  We asked to all students to select one publication, study it, make a poster of it and then explain the material by means of the poster to the two professors and their colleague students.

We just had the poster sessions last week, and overall the students did very well.  It is amazing to see how easily these young people pick up the ideas presented in the publications and how they explained it to the audience.  A big advantage of this way of examination is the fact that you learn something from it yourself.  It is really funny to see your own ideas explained by others in a way that is completely different from the way I would have done it myself.

So in conclusion I think it was a very successful experiment, worthwhile to repeat in the coming years.

Albert, 15-01-2013.

Time flies, almost at the speed of the old Concorde airplane, 2012 is almost over ….  So like every year I would like to take the opportunity to wish all my blog readers a Merry Christmas and a Happy New Year.  You are with a lot of people who are visiting my web-site every day.  In February 2012, there was a new record established : 856 unique visitors on one single day.  This is really incredible, especially the short conference reports that I am posting attract a lot of readers.  Thanks for stopping by !!!

Looking back to 2012, it was again a great year for Harvest Imaging.  The training activities continued to be successful.  Although a few scheduled courses had to be cancelled, about 90 % of the scheduled events could take place.  I am very happy with this high hit rate, taking into account the number of courses I am running every year and taking into account the economical issues our industry is facing.

In 2012 I had the opportunity to attend again a couple of interesting conferences and to meet a lot of enthousiastic imaging fanatics.  Unfortunately I had to abstain from the Electronic Imaging conference because of obligations with a PhD defense.  This was really a bad coincidence, because I was honoured with the Award of Electronic Imaging Scientist of the Year.  After all not a bad coincidence for my daughter who went to San Francisco to replace me and to pick up the award.  Remarkably successful was the Image Sensor Europe conference held in London (UK).  A great set of speakers, attractive topics and a nice new location.  I am curious to see whether the ISE2013 organizers can keep up the same quality level as in 2012, hopefully they will.

Two smaller Fraunhofer workshops in Germany were attended : the CMOS workshop in Duisburg and the Microoptics Imaging and Projection in Jena.  The Duisburg one was organized already for the 6^th or 7^th time, for the Jena one, it was the 1^st time.  I for sure enjoyed attending the latter, a bit off topic for me, but a lot of new things to learn.

Finally at the end of 2012 my new office became available.  During the Christmas holidays I will move my stuff out of my own house into the brand new office space.  I do hope that the new location will stimulate the creativity as much as my office space in our living room did.

To conclude, I wish all of you the very best for 2013, and hope that we will regularly “meet” through this blog.  2013 is announcing itself as going to be a very interesting year, because he is coming to Europe !!!  The Summer of 2013 will be hot, because Neil saddled up the Horse and is going to make a tour through the old continent.

Thanks for visiting the website of Harvest Imaging, hopefully see you next year again 😉

Albert, 19-12-2012.

The morning sessions of the third Symposium day concentrated further on the technology of microoptics.  Pierre Craen (poLight, Norway) gave a talk about a new auto-focusing system for mobile phone applications.  The technology is MEMS based : a polymer is sandwiched between two glass surfaces.  The lower one is a rigid glass plate, the top one is a glass membrane which is deformable through a piezofilm (driven at 20V).  The deformation of the glass membrane is transformed into the polymer and in this way a deformable lens can be created (looks a bit similar to the fluid lens of Varioptic, but now with glass and polymer instead of water and oil). 

During the presentation several measurements and data were shown.  What I could grab : 1 ms reaction time, 5 mW power consumption, very small, very thin (0.4 mm thickness), transmission > 95 %, diffraction limited, re-flowable at 260 deg.C, wafer scale technology (8”) and a wide temperature range (- 40 deg.C to 200 deg.C).  Also mentioned was the limitation of the technology to small apertures (up to 1.7 mm, corresponding to maximum 1/3” or 1/2.5” image sensors).  The technology is named : Tunable Lens or TLens.

Is this technology capable of kicking the VCMs ou of the mobile phones ?  According to Eric Mounier (Yole Developpement, France) VCMs still have 95 % market share.  Nice opportunity for poLight, but also nice challenge for the TLens.    

Albert, 29-11-2012.

Here is a quick overview of the second day, mainly devoted to the technology of the micro-optics.

Stephan Heimgartner (Heptagon) started the day with a talk about wafer-level micro-optics for computational imaging.  He highlighted the technology of Heptagon, ranging from wafer-level optics (= lenses on a glass wafer of 8”) to wafer level packaging of these lenses.  The most complex wafer level packaging technology does include 4 lenses (= 2 wafers with lenses on both sides), 2 spacer structures and an IR cut-off filter.  This stack of optical elements is used today on low-resolution sensors.  For the higher Mpix sensors this technology is not suitable.  The reason for this is the limitation that is defined by the tolerances of all the various materials and structures involved.

In the second half of the talk Stephan explained the Heptagon technology for multi-aperture cameras.  Remarkable is the location of the colour filters : on top of the micro-optical stack.  Also intriguing is the back focus adjustment of the structures that can be done after the lens stack is completed.  During the talk a prototype of a 2 x 2 multi-aperture camera was shown built on a 2M pixel sensor.

Zouhair Sbiaa (Nemotec) more or less confirmed what was already told by the previous speaker.  The optical modules built by means of the wafer-level technology are limited to two wafers due to tolerances.  Zouhair showed a proto-type of a micro-optical component on top of a HD 720p sensor.  This optical module was individually place on top of the sensor.

Although not indicated in the program, Steven Oliver (Lytro) gave a talk about their light field camera.  The talk started and ended with some marketing stuff, but in between there were some very interesting slides shown.  With the light field camera, more freedom can be generated in (after-)focusing of the image, but also some freedom in perspective view is possible.  Playing around with filters (in the software) can add more features to the image, also great was the demo with the movement of light and shadows.

More on the technical side : the 3.0 camera (intended for social media) has a high quality lens included (8x magnification, F2).  Just in front of the sensor, a micro-lens array is placed.  The latter had 330 x 330 micro-lenses, arranged in a hexagonal grid.  The pitch of the micro-lenses is 13.9 um, and every micro-lens is covering 10 x 10 pixels of the sensor.  The sensor itself is a 14M pixel device, finally cropped into 11M pixels.

Flavien Hirogoyen (ST Microelectronics) gave us literally a deep insight in the pixels by showing great results about his simulations of the optical characteristics of the CMOS pixels.  The pixels were 1.45 um in size, and it is remarkable how good the FTDT (Finite Difference Time Domain) simulations fit to the measured data of quantum efficiency.  The speaker showed results for monochromatic light as well as for white light of different colour temperatures.

Andreas Spickermann (IMS Fraunhofer, Duisburg) concluded the morning session with an overview of what his company is able to do in the field of CMOS image sensors.  Too much actually to list here, but worthwhile to mention : IMS Fraunhofer have their own 0.35 um opto process with pinned-photodiodes, colour filters and micro-lenses.  Besides many others, an interesting option is the use of a nitrogen-enriched Si3N4 layer for the passivation.  The layer has a relatively larger transmission in the region above 200 nm wavelength.

Palle Dinesen (Kaleido) kicked off in the afternoon with an all-glass solution approach for micro-optics.  He gave a nice overview of all issues involved with the making of the tools to mold the glass lenses.  Glass has the advantage over plastic to be much thinner, less sensitive to temperature variations .  The know-how of Kaleido is situated in the making of the tools (by a grinding method) as well as in the molding process.  What could be understood : the material is pre-heated before it enters in the molding chamber.  The molder chamber is kept at a relative high temperature, and after the molding the end-result gets a specific cooling-down treatment.  The formation of the lenses on two sides of the carrier is done in one step.  The technology was demonstrated by means of a proto-type on 2”, and now the technology is expanded to 4”.  Mass production will start in Q3/2013.

The final presentation came from Reinhard Voelkel (Suss Microoptics) who tried to give us an answer to the question : “How many channels do you need in an array ?”.  To get an answer on this question, the speaker checked what is present in nature.  Conclusion : or 2 (there were many examples in the room), or 8 (some spiders seem to have 8 eyes) or many (insects).  So draw your conclusion ….

One final word about the organization of the symposium : perfect !  The organization even distributed rain coats to the participants.

Albert, 28-11-2012.

Today, November 27th, the brand new Symposium on Microoptical Imaging and Projection started at the Fraunhofer Institute in Jena (Germany). Due to my late arrival at the Symposium and my lack of knowledge in projection, I only attended 7 talks, of which 5 were on imaging. Most of the imaging stuff today was about multi-aperture cameras. Unfortunately I have to report that not that much new stuff was presented in the first two talks. The presentations of Pelican Imaging (here given by Jacques Duparre) and the one from LinX Imaging (given by Ziv Attar) were repetitions of the ones I heard before. What I memorized from Ziv’s talk is the fact that a multi-aperture camera still has some challenging issues to solve. To name a few : manufacturability of the optics, packaging, sensor compatibility, image processing, lack of standards, processing power needed, and power consumption. After the talk, Ziv advised me to be positive, so here is a list of advantages of a multi-aperture camera : low height (Z dimension), zero colour cross-talk, simple colour filter technology, very simple colour correction matrix, depth sensing, extremely wide depth of focus, wide viewing angle, lack of auto-focusing system up to 10 M pixels, fully independent control of each camera in the array, wide dynamic range, … . Maybe I am still forgetting some.

Interesting was the work reported by Andreas Brueckner (Fraunhofer Institute, Jena). He presented a multi-aperture camera based on a regular 2D CMOS image sensor of 3M pixels provided with a dedicated lens array to make a multi-aperture camera out of it. Andreas presented also some images as well as numerical data. An engineer likes to see numbers (although Neil said “Numbers add up to nothing”). At the end of the talk, Andreas announced that they are working on a much higher resolution imager than the one used today.

Next was the talk of Edward Dowski (Ascentia Imaging, Boulder, CO), who is added a coding grid on top of the multi-aperture cameras, e.g. to depolarize the incoming light. Different apertures can be coded in a unique way relative to other channels in multi-aperture system. This enables in many applications the location estimation of general objects to sub-pixel precision.

The last multi-aperture solution was presented by Guillaume Druart (ONERA, Palaiseau, France) in which he is using the device for IR sensing ! A regular IR sensor is foreseen by an array of 4 x 4 lenslets, which allows the focal length of the individual lenslets to be 4 times less than a regular lens in front of the full resolution device. The lenslets are placed and/or designed such that the 16 sub-arrays do not “see” the same information. So out of the 16 low resolution images, a single high resolution end result is constructed. Nice video of moving image concluded the talk. See you tomorrow through a multi-aperture camera ?

Albert, 27-11-2012.

The conversion gain of an imager or imaging system is linking the output to the input. In the good old days of the CCDs it was simply the ratio of the voltage variation at the source follower output versus the amount of electrons supplied to the floating diffusion. The output voltage variation could be measured by means of an oscilloscope (good old days !).  And the amount of charge supplied to the floating diffusion could be characterized by measuring the reset drain current. A simple, but efficient method, because in most CCDs the reset drain voltage has/had a separate connection and is/was not connected to the supply of the source follower.

With today’s CMOS devices that is different. The in-pixel source follower and in-pixel reset transistor have a common connection, and separate measurement of the reset drain current is no longer possible. But there is still the Photon Transfer Curve (PTC) that can help to characterize the conversion gain of the complete CMOS imaging chain : how many volts or how many digital bits do we get out for every electron generated and transferred to the in-pixel floating diffusion ? Of course I do realize that we have spent a lot of time and blogs on the PTC which can be used to measure the conversion gain. I will not repeat all that great stuff over here. The various PTC options to obtain the conversion gain are the shot noise method (= noise versus effective signal) and the mean-variance method.

Besides the CCD reset-drain method and the PTC method, a third possibility exists to characterize the conversion gain. Although with today’s safety rules, this method is no longer that popular. But using a radio-active Fe55 isotope in front of the sensor can do the job. Fe55 has an energy of 5.9 keV and is generating in silicon about 1620 electrons. In the case the sensor has large pixels and the radiation source is kept “far away” from the sensor, the chance is pretty large that some pixels are hit by a single X-ray photon and most of them are not hit at all by the incoming X-rays. In this way some (large !) pixels will nicely collect all and just all 1620 electrons generated by a single incoming X-ray photon. Simple and efficient !

Albert, 15-11-2012.

Yesterday I quickly visited the Vision 2012 show in Stuttgart.  The buzz words at the show were 3D cameras, high speed, USB3.0.  On sensor level, there were four items that really impressed me :

–          A curved CCD sensor at the booth of Andanta.  There were already rumours that people are working on curved sensors and I was contacted a few years ago to prepare an R&D proposal for curved sensors.  But now it is the first time that I saw one.  The sensor was bent in two (!) directions with a curvature radius of 500 mm.  The device has 16 M pixels, with a size of 6 cm x 6 cm.  The packaged device was mounted on a PCB, but no working devices were shown,

–          A 300 mm wafer with full-size CMOS imagers (36 mm x 24 mm) at the booth of CMOSIS.  They displayed a complete 300 mm wafer with the sensor that is fabricated for the Leica M-camera.  Taking into account that I started my career with wafers of 50 mm ( 2 inches) diameter ….,

–          A global shutter HDR sensor at the booth of New Imaging Technology.  Knowing their pixel architecture I think it is a major achievement to incorporate a global shutter mode in the sensor.  NIT had a working camera to show to the visitors,

–          Another remarkable observation : when passing by the booth of Kappa, my eyes caught an older gentleman who was explaining the product portfolio of Kappa.  He had a CCD attached to the collar of his jacket, and I immediately recognized the device.  Maybe I am the only one at the whole Vision show that has some connections to it :  it was an NXA1011 of Philips, the one I designed in 1983 when I started working at Philips Research.  This imaging world is really a small world !

Albert, 7 November 2012.

In this discussion, dead pixels are defined as pixels that do not react to light at all.  There are two types of dead pixels :

–       Stuck at “1”, or pixels that are always saturated (white defects),

–       Stuck at “0”, or pixels that are always empty (black defects).

The location of dead pixels can be of importance in the case defect pixel correction algorithms will be used in the camera, and in the case these algorithms rely on a priori knowledge of the defect pixel coordinates.  The technique to find the location of dead pixels is very simple : because the defect pixels do not react to light, they show a pretty low value for the temporal noise under light conditions.  So what can be done is the following :

–       grab several images (e.g. 100), with sufficient light on the sensor to fill the pixels to a level of (about) 50 % of saturation,

–       calculate on pixel level the temporal noise that the pixels demonstrate in the images grabbed,

–       rank all pixels according to the calculated noise, the pixel with the lowest noise level comes first.  If this noise level is close to zero, the pixel is most probably a dead pixel,

–       track the value of these suspected dead pixels over the frames grabbed.

The result of this exercise is shown in the Figure 1 (the images used for this measurement is different from the data used in previous blogs).

Figure 1 : Behaviour of a first group of five dead pixels as a function of time.

The test images are taken with an exposure time of 20 ms.  On the horizontal axis the number of the image grabbed is shown (this can be seen as a time axis), and on the vertical axis the value of 5 dead pixels is shown.  (The pixel with the lowest number shows the lowest temporal noise.)  All five pixels shown demonstrate a noise level almost equal to 0 DN because all five pixels are fully saturated.

Another group of five pixels is shown in Figure 2.

Figure 2 : Behaviour of a second group of five dead pixels as a function of time.

In Figure 2, two more “white” pixels are shown, but also three “black” defects.  Although not visible in this graph, the temporal noise of the black pixels is not 0 DN, it still contains the noise of the output stage and the ADC.  This is not necessarily the case for the “white” defects, because they can be clipped by the ADC.

A final remark : this noise-based method to localize defect pixels cannot be used to find the “sick” pixels.  The latter ones are defined as pixels with a much larger or much smaller sensitivity than the average sensitivity of the sensor.  Because sick pixels still react to light, they can be found as being the outliers in the histogram of all pixels with light input.

Albert, 17-10-2012.

RTS stands for Random Telegraph Signal and is a very specific type of noise.  The pixels that do show RTS effects seem to switch between different (more or less) discrete states.  In most case they flip up and down between two signal levels.  This RTS effect can originate through :

–       dark current effects in the pixel itself, then the two flipping signal levels will be observed also as 2 RTS levels, or,

–       the effect of a single trap in the source follower, then the two flipping signal levels in combination with the CDS can be observed as 3 RTS levels, without CDS they remain as 2 RTS levels.

How can these RTS pixels be characterized or located ?  Actually the technique is very simple : because RTS pixels flip up and down between different states, they show a pretty large temporal noise.  So what can be done is the following :

–       grab several images (e.g. 100), with a relative long exposure time (100 ms or longer),

–       calculate on pixel level the temporal noise that the pixel demonstrates in the images grabbed,

–       rank all pixels according to the calculated noise, the pixels with the highest noise level come first and these are most probably RTS pixels,

–       track the value of these suspected RTS pixels over the frames grabbed.

The result of this exercise is shown in the Figure 1(the data used for this measurement is different from the data used in previous blogs).

Figure 1 : Behaviour of a first group of five RTS as a function of time.

The images are taken with an exposure time of 2 s.  On the horizontal axis the number of the image grabbed is shown (this can be seen as a time axis), and on the vertical axis the value of 5 RTS pixels is shown.  (The pixel with the lowest number shows the largest temporal noise.)  Pixel 0, 1 and 2 change only once over the time of (90 x 2 s =) 3 min.  Pixel 3 is more lively and has several changes, notice that the time the pixel remains in the low state or the high state seems to be completely unpredictable.

Another group of five pixels is shown in Figure 2.

Figure 2 : Behaviour of a second group of five RTS as a function of time.

In Figure 2, pixel 5 is the one that attracts the attention : not only several transitions can be seen, but apparently the pixel switches between 3 states.  It is not immediately clear whether the third state is due to :

–       RTS of the dark current,

–       RTS of the source follower, or,

–       (most likely) due to a switching between two states somewhere half way the exposure time.

As can be learned, the RTS pixels can have very strange transition patterns, which are not predictable at all.  This leads to the challenge of calibrating these pixels during the manufacturing of the cameras.  In the case such a pixel behaves in the low state during calibration and in a high state during the application, the camera can show a white spot in the image captured …

Albert, 04-10-2012.