Good Bye 2019 !

December 15th, 2019

Good Bye 2019 !

Just back from the yearly Harvest Imaging Forum 2019, and then it is time to reflect back on 2019.  The year was very busy again, but not just because of the work with Harvest Imaging, but also because I lost my last parent.  Unfortunately these things happen if one gets older, at a particular moment, your parents’ life is coming to an end …

As far as the business of Harvest Imaging is concerned, it was a kind of another year very similar to the previous ones.  Courses continued, public as well as in-house trainings were added and added.  It is really amazing how many companies are working in the digital imaging field, and how many fresh engineers start an imaging career.  Digital image capturing was, is and will be a growing activity.  Many companies are looking after (experienced) imaging engineers, our profession still has a great future !

Harvest Imaging continued with the consulting activities as well.  All this type of work is protected by NDA’s and more information about the content of the projects cannot be revealed.

The project about reproducibility, variability and reliability of CMOS imagers was continued.  Another 1000 measurements are added to the list I had already.  On a very short term the 7th characterization session in a row of 10 (to cover a span of 5 years) will start.  Then all cameras will be placed on the testbench and will be measured over and over again !  The characteristics of the cameras/sensors at room temperature seem to be very predictable, but things change drastically when the temperature and the humidity of the stressing conditions are increased.  Another experiment is started in 2019 : putting the devices under a strong light source that is simulating sun light.  Also in this experiment surprising first results are obtained, but many more measurements and stress moments are needed before a final conclusion can be taken.  As is very often the case : a new measurement raises two new questions !

On a more general technical and scientific level, 2019 was a another year with an International Image Sensor Workshop, held at Snow Bird (Utah, US).  In combination with ISSCC and IEDM, again great technical and scientific achievements were published, a lot of material to study and a lot of stuff to learn from.  Thanks for all the authors to bring their work into the public domain, it is an incredible source of information where the imaging community can learn from.

So now that 2019 is almost completed, the question is : “What will 2020 bring ?”.  The technical courses and trainings of Harvest Imaging will continue, that is for sure.  Several courses are already booked.  A new course will be added to the existing basket : “The colour pipeline of a CMOS camera”.  It is a kind of a chapter out of the course “Digital camera systems” (a 4-days course that existed already), which is extended to cover 2 days.  Besides that, some crazy idea exists to start with a two-week Summer School.  Harvest Imaging is working with potential customers to see whether there could be enough interest to justify a commercial success of such an initiative.  Something to look forward to.

Wishing all my readers a Merry Christmas and a Happy New Year.  “See” you soon.

Albert, 15-12-2019.

 

Agenda of the Harvest Imaging Forum 2019

November 14th, 2019

Please find below the agenda of the Harvest Imaging Forum 2019.  Do not pay too much attention on the timing, because the forum will run in 2 parallel sessions.  The (indicative) timing shown is valid for the first session, the second one will get another timing.

AGENDA
Harvest Imaging Forum :
On-Chip Feature Extraction & Direct ToF 3D Imaging
Day 1 9:00 – 9:15 Introduction to the forum
9:15 – 10:45 1. Basics of Image Sensors
1-1 3T/4T pixels
1-2 Column CDS / Column ADC / Column Circuits
1-3 Signal processing: In-pixel / In-column / On-chip
1-4 Global shutter
1-5 3D stacking / BSI image sensors
10:45 – 11:15 Break
11:15 – 12:45 2. Overview of Functional Imaging
2-1 High-speed imagers
2-2 Wide dynamic range imagers
2-3 Event driven imagers
2-4 3D range finding
2-5 Angular sensitive imagers
12:45 – 14:00 Lunch break
14:00 – 15:30 3. Features to extract
3-1 ROI / Illumination position detection
3-2 Edge detection / edge direction detection
3-3 Gravity center detection
3-4 Motion detection
3-5 Optical flow detection
3-6 Time detection / time stamping
3-7 Event detection
15:30 – 16:00 Break
16:00 – 17:30 4. Examples of Functional Image Sensors for 3D range finding
4-1 In-pixel / Column parallel Position detection circuits
4-2 In-pixel / row parallel demodulation circuits
4-3 Lock-in pixels
19:00 – 20:30 Dinner
Day 2 9:00 – 10:30 1. Introduction
1-1 Markets and applications of 3D image sensors
1-2 Brief overview of 3D imaging techniques
1-3 Time-of-Flight principle
1-4 Indirect and direct ToF
1-5 Detector technologies
1-6 Which technology for which application
10:30 – 11:00 Break
11:00 – 12:30 2. Direct Time-of-Flight Modelling
2-1 System level considerations/optical power budget
2-2 Ideal building blocks
2-3 DToF system modelling
2-4 Performance parameters
2-5 Algorithms
2-6 Comparison with IToF
12:30 – 13:45 Lunch break
13:45 – 15:15 3. Single-Photon Avalanche Diodes
3-1 What is a SPAD: operating principle
3-2 Control and processing electronics for SPADs4)
3-3 State of the art SPAD technologies

4. DToF Sensor architectures
4-1 Building blocks for DToF
4-2 Single-point DToF sensors
4-3 3D image sensors

15:15 – 15:45 Break
15:45 – 17:15 5. 3D Image Sensors Systems
5-1 Illumination sources
5-2 Background light rejection
5-3 Algorithms and data processing
5-4 Characterization of a DToF 3D camera
5-5 Laser eye-safety considerations
17:15 – 17:30 Closure of the forum

Albert, 14-11-2019.

40 Years Ago (4)

November 14th, 2019

My PhD project in the late ’70s involved the development of Indium-Tin-Oxide (ITO) CCD electrodes.  ITO is a transparent and conductive material, from the standpoint of view of those characteristics, ITO is ideally suited to be used as a CCD electrode.  In the first year of my research, I succeeded in the deposition of transparent ITO that was fully oxidized, and was actually non conductive.  By means of an appropriate anneal the ITO films could be made conductive as well.  Actually what was realized was a kind of conductive glass.

In the second year of my PhD project (= 40 years ago) I focused on using the ITO film as a gate electrode in a MOS structure.  And that turned out to be much more difficult than expected.  I still remember that I had to set up a measurement system to perform HF and LF capacitance measurements on the realized MOS capacitor with the ITO top gate.  A hard copy of the developed software code in Fortran is still present in my archives.  Unfortunately the RF sputtering technique applied to deposit the ITO films created too much radiation damage in the MOS structures.  As a result, the MOS capacitors showed a very high number of surface states.  A high number of surface or interface states would ultimately result in a very high dark current in the final CCDs.  Too bad !  I tried many annealing methods, various annealing atmospheres in combination with various temperatures, but none of them resulted in a low number of surface states.  I still remember that  I worked in close cooperation with the solar cell R&D team.  They offered me the opportunity to anneal my “dirty” structures in their furnaces at high temperatures, even in a reactor in which the structures could be annealed in hydrogen.  But no processing recipe could be found that resulted in a low number of surface states.  So the only solution was to give up the damaging RF sputter technique and to make use of magnetron sputtering for deposition of the ITO films.  Magnetron sputtering was known to result in a low level of radiation damage and seemed to be suited for deposition of gate material on MOS structures.

I remember that the university was willing to buy a new magnetron sputtering system from Balzers.  But Balzers was not really familiar with the sputtering of ITO, so I had to go to the company in Liechtenstein to perform ITO experiments.  This was my very first business trip ever, all the way flying to Zurich and then the train to Sargans if I recall well.  Once at Balzers’facility we succeeded immediately to deposit the right combination of Indium Oxide and Tin Oxide, so we gave a “go” to buy the machine.  It was Mr. Klinkemay of Balzers (probably misspelled ?!) who did the installation of the machine in our laboratory at the university.  Also with the final machine the first films of ITO were realized very quickly.  So ready to deposit ITO films with a low level of radiation damage !  My work with the magnetron sputter machine became so well accepted in the lab, that the machine was provided with a second aluminum target and the machine was going to be used in our lab as the “standard” deposition method for aluminum metallization of integrated circuits.  As a result, I almost had to beg on my bare knees to use my “own” machine for the ITO depositions.  After a while the management of the lab saw the problems, and decided to buy a second machine.  What a luxury, I got my own machine back for my own ITO depositions !  An incredible amount of test MOS capacitors were deposited, annealed and measured.  A Russian postdoc helped me out with the measurements (unfortunately I lost his name).  And yes, finally we succeeded in realizing MOS capacitors with an ITO gate material that showed a low number of surface states, not as low as a capacitor with a poly-silicon gate, but more than an order of magnitude less than the devices fabricated with the RF sputter machine.  A major step forward !

Albert, 14-11-2019.

International Image Sensor Workshop 2019 (4)

June 28th, 2019

Day number 4 is a relative short day with only 9 presentations.

Radiation hardening was a topic that came back in 2 papers, one of Brookman Technology and one of ISAE, Toulouse.  In both papers emphasis was made on the tiny details in the lay-out of a pixel to make the pixel more radiation hard.  The devil really sits in the details.  The golden rule remains : keep the electric fields as low as possible !  Surprisingly, there was only one TDI paper presented at the workshop, combined with the single topic on multi-spectral imaging.  It was a report by IMEC of a kind of a follow-up device presented at previous workshops.

Global shutter devices got a separate session : Brillnics with a device that combines GS with HDR in the voltage domain by using a stacking technology, TPSCo with a 2.5 um GS in the charge domain (this paper was a repetition of their IEDM 2018 paper) as well as a 2.8 um GS with near-IR enhanced sensitivity.  A remark made by TPSCo that the next version of their GS shutter device will have 2.2 um pixel pitch.  The work of TPSCo is all front-side illuminated.

ON Semiconductor presented a GS pixel with correction processing by means of a pixel with a single PD, single FD, but double storage node.  The final paper of the workshop was delivered by ams : a BSI pixel with background light suppression intended for structured light applications and object tracking.  A clever pixel, based on 6T (2 resets, 2 transfer gates, 1 source follower, 1 row select and two floating diffusions) and 1C (connected between the 2 floating diffusions) in the pixel, allows to shoot a sequence of multiple frames one after another and only stores the difference between the frames.  Pixel size is just 2.8 um.  Testdevices seem to be available, but measurement results no yet.  Maybe next workshop !?

The IISW2019 was again a big success, many excellent presentations, all high quality papers, good atmosphere, perfect organization.  The interest of the participants can be best judged by the amount of people that are willing to stay till the very last paper is presented.  And it is really remarkable that during all the image sensor workshops, almost everyone really stays till the very end.

Congratulations and thanks to the organization of IISW2019.  See you all again in Europe for the IISW2021.

 

Albert, 28-06-2019.

International Image Sensor Workshop 2019 (3)

June 26th, 2019

What a day today !  Actually a very long day with presentations from the early morning till the late afternoon.  And it still is not over, because there will be the workshop banquet tonight.

It is really a not-to-finish task to give an overview of the talks of today.  A lot of different subjects were presented, for instance high dynamic range, UV sensitive devices, energy harvesting, SPADs, indirect ToF, flicker mitigation, 16-bit analog-to-digital converter, high speed, endoscopes for non-invasive surgery, near-IR imaging with and without silicon detectors, fundus camera, and much more.  Although the amount of paper submissions for this 2019 workshop was lower than in 2017, the quality of the presentations is most probably the best ever.  Congratulations to all speakers !

 

Albert, 27-06-2019.

International Image Sensor Workshop 2019 (2)

June 26th, 2019

Day 2 was a short one.  Only 12 presentations, which could be captured by a title such as : “new pixel architectures and single photon detection”.  Besides the regular papers an invited one was presented about Deep Learning.

A very interesting presentation was given by Francois Roy (ST Microelectronics) about a pixel architecture WITHOUT any implantation in the photodiode.  The pixel was based on the C-DTI technology of ST in which a gate inside the deep-trench isolation is used to bias the interface of the silicon into accumulation and fully deplete the epi-layer.  In this way a pinned photodiode is created in the 3rd dimension, for which a low dark current is expected.  A presentation of a sensor in the same technology was already given at IEDM 2018 and seems to be present in a well-known consumer product.  This IISW2019 paper showed a back-side illuminated version of this technology, in which the back-surface passivation is realized by “charged” dielectric layers on the back.  Also the latter seems to be a trend in BSI CIS.  The reported dark current of this rolling shutter imager : 5 holes/sec at 60 degrees C for a 2 um pixel (2T5), corresponding to a dark current density of 2 pA/cm2 (holes) at 60 degrees C, coming into the sub-pA/cm2 at room temperature.

Personally I am taken with this technology because it is a very nice combination of silicon device physics and silicon processing technology, bringing a lot of the good old CCDs back, but now in CMOS technology.  Back to the future after 50 years !

It is really funny what can be done by these DTIs.  Yesterday a paper of Samsung showed the use of DTI to split a photodiode into two parts to allow the PDAF function within every pixel.  The width of the a single PPD was reduced to 0.61 um.  Today, also Samsung, introduced “dummy” DTIs in a pixel to enhance the scattering of incoming photons and to enhance the quantum efficiency of the pixels.  A quantum efficiency of 43 % was reported for a wavelength of 940 nm.

But the award for the most artistic pixel lay-out goes to Shizuoka University.  The pixel lay-out looks like a cross-section of a aircraft engine.  The pixel has a central photodiode surrounded by 8 taps + 8 floating diffusions in an array of 22.4 um x 22.4 um.  The pixel is used for indirect time-of-flight applications with an inaccuracy of 7.2 mm at a distance of 6.2 m.

 

Albert, 26/06/2019.

International Image Sensor Workshop 2019 (1)

June 25th, 2019

Very brief : on the first day about 10 papers were presented and 33 posters were announced.  I recognized 3 major trends :

  1. 0.8 um pixel size is there !  The 3 major suppliers are working on 0.8 um and/or have it available on the market : Sony, OmniVision and Samsung.  A very careful prediction is that in 2021-2022 0.7 um pixel pitch will become available.
  2. many people are working on the dominant noise source in a CIS, being the source-follower.  Several papers came with an ownful lot of measurements to explain and further optimize the source-follower noise, including RTS noise.
  3. dark current is still a hot topic, and mainly dark current of the floating diffusion node.  The latter is often used as a storage node and then the dark current is really becoming an issue.  A few years ago, a rough comparison between the dark current of the PPD and the dark current of the FD told that there existed a factor of 1000 in dark current density between the two (guess which one is the lowest).  At this moment the dark current of the FD is reduced already more than one order of magnitude compared to what we had a few years ago.  Still some work to do !

Great quality of papers/presentations, very open discussions, very well organized workshop.  More info to follow !!

Regards,

Albert, 25-06-2019.

Image Sensors London 2019

March 13th, 2019

I promised to report about IS London, but unfortunately I do not have (yet) access to any material (= presentations).  I cannot report about the presentations from the top of my head.  As soon as the presentations are available I will put some stuff together for my readers.

Albert, 13-03-2019.

ISSCC (3)

February 21st, 2019

University of Toronto in cooperation with Synopsys and FBK presented “Dual-tap pipelined-code-memory coded-exposure pixel CMOS image sensor for multi-exposure single-frame computational imaging” (try to explain this to your mother !).  The basic idea comes down to the fact that with a coded aperture quite some information is thrown away (e.g. 50 %) when a particular aperture is opaque for the incoming light.  Only when the aperture is transparent, the incoming information is used.  In this paper, a pixel is presented that has one large PPD and two TG-FD-SF combinations.  The information is read out and accumulated through the first FD or through the second FD.  In this way no incoming photons are lost.  The content of the first FD can be seen as a kind of complement to the content of the second FD and vice versa.  The pixel looks very similar to some pixels presented in ToF applications.  But for the coding (switching between the two FDs), an in-pixel memory is needed.  That is composed out of two latches.  The pixel size is 11.2 um, a fill factor of 45.3 % is achieved, 27 % of the pixel area goes to the memory and extra logic in the pixel.  The contrast between the two taps is reported to be 99 % at 180 fps.  The device is fabricated in a 110 nm CIS process.

Applications mentioned for this sensor are one-shot structured light, one-shot  photometric stereo, compressive sensing, etc.

 

The sixth paper in the session came from Panasonic : A 400 x 400 pixel 6 um pitch vertical avalanche photodiodes (VAPD) CMOS image sensor based on 150ps-fast capacitive relaxation quenching (RQ) in Geiger mode for synthesis of arbitrary gain images”.   The main goal of this work is to incorporate a single photon avalanche photodiode function into a conventional CMOS image sensor pixels.  The pixel proposed in this paper looks identical to a 4T pixel, except that the PPD is replace by an avalanche photodiode.  Because the gain of the avalanche photodiode is not known, it looks like that the application for this device is limited to binary images, which can be used in time-of-flight, surveillance, AI and robotics.

 

The next paper was presented by Univ. of Edinburgh in cooperation with ST and Heriot-Watt University : “A 246×256 40nm/90nm CMOS 3D-stacked 120dB dynamic range reconfigurable time resolved SPAD imager”.  The design challenges seen by the authors are (1 Mpixel SPAD @ 100 MCps/pixel in high background conditions) : 100 Tphoton events/second. Being more than 1 Pb/s (10 bits conversion) and 50 W TDC power.  The presentation was built-up to highlight the advantages of SPADs (time resolution, no ADC, high dynamic range, single photon sensitivity, low median dark noise) and to counteract the disadvantages of SPADs (power consumption, TDC area, high I/O data rate, low fill-factor, many hot pixels.

During the presentation all the drawbacks were addressed one after another and solutions were proposed and implemented to counteract them :

  • Stacking of the backside illumination SPAD above the readout IC (40 nm/90 nm process) is solving the fill factor issue, (pixel pitch of 9.2 um),
  • In-pixel histogramming reduces the I/O data rate,
  • The pixels are composed out of multi-SPADs with the option to inhibit pixels with a large dark current count (also called “screamers”),
  • An event driven clocking is reducing the power consumption,
  • Implementation of compact TDCs, ring oscillator based in combination with shift-register and counters (TDC area of 130 um2).

Albert, 21-02-19

ISSCC (2)

February 20th, 2019

“A Data Compressive 1.5b/2.75b Log-Gradient QVGA Image Sensor with Multi-Scale Readout for Always-On Object Detection” by Stanford Univ. and Robert Bosch.

Object detection in the classical way can be done by the combination of a convolutional neural network connected behind an image sensor.  But that solution is pretty power hungry and not really suited for an always-on application.  This paper suggests to do a coarse detection and feature extraction right after the sensor and if something is detected, a wake-up signal is generated to activate a convolutional neural network.  The image sensor with the coarse detector and feature extractor (in the analog domain) are integrated on the image sensor chip.  A very popular feature extraction is based on a so-called Histogram of Oriented Gradients (HOG).  So one tries to find for 8×8 pixels the orientation of the gradient in the image data, and this method can be repeated for multiple scales of the image.

The trick of the HOG implementation in this paper is not looking after the difference in image values to detect an orientation in the image data, but to go after the logarithmic ratio of pixel values in the image data.  Making ratios instead of differences makes the concept invariant to the illumination level.  Simple, but very effective idea.  The log gradients can be quantized to 1.5 bit or 2.75 bit and still being robust to illumination levels.

The design highlights of this device are recognized as : the log-gradient image sensor using standard 4T pixels in a Q-VGA configuration, compressing the data to 1.5 bits or 2.75 bits, being a 25 x data compression; option for pixel binning at the readout for multi-scale object detection; column parallel ratio-to-digital converters to digitize low resolution ratios of the pixels.

If it comes down to performance numbers, an energy per pixel of 127 pJ is reported, 0.13 um 1P4M process, pixel size of 5 um.  Example images were shown to proof the concept of the image sensor.

“A 76mW 500fps VGA CMOS Image Sensor with Time-Stretched Single-Slope ADCs Achieving 1.95 e Random Noise” by Yonsei University.  Single-slope ADCs on column level, are widely used in CIS.  So everyone knows the advantages, but also the disadvantages of requiring a lot of clock cycles.  In the case one wants a fast conversion rate of for instance 1 us, a clock frequency in the GHz range is needed.  This paper tries to find a solution for this issue, a kind of hybrid ADC is proposed.  In the case of a 10 bit ADC, the 6 most-significant bits are converted by a classical single slope ADC.  What is further measured (to find the 4 least-significant bits) is the time between the toggling of the comparator and the next clock cycle of the single-slope ADC.  The toggling moment of the comparator is the start of the so-called time stretching activity.  The end of the time stretching activity is equal to the falling edge of the ADC clock divided by 16.  Why divided by 16 ?  For the simple reason that in that case the original clock of the ADC can also be used to measure the number of clock cycles in the time stretched period.  Very simple, very clever idea !  The total amount of pulses in the new ADC (10 bits) is now 64 cycles for the first 6 bits, and maximum 16 cycles for the time stretched value.  In total 80 cycles instead of 1024 (12.8 x faster !), and a conversion of 0.8 us can be realized by a clock of 100 MHz.  The new ADC not only results in a faster device, but also it consumes much less power (76 mW instead of 365 mW with the standard SS-ADC).

The reported device is a VGA sensor with 4 um pixel pitch, fabricated in 110 nm 1P4M technology.  The noise values listed are 1.95 e with a gain of 8, 2.85 e with a gain of 1.

Albert, 20-02-19