Three-dimensional image sensing device and method of sensing three-dimensional images

• US 9,667,883 B2
• Filed: 01/06/2014
• Issued: 05/30/2017
• Est. Priority Date: 01/07/2013
• Status: Active Grant

• Current Assignee: Eminence Electronic Technology Pvt. Ltd.
• Original Assignee: Eminence Electronic Technology Pvt. Ltd.
• Inventors: Chang, Tom, Wu, Kao-Pin, Tsai, Kun-Huang, Hung, Shang-Ming, Kuo-Tsai, Tseng, Chuang, Cheng-Ta, Fang, Chih-Jen
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US14/148,693
• Publication Number:
  US 20140192160A1
• Time in Patent Office: 1,240 Days
• Field of Search: 348 46
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/745 Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination

Abstract

A three-dimensional image sensing device includes a light source, a sensing module, and a signal processing module. The sensing module includes a pixel array, a control unit, and a light source driver. The light source generates flashing light with a K multiple of a frequency of flicker noise or a predetermined frequency. The pixel array samples the flashing light to generate a sampling result. The control unit executes an image processing on the sampling result to generate a spectrum. The light source driver drives the light source according to the K multiple of the frequency or the predetermined frequency. The signal processing module generates the K multiple of the frequency according to the spectrum, or outputs the predetermined frequency to the light source driver, and generates depth information according to a plurality of first images/a plurality of second images during turning-on/turning-off of the light source included in the sampling result.

Claims

1. A three-dimensional image sensing device, comprising:

a light source for generating flashing light with a K multiple of a frequency of flicker noise or a predetermined frequency, wherein K is a positive real number, and the flicker noise is generated by an alternating current source;

a sensing module, comprising;

a pixel array for sampling the flashing light and an ambient visible light to generate a sampling result, wherein the sampling result comprises a plurality of first images during turning-on of the light source and a plurality of second images during turning-off of the light source;

a control circuit coupled to the pixel array to execute an image processing on the sampling result to generate a spectrum corresponding to the plurality of second images of the sampling result; and

a light source driver circuit coupled to the light source that drives the light source; and

a signal processing circuit having an output terminal coupled to the light source driver circuit of the sensing module and an input terminal coupled to the control circuit of the sensing module generating the K multiple of the frequency according to the spectrum, wherein the light source driver circuit drives the light source to generate the flashing light with the K multiple of the frequency related to the spectrum, wherein in one period of the sampling result corresponding to the ambient visible light has a positive half period and a negative half period, the positive half period of the sampling result is divided into K/2 first positive areas corresponding to the first images and K/2 second positive areas corresponding to the second images, and the negative half period of the ambient visible light is divided into K/2 first negative areas corresponding to the first images and K/2 second negative areas corresponding to the second images, the K/2 first positive areas and the K/2 second positive areas are alternate in the positive half period of the sampling result, and the K/2 first negative areas and the K/2 second negative areas are alternate negative half period of the sampling result;

wherein when the pixel array samples the flash light with the K multiple of frequency and the ambient visible light, the signal processing circuit calculates a sum of the K/2 first positive areas and K/2 first negative areas and the sum is 0;

or the signal processing circuit calculates a sum of the K/2 second positive areas and K/2 second negative areas and the sum is 0, and

when the K multiple of the frequency of flicker noise is within a predetermined frequency range, the signal processing circuit outputs the K multiple of the frequency of flicker noise to the light source driver circuit, and when the K multiple of the frequency of flicker noise is out of the predetermined frequency range, the signal processing circuit outputs the predetermined frequency to the light source driver circuit.

2. The three-dimensional image sensing device of claim 1, wherein the light source is an infrared ray light source or an infrared ray laser light source.

3. The three-dimensional image sensing device of claim 1, wherein the control circuit comprises:

an analog-to-digital converter for generating a digital signal according to the sampling result; and

a discrete Fourier transform processor for generating the spectrum corresponding to the sampling result according to the digital signal.

4. The three-dimensional image sensing device of claim 1, wherein the signal processing circuit comprises a flicker noise identifier circuit and a frequency generator coupled to the flicker noise identifier circuit, wherein:

the flicker noise identifier circuit determines the flicker noise corresponding to K multiple of an alternating current frequency according to the spectrum, and generates a control signal corresponding to the K multiple of the frequency according to the flicker noise; and

the frequency generator generates the K multiple of the frequency according to the control signal.

5. The three-dimensional image sensing device of claim 4, wherein the frequency generator is a delay time oscillator, a resistor-capacitor oscillator, a phase locked loop, a crystal oscillator, a resonator, or a frequency divider.

6. The three-dimensional image sensing device of claim 1, wherein exposure time of the pixel array during turning on of the light source is equal to exposure time of the pixel array during turning off of the light source.

7. The three-dimensional image sensing device of claim 1, wherein the signal processing circuit is further used for determining a gesture of a user according to a first image of the plurality of first images and a second image of the plurality of second images corresponding to the first image.

8. The three-dimensional image sensing device of claim 1, wherein the sensing module further comprises:

an infrared ray filter for attenuating light except infrared ray.

9. The three-dimensional image sensing device of claim 1, wherein K is N or

10. A method of sensing three-dimensional images, wherein a three-dimensional image sensing device applied to the method comprises a light source, a sensing module, and a signal processing circuit, the method comprising:

the sensing module driving the light source to generate flashing light with a K multiple of a frequency of flicker noise or a predetermined frequency according to the K multiple of the frequency or the predetermined frequency, wherein K is a positive real number, and the flicker noise is generated by an alternating current source;

the sensing module sampling the flashing light and an ambient visible light to generate a sampling result, wherein the sampling result comprises a plurality of first images during turning-on of the light source and a plurality of second images during turning-off of the light source;

the sensing module executing an image processing on the sampling result to generate a spectrum corresponding to the plurality of second images of the sampling result; and

the signal processing circuit generating and outputting the K multiple of the frequency to the sensing module according to the spectrum to drive the light source to generate the flashing light with the K multiple of the frequency, wherein in one period of the sampling result corresponding to the ambient visible light has a positive half period and a negative half period, the positive half period of the sampling result is divided into K/2 first positive areas corresponding to the first images and K/2 second positive areas corresponding to the second images, and the negative half period of the ambient visible light is divided into K/2 first negative areas corresponding to the first images and K/2 second negative areas corresponding to the second images, the K/2 first positive areas and the K/2 second positive areas are alternate in the positive half period of the sampling result, and the K/2 first negative areas and the K/2 second negative areas are alternate negative half period of the sampling result;

wherein when the sensing module samples the flash light with the K multiple of frequency and the ambient visible light, the signal processing circuit calculates a sum of the K/2 first positive areas and K/2 first negative areas and the sum is 0;

or the signal processing circuit calculates a sum of the K/2 second positive areas and K/2 second negative areas and the sum is 0, and

wherein when the K multiple of the frequency of flicker noise is within a predetermined frequency range, the signal processing circuit outputs the K multiple of the frequency of flicker noise to a light source driver, and when the K multiple of the frequency of flicker noise is out of the predetermined frequency range, the signal processing circuit outputs the predetermined frequency to the light source driver.

11. The method of claim 10, wherein the sensing module executing the image processing on the sampling result to generate the spectrum corresponding to the plurality of second images of the sampling result comprises:

generating a digital signal according to the plurality of second images of the sampling result; and

generating the spectrum corresponding to the plurality of second images of the sampling result according to the digital signal.

12. The method of claim 10, wherein the signal processing circuit generating and outputting the K multiple of the frequency to the sensing module according to the spectrum comprises:

determining the flicker noise corresponding to K multiple of an alternating current frequency according to the spectrum;

generating a control signal corresponding to the K multiple of the frequency according to the flicker noise; and

generating and outputting the K multiple of the frequency to the sensing module according to the control signal.

13. The method of claim 10, wherein exposure time of a pixel array during turning on of the light source is equal to exposure time of the pixel array during turning off of the light source.

14. The method of claim 10, wherein K is N or