Image pickup apparatus, display-and-image-pickup apparatus and image pickup processing apparatus

• US 8,319,749 B2
• Filed: 02/19/2008
• Issued: 11/27/2012
• Est. Priority Date: 02/23/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Harada, Tsutomu, Hashimoto, Shinya, Maekawa, Tadashi, Okada, Naoya, Yamaguchi, Kazunori, Tateuchi, Mitsuru, Tsuzaki, Ryoichi, Yamanaka, Go
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US12/298,094
• Publication Number:
  US 20100045811A1
• Time in Patent Office: 1,743 Days
• Field of Search: 345/156, 345173-178, 715/700, 178 1801- 1809, 178/18.11
• US Class Current: 345/175
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/63 applied to dark current
• CPC Class Codes: H04N 25/674 based on the scene itself, e.g. defocusing
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An image pickup apparatus capable of easily improving the detection accuracy of a touching or proximity object is provided. An offset noise (a reset image R) is obtained by an image pickup process in a state (a reset state) in which an image pickup device is initialized. Moreover, a fingertip extraction process is performed in consideration of the offset noise on the basis of a picked-up image (a shadow image A or a display light-using image B) which is obtained by picking up an image of a proximity object to obtain object information about the proximity object. While an offset noise in the apparatus is easily reduced, object information about the proximity object can be obtained.

Claims

1. An image pickup apparatus comprising:

an image pickup panel including a plurality of image pickup devices;

an image pickup control means for controlling the image pickup devices so as to obtain an offset noise by an image pickup process in a state in which the image pickup devices are initialized; and

an image processing means for obtaining object information as information about at least one of the position, the shape and the size of a proximity object in consideration of the offset noise on the basis of a picked-up image which is obtained by picking up an image of the proximity object.

2. The image pickup apparatus according to claim 1, wherein

the image processing means obtains the object information in consideration of the offset noise on the basis of a shadow image which is obtained by picking up an image of a shadow of the proximity object.

3. The image pickup apparatus according to claim 2, wherein

the image processing means obtains the object information on the basis of a noise-processed image which is obtained by removing the offset noise from the shadow image.

4. The image pickup apparatus according to claim 3, wherein

the image processing means obtains the object information on the basis of an image which is obtained by adding a predetermined bias value to the noise-processed image.

5. The image pickup apparatus according to claim 4, wherein

the bias value is determined on the basis of an image obtained by removing the offset noise from a dark-state image which is obtained by an image pickup process in a state in which the environment around the image pickup apparatus is dark.

6. The image pickup apparatus according to claim 2, wherein

the image processing means obtains the object information in consideration of the offset noise on the basis of the shadow image and an illumination-light-using image which is obtained by picking up an image of the proximity object through the use of illumination light from the image pickup panel.

7. The image pickup apparatus according to claim 6, wherein

the image processing means obtains the object information on the basis of a differential image between a noise-processed image which is obtained by removing the offset noise from the illumination-light-using image and a noise-processed image which is obtained by removing the offset noise from the shadow image.

8. The image pickup apparatus according to claim 6, wherein

the image processing means obtains the object information on the basis of a differential image between an image which is obtained by adding a predetermined bias value to the noise-processed image obtained by removing the offset noise from the illumination-light-using image, and an image which is obtained by adding the bias value to the noise-processed image obtained by removing the offset noise from the shadow image.

9. The image pickup apparatus according to claim 8, wherein

the bias value is determined on the basis of a differential image between a dark-state image obtained by an image pickup process in a state where the environment around the image pickup apparatus is dark, and an initialized image obtained by the image pickup process in the state in which the image pickup devices are initialized.

10. The image pickup apparatus according to claim 6, wherein

the illumination light is display light from the image pickup panel.

11. The image pickup apparatus according to claim 6, wherein

a unit operation cycle is configured to include a process of obtaining the shadow image and a process of obtaining the illumination-light-using image, and the image pickup devices are initialized before the processes of obtaining the shadow image and illumination-light-using image, and

the image pickup control means perform control such that a process of obtaining the offset noise is performed only in an initialized state before any one of processes of obtaining the shadow image and obtaining the illumination light in each unit operation cycle.

12. The image pickup apparatus according to claim 6, wherein

a unit operation cycle is configured to include a process of obtaining the shadow image and a process of obtaining the illumination-light-using image, and the image pickup devices are initialized before the processes of obtaining the shadow image and the illumination-light-using image, and

the image pickup control means perform control such that a process of obtaining the offset noise is performed only in one initialized state in a plurality of unit operation cycles.

13. The image pickup apparatus according to claim 1, wherein

the image pickup devices in the image pickup panel are formed on a non-silicon (Si) substrate.

14. The image pickup apparatus according to claim 13, wherein

the non-silicon substrate is a glass substrate or a plastic substrate.

15. A display-and-image-pickup apparatus comprising:

a display image pickup panel including a plurality of display devices and a plurality of image pickup devices;

an image pickup control means for controlling the image pickup devices so as to obtain an offset noise by an image pickup process in a state in which the image pickup devices are initialized;

an image processing means for obtaining object information as information about at least one of the position, the shape and the size of a proximity object in consideration of the offset noise on the basis of a picked-up image which is obtained by picking up an image of the proximity object; and

a display control means for controlling the display devices so that the obtained object information is displayed on the display image pickup panel.

16. The display-and-image-pickup apparatus according to claim 15, wherein

the display devices and the image pickup devices in the display image pickup panel are formed on a non-silicon (Si) substrate.

17. An image pickup processing apparatus being a processing apparatus applied to an image pickup panel including a plurality of image pickup devices, the image pickup processing apparatus comprising:

an image pickup control means for controlling the image pickup devices so as to obtain an offset noise by an image pickup process in a state in which the image pickup devices are initialized; and

an image processing means for obtaining object information as information about at least one of the position, the shape and the size of a proximity object in consideration of the offset noise on the basis of a picked-up image which is obtained by picking up an image of the proximity object.

18. The image pickup processing apparatus according to claim 17, wherein

the image pickup devices in the image pickup panel are formed on a non-silicon (Si) substrate.