Imaging and display apparatus, information input apparatus, object detection medium, and object detection method

• US 8,610,670 B2
• Filed: 08/21/2007
• Issued: 12/17/2013
• Est. Priority Date: 08/31/2006
• Status: Active Grant

• Current Assignee: Japan Display West Incorporated (Japan Display Incorporated)
• Original Assignee: Japan Display West Incorporated (Japan Display Incorporated)
• Inventors: Harada, Tsutomu, Tamayama, Takeru, Yamguchi, Kazunori, Tateuchi, Mitsuru, Tsujimura, Kouji, Tsuzaki, Ryoichi
• Primary Examiner(s): Hicks, Charles V
• Application Number:
  US11/842,727
• Publication Number:
  US 20080055266A1
• Time in Patent Office: 2,310 Days
• Field of Search: 345156-178
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

The imaging and display apparatus is provided. The imaging and display apparatus includes the following elements. An imaging-and-display panel has an image display function and an imaging function. A generation unit generates a difference image representing a difference between a display-light-based image and a shadow image, the display-light-based image being obtained by the imaging-and-display panel by imaging an object close to the imaging-and-display panel using display light from the imaging-and-display panel, the shadow image being obtained by the imaging-and-display panel by imaging a shadow of the object. An image processing unit obtains information concerning at least one of a position, shape, and size of the object taking the shadow image and the difference image into account.

Claims

1. An imaging and display apparatus comprising:

an imaging-and-display panel having an image display function and an imaging function;

a backlight;

a processor; and

a memory device storing instructions which when executed by the processor cause the processor to operate with the imaging-and-display panel to;

(a) obtain a display-light-based image by imaging an object close to the imaging-and-display panel during a light-emission period when the backlight is turned on;

(b) obtain a shadow image by imaging the object during a non-light-emission period when the backlight is turned off;

(c) generate a difference image representing a difference between the display-light-based image and the shadow image; and

(d) obtain information concerning at least one of a position, shape, and size of the object taking the shadow image and the difference image into account.

2. The imaging and display apparatus of claim 1, wherein one operating cycle includes the non-light-emission period and the light-emission period, and in one operating cycle, the shadow image is obtained before the display-light-based image is obtained.

3. The imaging and display apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to, in response to a failure to obtain the information concerning at least one of the position, shape, and size of the object based on the difference image, obtain the information concerning at least one of the position, shape, and size of the object based on the shadow image and independent of the difference image.

4. The imaging and display apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to:

(a) select one of the shadow image and the difference image according to a predetermined index; and

(b) obtain the information concerning at least one of the position, shape, and size of the object based on the selected image.

5. The imaging and display apparatus of claim 4, wherein the instructions, when executed by the processor, cause the processor to, in response to a black image being displayed on the imaging-and-display panel, obtain the information concerning at least one of the position, shape, and size of the object based on the shadow image.

6. The imaging and display apparatus of claim 4, wherein the instructions, when executed by the processor, cause the processor to:

(a) determine whether the imaging and display apparatus is in a dark environment based on the shadow image; and

(b) in response to the imaging and display apparatus being in the dark environment, obtain the information concerning at least one of the position, shape, and size of the object based on the difference image.

7. The imaging and display apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to obtain the information concerning at least one of the position, shape, and size of the object using a composite image generated by combining the shadow image and the difference image.

8. The imaging and display apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to:

(a) generate a moving-average image of the shadow image; and

(b) obtain the information concerning at least one of the position, shape, and size of the object using the moving-average image.

9. The imaging and display apparatus of claim 8, wherein:

(a) the shadow image includes a plurality of pixel data items; and

(b) the instructions, when executed by the processor, cause the processor to generate the moving-average image based on the shadow image by;

(i) performing an averaging calculation of pixel data items in a pixel region of the shadow image, the pixel region including a target pixel and pixels surrounding the target pixel; and

(ii) sequentially shifting from the target pixel to a next target pixel while reflecting a result of the averaging calculation performed in the pixel region in an averaging calculation to be performed in a pixel region including the next target pixel so that the averaging calculation is performed over the entirety of the shadow image.

10. The imaging and display apparatus of claim 9, wherein the instructions, when executed by the processor, cause the processor to determine the size of the pixel region used for the averaging calculation based on the size of the object.

11. The imaging and display apparatus of claim 9, wherein the instructions, when executed by the processor, cause the processor to perform the averaging calculation by thinning out a pixel to be subjected to the averaging calculation.

12. The imaging and display apparatus of claim 8, wherein the instructions, when executed by the processor, cause the processor to obtain the information concerning at least one of the position, shape, and size of the object using a difference image representing a difference between the moving-average image and the shadow image.

13. The imaging and display apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to obtain the information concerning at least one of the position, shape, and size of each of a plurality of objects close to the imaging-and-display panel that are simultaneously positioned on the imaging-and-display panel.

14. The imaging and display apparatus of claim 1, wherein the obtained information concerning at least one of the position, shape, and size of the object is displayed by the imaging-and-display panel.

15. The imaging and display apparatus of claim 1, wherein:

(a) the backlight includes a plurality of imaging-and-display elements capable of performing a light-emission operation and a light-receiving operation in a time-division manner; and

(b) the non-light-emission period when the shadow image is obtained is a period during which the imaging-and-display elements do not perform a light-emitting operation.

16. The imaging and display apparatus of claim 15, wherein each of the imaging-and-display elements comprises an organic electroluminescent element.

20. The imaging and display apparatus of claim 1, wherein the imaging-and-display panel includes at least one sensor used to obtain both of the display-light-based image and the shadow image.

17. A non-transitory computer-readable medium storing instructions structured to cause a computer to:

(a) image, by an imaging-and-display panel including a backlight and having an image display function and an imaging function, an object close to the imaging-and-display panel during a non-light-emission period when the backlight is turned off;

(b) image, by the imaging-and-display panel, the object during a light-emission period when the backlight is turned on;

(c) generate a difference image representing a difference between a display-light-based image obtained in the second imaging step and a shadow image obtained in the first imaging step; and

(d) detect information concerning at least one of a position, shape, and size of the object taking the shadow image and the difference image into account.

18. An object detection method comprising:

(a) by an imaging-and-display panel including a backlight and having an image display function and an imaging function, obtaining a shadow image by imaging an object during a non-light-emission period when the backlight in is turned off;

(b) by the imaging-and-display panel, obtaining a display-light-based image by imaging the object during a light-emission period when the backlight is turned on;

(c) generating a difference image representing a difference between the obtained display-light-based image and the obtained shadow image; and

(d) detecting information concerning at least one of a position, shape, and size of the object taking the shadow image and the difference image into account.

19. An imaging and display apparatus comprising:

an imaging-and-display panel having an image display function and an imaging function;

a generation unit that generates a difference image representing a difference between a display-light-based image and a shadow image, the display-light-based image being obtained by the imaging-and-display panel by imaging an object close to the imaging-and-display panel during a light-emission period when the backlight in the imaging-and-display panel is turned on, the shadow image being obtained by the imaging-and-display panel by imaging the object during a non-light-emission period when the backlight is turned off; and

an image processing unit that obtains information concerning at least one of a position, shape, and size of the object taking the shadow image and the difference image into account.