Display apparatus and display method

• US 8,704,804 B2
• Filed: 10/02/2006
• Issued: 04/22/2014
• Est. Priority Date: 10/05/2005
• Status: Active Grant

• Current Assignee: Japan Display West Incorporated (Japan Display Incorporated)
• Original Assignee: Japan Display West Incorporated (Japan Display Incorporated)
• Inventors: Yamaguchi, Kazunori, Harada, Tsutomu, Tateuchi, Mitsuru
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): Rodriguez, Joseph G
• Application Number:
  US11/537,825
• Publication Number:
  US 20070120833A1
• Time in Patent Office: 2,759 Days
• Field of Search: 345/204, 345/173, 345/102, 715/863, 382/274
• US Class Current: 345/204
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A display apparatus and display method are provided. The display apparatus capable of displaying an image and receiving light simultaneously or alternately, the display apparatus including: a plurality of display devices laid out to form a matrix; a plurality of light receiving devices for receiving light hitting a display screen; a received-light image generation unit for generating image data representing the state of a substance brought into contact with the display screen or brought to a position in close proximity to the display screen; an image processing/evaluating unit for evaluating the position and size of a substance as well as evaluating changes in the position and in the size; and a command determination unit for recognizing a command entered by an operator. In the display apparatus, predetermined processing according to a command recognized by the command determination unit is carried out.

Claims

1. A display apparatus capable of displaying an image and receiving light simultaneously or alternately, said display apparatus comprising:

a processor;

a backlight;

a plurality of display devices configured to form a matrix;

a plurality of light receiving devices operatively connected to the processor, the plurality of light receiving devices placed at a location in close proximity to one of said display devices; and

a memory device operatively coupled to the processor, the memory device storing instructions which when executed by the processor, cause the processor to operate with the plurality of light receiving devices and the backlight to;

(a) turn on the backlight during a backlight on period of a frame and turn off the backlight during a backlight off period of the frame;

(b) read out received-light signals representing light received by the light receiving devices during the backlight on period and the backlight off period;

(c) generate image data representing a state of a substance brought into contact with said display screen or close proximity thereof based on the received-light signals, said image data including a plurality of pixels, said pixels including a first type of pixel and a second type of pixel;

(d) for said first type of pixel, determine whether a designated number of the second type of pixels surround said first type of pixel;

(e) if the designated number of the second type of pixels surround said first type of pixel, set a first pixel value for said first type of pixel;

(f) if the designated number of the second type of pixels do not surround said first type of pixel, set a second different pixel value for said first type of pixel;

(g) determine whether a plurality of said first type of pixels form a linked element;

(h) eliminate a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels;

(i) append a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels;

(j) for at least two of the plurality of linked elements with an appended label, evaluate;

(i) a position and size of a substance brought into contact with said display screen or close proximity thereof; and

(ii) changes in said position and in said size based on the set pixel value for said first type of pixel of the generated image data; and

(k) recognize, upon detection of a plurality of contact portions each touched or approached by said substance, a command entered by an operator based on at least one of;

(i) a relation between positions of the plurality of contact portions each touched by a substance or approached by said substance;

(ii) a change of the position of each of said contact portions; and

(iii) the size of each of said contact portions; and

(l) for the plurality of contact portions which correspond to linked elements, carry out predetermined processing according to the recognized command; and

wherein the backlight on period and the backlight off period each include a first half and a second half such that;

the received-light signals are reset during the first half of the backlight on period and the backlight off period, and

the received-light signals are read out during the second half of the backlight on period and the backlight off period.

2. The display apparatus of claim 1, wherein said predetermined processing includes changing a displayed image displayed by said display devices.

3. The display apparatus of claim 1, wherein said predetermined processing includes changing an operation of an application for generating a displayed image displayed by said display devices.

4. The display apparatus of claim 1, wherein an image to be detected in processing is an image of a substance placed on said display screen or a shape of a shadow of a substance placed at a position in close proximity to said display screen.

7. The display apparatus of claim 1, wherein the processor operates with the plurality of light receiving devices to:

(l) for each of the plurality of linked elements, count a number of the first type of pixels forming the linked element.

8. The display apparatus of claim 7, wherein the processor operates with the plurality of light receiving devices to:

(m) eliminate at least one of the plurality of linked elements with the appended label, formed of a number of the first type of pixels below a second predetermined number of pixels, from the plurality of linked elements with the appended label before evaluating the plurality of linked elements with the appended label.

9. The display apparatus of claim 1, wherein each of the plurality of display devices has a corresponding light receiving device.

10. The display apparatus of claim 1, wherein the processor operates with the plurality of light receiving devices to:

generate a received-light difference signal representing a difference between a received-light signal read out during the backlight on period and a received-light signal read out during the backlight off period,

wherein the image data is generated based on the received-light difference signal.

5. A display method capable of displaying an image and receiving light simultaneously or alternately, said display method comprising:

displaying an image by using a plurality of display devices forming a matrix;

turning on a backlight during a backlight on period of a frame and turning off the backlight during a backlight off period of the frame;

reading out received-light signals representing light received by a plurality of light receiving devices placed adjacent to at least one of the display devices during the backlight on period and the backlight off period;

generating image data which includes a plurality of pixels based on the received-light signals, said pixels including a first type of pixel and a second type of pixel;

for said first type of pixel, determining whether a designated number of the second type of pixels surround said first type of pixel;

if the designated number of the second type of pixels surround said first type of pixel, setting a first pixel value for said first type of pixel;

if the designated number of the second type of pixels do not surround said first type of pixel, setting a second different pixel value for said first type of pixel;

determining whether a plurality of said first type of pixels form a linked element;

eliminating a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels;

appending a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels; and

carrying out predetermined processing for a plurality of contact portions corresponding to the linked elements, which are detected when a substance is brought into contact with said contact portion or close proximity thereof, in accordance with at least one of;

(a) a relation between positions of said contact portions;

(b) a change of the position of each of said contact portions; and

(c) the size of each of said contact portions; and

wherein the backlight on period and the backlight off period each include a first half and a second half such that;

the received-light signals are reset during the first half of the backlight on period and the backlight off period, and

the received-light signals are read out during the second half of the backlight on period and the backlight off period.

6. An image processing method used for a display apparatus, said display apparatus capable of:

(a) displaying an image by using a plurality of display devices configured to form a matrix;

(b) turning on a backlight during a backlight on period of a frame and turning off the backlight during a backlight off period of the frame; and

(c) reading out received-light signals representing light received by a plurality of light receiving devices placed adjacent to at least one of the display devices during the backlight on period and the backlight off period, said image processing method comprising;

generating image data which includes a plurality of pixels based on said received-light signals, said pixels including a first type of pixel and a second type of pixel;

for said first type of pixel, determining whether a designated number of the second type of pixels surround said first type of pixel;

if the designated number of the second type of pixels surround said first type of pixel, setting a first pixel value for said first type of pixel;

if the designated number of the second type of pixels do not surround said first type of pixel, setting a second different pixel value for said first type of pixel;

determining whether a plurality of said first type of pixels form a linked element;

eliminating a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels;

appending a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels; and

carrying out predetermined processing for a plurality of contact portions corresponding to the linked elements, which are detected when a substance is brought into contact with said contact portion or close proximity thereof, in accordance with at least one of;

(a) a relation between positions of said contact portions;

(b) a change of the position of each of said contact portions; and

(c) the size of each of said contact portions; and

wherein the backlight on period and the backlight off period each include a first half and a second half such that;

the received-light signals are reset during the first half of the backlight on period and the backlight off period, and

the received-light signals are read out during the second half of the backlight on period and the backlight off period.