Display apparatus and display method
First Claim
1. A display apparatus capable of displaying an image and receiving light simultaneously or alternately, said display apparatus comprising:
- a processor;
a backlight;
a plurality of display devices configured to form a matrix;
a plurality of light receiving devices operatively connected to the processor, the plurality of light receiving devices placed at a location in close proximity to one of said display devices; and
a memory device operatively coupled to the processor, the memory device storing instructions which when executed by the processor, cause the processor to operate with the plurality of light receiving devices and the backlight to;
(a) turn on the backlight during a backlight on period of a frame and turn off the backlight during a backlight off period of the frame;
(b) read out received-light signals representing light received by the light receiving devices during the backlight on period and the backlight off period;
(c) generate image data representing a state of a substance brought into contact with said display screen or close proximity thereof based on the received-light signals, said image data including a plurality of pixels, said pixels including a first type of pixel and a second type of pixel;
(d) for said first type of pixel, determine whether a designated number of the second type of pixels surround said first type of pixel;
(e) if the designated number of the second type of pixels surround said first type of pixel, set a first pixel value for said first type of pixel;
(f) if the designated number of the second type of pixels do not surround said first type of pixel, set a second different pixel value for said first type of pixel;
(g) determine whether a plurality of said first type of pixels form a linked element;
(h) eliminate a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels;
(i) append a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels;
(j) for at least two of the plurality of linked elements with an appended label, evaluate;
(i) a position and size of a substance brought into contact with said display screen or close proximity thereof; and
(ii) changes in said position and in said size based on the set pixel value for said first type of pixel of the generated image data; and
(k) recognize, upon detection of a plurality of contact portions each touched or approached by said substance, a command entered by an operator based on at least one of;
(i) a relation between positions of the plurality of contact portions each touched by a substance or approached by said substance;
(ii) a change of the position of each of said contact portions; and
(iii) the size of each of said contact portions; and
(l) for the plurality of contact portions which correspond to linked elements, carry out predetermined processing according to the recognized command; and
wherein the backlight on period and the backlight off period each include a first half and a second half such that;
the received-light signals are reset during the first half of the backlight on period and the backlight off period, andthe received-light signals are read out during the second half of the backlight on period and the backlight off period.
2 Assignments
0 Petitions
Accused Products
Abstract
A display apparatus and display method are provided. The display apparatus capable of displaying an image and receiving light simultaneously or alternately, the display apparatus including: a plurality of display devices laid out to form a matrix; a plurality of light receiving devices for receiving light hitting a display screen; a received-light image generation unit for generating image data representing the state of a substance brought into contact with the display screen or brought to a position in close proximity to the display screen; an image processing/evaluating unit for evaluating the position and size of a substance as well as evaluating changes in the position and in the size; and a command determination unit for recognizing a command entered by an operator. In the display apparatus, predetermined processing according to a command recognized by the command determination unit is carried out.
-
Citations
10 Claims
-
1. A display apparatus capable of displaying an image and receiving light simultaneously or alternately, said display apparatus comprising:
-
a processor; a backlight; a plurality of display devices configured to form a matrix; a plurality of light receiving devices operatively connected to the processor, the plurality of light receiving devices placed at a location in close proximity to one of said display devices; and a memory device operatively coupled to the processor, the memory device storing instructions which when executed by the processor, cause the processor to operate with the plurality of light receiving devices and the backlight to; (a) turn on the backlight during a backlight on period of a frame and turn off the backlight during a backlight off period of the frame; (b) read out received-light signals representing light received by the light receiving devices during the backlight on period and the backlight off period; (c) generate image data representing a state of a substance brought into contact with said display screen or close proximity thereof based on the received-light signals, said image data including a plurality of pixels, said pixels including a first type of pixel and a second type of pixel; (d) for said first type of pixel, determine whether a designated number of the second type of pixels surround said first type of pixel; (e) if the designated number of the second type of pixels surround said first type of pixel, set a first pixel value for said first type of pixel; (f) if the designated number of the second type of pixels do not surround said first type of pixel, set a second different pixel value for said first type of pixel; (g) determine whether a plurality of said first type of pixels form a linked element; (h) eliminate a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels; (i) append a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels; (j) for at least two of the plurality of linked elements with an appended label, evaluate; (i) a position and size of a substance brought into contact with said display screen or close proximity thereof; and (ii) changes in said position and in said size based on the set pixel value for said first type of pixel of the generated image data; and (k) recognize, upon detection of a plurality of contact portions each touched or approached by said substance, a command entered by an operator based on at least one of; (i) a relation between positions of the plurality of contact portions each touched by a substance or approached by said substance; (ii) a change of the position of each of said contact portions; and (iii) the size of each of said contact portions; and (l) for the plurality of contact portions which correspond to linked elements, carry out predetermined processing according to the recognized command; and wherein the backlight on period and the backlight off period each include a first half and a second half such that; the received-light signals are reset during the first half of the backlight on period and the backlight off period, and the received-light signals are read out during the second half of the backlight on period and the backlight off period. - View Dependent Claims (2, 3, 4, 7, 8, 9, 10)
-
-
5. A display method capable of displaying an image and receiving light simultaneously or alternately, said display method comprising:
-
displaying an image by using a plurality of display devices forming a matrix; turning on a backlight during a backlight on period of a frame and turning off the backlight during a backlight off period of the frame; reading out received-light signals representing light received by a plurality of light receiving devices placed adjacent to at least one of the display devices during the backlight on period and the backlight off period; generating image data which includes a plurality of pixels based on the received-light signals, said pixels including a first type of pixel and a second type of pixel; for said first type of pixel, determining whether a designated number of the second type of pixels surround said first type of pixel; if the designated number of the second type of pixels surround said first type of pixel, setting a first pixel value for said first type of pixel; if the designated number of the second type of pixels do not surround said first type of pixel, setting a second different pixel value for said first type of pixel; determining whether a plurality of said first type of pixels form a linked element; eliminating a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels; appending a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels; and carrying out predetermined processing for a plurality of contact portions corresponding to the linked elements, which are detected when a substance is brought into contact with said contact portion or close proximity thereof, in accordance with at least one of; (a) a relation between positions of said contact portions; (b) a change of the position of each of said contact portions; and (c) the size of each of said contact portions; and wherein the backlight on period and the backlight off period each include a first half and a second half such that; the received-light signals are reset during the first half of the backlight on period and the backlight off period, and the received-light signals are read out during the second half of the backlight on period and the backlight off period.
-
-
6. An image processing method used for a display apparatus, said display apparatus capable of:
- (a) displaying an image by using a plurality of display devices configured to form a matrix;
(b) turning on a backlight during a backlight on period of a frame and turning off the backlight during a backlight off period of the frame; and
(c) reading out received-light signals representing light received by a plurality of light receiving devices placed adjacent to at least one of the display devices during the backlight on period and the backlight off period, said image processing method comprising;generating image data which includes a plurality of pixels based on said received-light signals, said pixels including a first type of pixel and a second type of pixel; for said first type of pixel, determining whether a designated number of the second type of pixels surround said first type of pixel; if the designated number of the second type of pixels surround said first type of pixel, setting a first pixel value for said first type of pixel; if the designated number of the second type of pixels do not surround said first type of pixel, setting a second different pixel value for said first type of pixel; determining whether a plurality of said first type of pixels form a linked element; eliminating a linked element from the plurality of linked elements before appending a label to the plurality of linked elements if the linked element is formed of a number of the first type of pixels less than a first predetermined number of pixels; appending a label to at least two of the plurality of linked elements formed of a number of the first type of pixels greater than or equal to the first predetermined number of pixels; and carrying out predetermined processing for a plurality of contact portions corresponding to the linked elements, which are detected when a substance is brought into contact with said contact portion or close proximity thereof, in accordance with at least one of; (a) a relation between positions of said contact portions; (b) a change of the position of each of said contact portions; and (c) the size of each of said contact portions; and wherein the backlight on period and the backlight off period each include a first half and a second half such that; the received-light signals are reset during the first half of the backlight on period and the backlight off period, and the received-light signals are read out during the second half of the backlight on period and the backlight off period.
- (a) displaying an image by using a plurality of display devices configured to form a matrix;
Specification