Display device including function to input information from screen by light

• US 20060170658A1
• Filed: 12/15/2005
• Published: 08/03/2006
• Est. Priority Date: 02/03/2005
• Status: Active Grant

• Current Assignee: Japan Display Central Incorporated (Japan Display Incorporated)
• Original Assignee: Toshiba Matsushita Display Technology (Japan Display Incorporated)
• Inventors: Kanuma, Toshinori, Nakamura, Takashi, Yoshida, Masahiro, Hayashi, Hirotaka, Imai, Takayuki, Ishikawa, Miyuki
• Granted Patent: US 7,800,594 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/042 by opto-electronic means

Abstract

In order to enhance accuracy of determination as to whether an object has contacted a screen and to enhance accuracy of calculation of a coordinate position of the object, edges of an imaged image are detected by an edge detection circuit 76, and by using the edges, it is determined by a contact determination circuit 77 whether or not the object has contacted the screen. Moreover, in order to appropriately control sensitivity of optical sensors in response to external light, by a calibration circuit 93, a drive condition of the optical sensors is changed based on output values of the optical sensors, which are varied in response to the external light.

Claims

1. A display device, comprising:

a display unit including a display function to display an image on a screen, and an optical input function to image an image of an object which has approached the screen;

an edge detection circuit configured to detect edges of the imaged image, thereby forming an edge image;

a contact determination circuit configured to determine whether or not the object has contacted the screen by using the edge image; and

a coordinate calculation circuit configured to calculate a position coordinate of the object by using the edge image when it is determined that the object has contacted the screen.

2. The display device according to claim 1,

wherein the contact determination circuit investigates a moving direction of each of the edges, and determines that the object has contacted the screen when there are edges moving in directions reverse to each other.

3. The display device according to claim 1,

wherein the contact determination circuit extracts a region showing the object by using the edge image, and determines that the object has contacted the screen when an area of the region has reached a predetermined threshold value or more.

4. The display device according to claim 1,

wherein the coordinate calculation circuit calculates a center of gravity of the edges as the position coordinate of the object.

5. The display device according to claim 3,

wherein the coordinate calculation circuit calculates a center of gravity of the region where the area has reached the predetermined threshold value or more as the position coordinate of the object.

6. A display device, comprising:

a display unit including a display function to display an image on a screen, and an optical input function to image an image of an object which has approached the screen;

an inter-frame difference processing circuit configured to obtain a difference image formed by taking a difference between a multi-gradation image in a current frame and a multi-gradation image in a past frame for the imaged image;

a contact determination circuit configured to determine whether or not the object has contacted the screen by using the difference image; and

a coordinate calculation circuit configured to calculate a position coordinate of the object by using the difference image when it is determined that the object has contacted the screen.

7. The display device according to claim 6,

wherein the contact determination circuit extracts a region showing the object by using the difference image, and determines that the object has contacted the screen when an area of the region has reached a predetermined threshold value or more.

8. The display device according to claim 7,

wherein the coordinate calculation circuit calculates a center of gravity of the region where the area has reached the predetermined threshold value or more as the position coordinate of the object.

9. A display device, comprising:

a display unit including a display function to display an image on a screen, and an optical input function to image an image of an object which has approached the screen;

a modulation circuit configured to convert the imaged image into a multi-gradation image;

an edge detection circuit configured to detect edges by using the multi-gradation image;

an inter-frame difference processing circuit configured to obtain a difference image formed by taking a difference between the multi-gradation image in a current frame and the multi-gradation image in a past frame;

a coordinate calculation circuit configured to calculate a position coordinate of the object by using at least one of an edge image and the difference image; and

a contact determination circuit configured to determine whether or not the object has contacted the screen by using at least one of the edge image and the difference image.

11. The display device according to claim 9,

wherein the contact determination circuit counts a total number of pixels in each of which a gradation value in the difference image is larger or smaller than a predetermined threshold value, and performs the determination of the contact by using the number of pixels.

12. The display device according to claim 11,

wherein the determination of the contact using the number of pixels includes an arithmetic operation which corrects the total number of pixels to a value smaller than an original value thereof when the number of pixels is larger than a predetermined reference value.

13. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the difference image is larger or smaller than a predetermined threshold value, the contact determination circuit accumulates weight coefficients, each depending on a difference between the gradation value and the threshold value, and performs the determination of the contact by using a value as a result of the accumulation.

14. The display device according to claim 13,

wherein the weight coefficients are proportional to the differences between the gradation values and the threshold value.

15. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the difference image is larger or smaller than a predetermined threshold value, the coordinate calculation unit obtains the position coordinate by calculating an average value of positions of the pixels.

16. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the difference image is larger or smaller than a predetermined threshold value, the coordinate calculation circuit obtains the position coordinate by a center-of-gravity calculation using weight coefficients, each depending on a difference between the gradation value and the threshold value.

17. The display device according to claim 16,

wherein the weight coefficients are proportional to the differences between the gradation values and the threshold value.

18. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the edge image is larger or smaller than a predetermined threshold value, the contact determination circuit accumulates differences between the gradation values and the threshold value, and performs the contact determination by using a value as a result of the accumulation.

19. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the edge image is larger or smaller than a predetermined threshold value, the contact determination circuit accumulates weight coefficients, each depending on a difference between the gradation value and the threshold value, and performs the determination of the contact by using a value as a result of the accumulation.

20. The display device according to claim 19,

wherein the weight coefficients are proportional to the differences between the gradation values and the threshold value.

21. The display device according to claim 9,

wherein the contact determination circuit counts a total number of pixels in each of which a gradation value in the edge image is larger or smaller than a predetermined threshold value, and performs the determination of the contact by using the number of pixels.

22. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the edge image is larger or smaller than a certain threshold value, the coordinate calculation circuit obtains the position coordinate by calculating a coordinate of a center of gravity by using weight coefficients, each depending on a difference between the gradation value and the threshold value.

23. The display device according to claim 22,

wherein the weight coefficients are proportional to the differences between the gradation values and the threshold value.

24. The display device according to claim 9,

wherein, for pixels in each of which a gradation value in the edge image is larger or smaller than a threshold value, the coordinate calculation circuit obtains the position coordinate by calculating an average value of positions of the pixels.

25. The display device according to claim 22,

wherein the threshold value is changed depending on a distribution of the gradation values of the edge images.

26. The display device according to claim 25,

wherein an average value between a maximum value or a minimum value in the distribution of the gradation values of the edge images and the preset threshold value is set as a new threshold value.

27. The display device according to claim 9,

wherein an arithmetic operation in the determination of the contact includes a comparison arithmetic operation between a multiplied value and a predetermined threshold value, the multiplied value being obtained by multiplying a value obtained in such a way that the accumulated value of a predetermined number of frames before is subtracted from the accumulated value in a certain frame according to any one of claims 18 and 19, by the total number of pixels according to any one of claims 11 and 12.

28. The display device according to claim 9,

wherein an arithmetic operation in the determination of the contact includes a comparison arithmetic operation between a predetermined threshold value and a difference between a maximum value and minimum value of the accumulated value according to any one of claims 18 and 19 in a plurality of the frames.

29. The display device according to claim 9,

wherein an arithmetic operation in the determination of the contact includes a comparison arithmetic operation between the accumulated value according to any one of claims 18 and 19 and a predetermined threshold value.

30. The display device according to claim 9,

wherein an arithmetic operation in the determination of the contact includes a comparison arithmetic operation between a predetermined threshold value and a difference between a maximum value and minimum value of a plurality of the position coordinates obtained by using the edge images according to any one of claims 22 to 26 in a plurality of the frames, the position coordinates being prestored for the plurality of frames.

31. The display device according to claim 9,

wherein an arithmetic operation in the determination of the contact includes a comparison arithmetic operation between a predetermined threshold value and a distance between the position coordinate obtained by using the difference image according to any one of claims 15 to 17 and the position coordinate obtained by using the edge image according to any one of claims 22 to 26.

32. The display device according to claim 9,

wherein the determination of the contact is one, of which result is represented by respective states including an idle state, an approaching state, a contact state, and a separating state, and

the contact determination circuit includes a contact determination state machine configured to decide a state of a next frame by using a result of the comparison arithmetic operation according to any one of claims 27 to 31, a current state of the frames, a number of frames which have appeared in a current state, and a number of time-out frames set to correspond to each state.

10. A display device, comprising:

a display unit including a display function to display plural switches on a screen, and an optical input function to take an image of an object which has approached the switches;

a counting circuit configured to count a number of white pixels for each of the switches for the imaged image, to calculate a difference value between a counted value in a current frame and a counted value in a past frame, and to output, when a maximum one of the difference values for each of the switches is a predetermined threshold value or more, a signal indicating an effect that the object has contacted the switch;

an inter-frame difference processing circuit configured to obtain a difference image formed by taking a difference between a multi-gradation image in the current frame and a multi-gradation image in the past frame, to extract a region showing the object from the difference image, to calculate a center of gravity of the region, and to output a signal indicating an affect that the object has contacted the switch located at the center of gravity;

an edge detection circuit configured to detect edges from the multi-gradation images, to calculate a center of gravity of the edge in which intensity is a predetermined threshold value or more, and to output a signal indicating an effect that the object has contacted the switch located at the center of gravity;

a contact probability calculation circuit configured to calculate a contact probability for each of the switches based on the signals individually outputted from the counting circuit, the inter-frame difference circuit, and the edge detection circuit.

33. A display device, comprising:

a display unit configured to display an image on a screen, and to image an object which has approached the screen by using optical sensors; and

a sensor drive unit configured to change a drive condition of the optical sensors based on a statistical amount of output values of the optical sensors.

34. The display device according to claim 33,

wherein the drive condition is at least one of an exposure time and precharge time of the optical sensors.

35. The display device according to claim 33,

wherein the display unit is one configured to display, on a screen, a region which an object is made to approach, and

the sensor drive unit changes the drive condition based on a statistical amount of output values of the optical sensors arranged in a region on the screen, the region other than the region which the object is made to approach.

36. The display device according to claim 33,

wherein the sensor drive unit outputs a control signal which controls the drive condition so that upper gradation values including a center in a histogram of gradation values obtained by modulating the output values can be set within a predetermined range.

37. The display device according to claim 36,

wherein the sensor drive unit is capable changing the upper gradation values in response to sizes of the screen and the object.

38. The display device according to claim 36,

wherein the sensor drive unit decides the control signal by using a table showing a relationship between an exposure time of the optical sensors, a precharge voltage thereof, and the predetermined range.

39. The display device according to claim 38,

wherein the table is one, in which the exposure time is maximized in an allowable range, and the precharge voltage is changed.

40. The display device according to claim 38,

wherein the sensor drive unit decides the control signal by a linear search method by using the table.

41. The display device according to claim 38,

wherein the sensor drive unit decides the control signal by a binary search method by using the table.

42. The display device according to claim 36,

wherein the sensor drive unit uses gradation values of pixels which have been subjected to thinning processing performed for each of plural regions obtained by partitioning the screen in a fixed range.

43. The display device according to claim 33,

wherein the sensor drive unit provides a predetermined standby period between a point of time when the output values of the optical sensors are changed to an extent where the drive condition of the optical sensors is to be changed and a point of time when the drive condition of the optical sensors is actually changed.

44. The display device according to claim 43, further comprising a hold unit configured to hold the statistical amount during the predetermined standby period.

45. A display device, comprising:

a pixel region including plural pixels;

optical sensor elements provided for each of the pixels; and

a determination unit configured to compare a value to be determined, the value being based on multi-gradation data of each of the pixels, and a threshold value with each other in magnitude, the multi-gradation data corresponding to intensity of light received by the optical sensor elements.

46. The display device according to claim 45,

wherein the value to be determined is gradation values of the pixels in a region to be determined on the pixel region, and

the threshold value is a value located between an average value of gradation values of the pixels in a region from which reflected light reflected on an object has been detected and an average value of gradation values of the pixels in a region from which light other than the reflected light has been detected.

47. The display device according to claim 45,

wherein the value to be determined is a contrast value obtained by dividing a maximum value of gradation values of the pixels in a region to be determined on the pixel region by a minimum value thereof, and

the threshold value is a value located between a contrast value obtained by dividing a maximum value of gradation values of the pixels in a region from which reflected light reflected on an object has been detected by a minimum value thereof and a contrast value obtained by dividing a maximum value of gradation values of the pixels in a region from which light other than the reflected light has been detected by a minimum value thereof

48. The display device according to claim 45, further comprising a protection plate on the pixel region,

wherein, when an object is not present on a surface of the protection plate, multi-gradation data corresponding to reflected light reflected on the protection plate is set to be white.