Electronic apparatus with sensor-integrated display device and method of controlling the same

• US 9,740,395 B2
• Filed: 03/14/2014
• Issued: 08/22/2017
• Est. Priority Date: 03/29/2013
• Status: Active Grant

• Current Assignee: Japan Display Incorporated
• Original Assignee: Japan Display Incorporated
• Inventors: Yamada, Jouji, Nakagawa, Hirofumi, Yamamoto, Michio, Azumi, Kohei, Hayashi, Makoto, Mizuhashi, Hiroshi, Ikeno, Kozo, Kida, Yoshitoshi
• Primary Examiner(s): Crawley, Keith
• Application Number:
  US14/211,818
• Publication Number:
  US 20140292687A1
• Time in Patent Office: 1,257 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/117 Biometrics derived from hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

According to one embodiment, an electronic apparatus includes a sensor-integrated display device integrally including a display surface and a sensor surface, a data transfer device, an image data generation module, and a processing module. The data transfer device is configured to generate and output three-dimensional information. The image data generation module is configured to generate three-dimensional image data at a plurality of sensing points on the sensor surface based on the three-dimensional information. The processing module is configured to analyze an operation content of a conductor above the sensor surface based on the image data.

Claims

1. An electronic apparatus comprising:

a sensor-integrated display device integrally comprising a display surface configured to output display information and a sensor surface configured to input operation information, wherein the sensor surface senses three-dimensional data which is an analog signal containing coordinates and a capacitance value;

a data transfer device configured to generate and output three-dimensional information which is a digital signal by digitizing the three-dimensional data sensed at the sensor surface; and

an application operation device comprising an image data generation module configured to generate three-dimensional image data at a plurality of sensing points on the sensor surface based on the three-dimensional information output by the data transfer device and a processing module configured to analyze an operation content of a conductor above the sensor surface based on the image data generated by the image data generation module,

wherein

the sensor-integrated display device comprises an array substrate including a common electrode, and a countersubstrate opposed to the array substrate and including a sensor detection electrode on the sensor surface,

the data transfer device is configured to supply a common voltage to the common electrode in a display period for outputting the display information, and to supply a sensor drive signal to the common electrode and detect a sensor signal output from the sensor detection electrode in a blanking period,

the data transfer device includes a sensor signal detector configured to slice the analog signal, perform noise reduction for the sliced signal, and generate the noise-reduced raw read image data as the three-dimensional information, the sensor signal detector including a timing controller, an integration circuit, a sample-and-hold and analog-to-digital converter, and a digital filter,

the data transfer device is configured to transfer the three-dimensional information to the image data generation module in synchronization with display drive timing for displaying the display information on the display surface,

the data transfer device further includes a timing circuit and a digital-to-analog converter configured to output display data to the sensor-integrated display device and to output a blanking detection signal to the timing controller according to the display data supplied from the application operation device, and

the timing controller is configured to execute varying frequencies of access pulses output to the sensor detection electrode and to set output timing on the access pulses with a signal from the application operation device, is configured to supply a reset timing signal to the integration circuit, and is configured to supply a clock to the sample-and-hold and analog-to-digital converter and the digital filter.

2. The apparatus according to claim 1, wherein the three-dimensional information is operation information indicating proximity of the conductor to the sensor surface.

3. The apparatus according to claim 1, further comprising a coordinate processing module,

wherein each of the image data generation module and the coordinate processing module is configured to enable processing of multiple types of applications and is provided in an application operation device formed of a semiconductor integrated circuit comprising a baseband engine.

4. The apparatus according to claim 1, wherein the image data generation module generates the image data based on the three-dimensional information on all of the sensing points on the sensor surface in synchronization with display drive timing for displaying the display information on the display surface.

5. The apparatus according to claim 1, wherein the processing module comprises application processing module configured to activate application processing associated with the analyzed operation content.

6. The apparatus according to claim 1, wherein the processing module comprises an operation input recognition processing module configured to recognize an image of a specific portion of a human body of an operator who operates above the sensor surface.

7. The apparatus according to claim 1, wherein the processing module comprises a biometric authentication processing module configured to execute biometric authentication based on at least one of physical characteristics and behavioral characteristics of an operator who operates above the sensor surface.

8. A method of controlling an electronic apparatus comprising a sensor-integrated display device integrally comprising a display surface configured to output display information and a sensor surface configured to input operation information, a data transfer device, and an application operation device, the sensor-integrated display device comprising an array substrate including a common electrode, and a countersubstrate opposed to the array substrate and including a sensor detection electrode on the sensor surface, the data transfer device comprising a sensor signal detector including a timing controller, an integration circuit, a sample-and-hold and analog-to-digital converter, and a digital filter, the method comprising:

sensing three-dimensional data which is an analog signal containing coordinates and a capacitance value at the sensor surface;

generating and acquiring three-dimensional information which is a digital signal by digitizing the three-dimensional data sensed at the sensor surface;

generating three-dimensional image data at a plurality of sensing points on the sensor surface based on the acquired three-dimensional information;

recognizing an operation content of a conductor above the sensor surface based on the generated three-dimensional image data;

controlling an application operation in accordance with the recognized operation content;

supplying a common voltage to the common electrode in a display period for outputting the display information, and supplying a sensor drive signal to the common electrode and detecting a sensor signal output from the sensor detection electrode in a blanking period;

outputting display data to the sensor-integrated display device and outputting a blanking detection signal to the timing controller according to the display data supplied from the application operation device; and

executing varying frequencies of access pulses output to the sensor detection electrode and setting output timing on the access pulses with a signal from the application operation device, supplying a reset timing signal to the integration circuit, and supplying a clock to the sample-and-hold and analog-to-digital converter and the digital filter,

wherein

generating the three-dimensional information comprises slicing the analog signal and performing noise reduction for the sliced signal to produce noise-reduced raw read image data as the three-dimensional information,

the three-dimensional information is operation information indicating proximity of the conductor to the sensor surface, and

the three-dimensional information is acquired by block transfer synchronized with display drive timing for displaying the display information on the display surface.