METHOD FOR CONTROLLING TRIGGERING OF HUMAN-COMPUTER INTERACTION OPERATION AND APPARATUS THEREOF

• US 20150293589A1
• Filed: 06/25/2015
• Published: 10/15/2015
• Est. Priority Date: 12/28/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Inventors: ZHOU, Bin, SHENG, Sen
• Granted Patent: US 9,829,974 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method for controlling triggering of a human-computer interaction operation and an apparatus are provided according to the embodiments of the disclosure. The method includes: acquiring images shot by a shooting device, and displaying the acquired images on a display screen in a blurring manner; detecting a difference between acquired images in each frame, recognizing a designated outline on the image according to the difference between acquired images in each frame, and calculating a location of the recognized designated outline on the display screen; and determining, in real time, whether the designated outline at the location on the display screen intersects with a designated area displayed on the display screen, and triggering an operation corresponding to the designated area if the designated outline at the location on the display screen intersects with the designated area displayed on the display screen.

Claims

1. A method for controlling triggering of a human-computer interaction operation, comprising:

acquiring images shot by a shooting device, and displaying the acquired images on a display screen in a blurring manner;

detecting a difference between the acquired images in each frame, recognizing a designated outline on the acquired image according to the difference between the acquired images in each frame, and calculating a location of the recognized designated outline on the display screen; and

determining, in real time, whether the designated outline at the location on the display screen intersects with a designated area displayed on the display screen, and triggering an operation corresponding to the designated area if the designated outline at the location on the display screen intersects with the designated area displayed on the display screen.

2. The method according to claim 1, wherein

in case that the designated outline at the location on the display screen intersects with the designated area displayed on the display screen, the triggered operation corresponding to the designated area, comprises:

recording a period of time in which the designated outline intersects with the designated area;

determining whether the designated outline on the display screen moves out of the designated area;

stopping recording the intersection time if the designated outline on the display screen moves out of the designated area, and continuing recording the period of time if the designated outline on the display screen does not move out of the designated area.

3. The method according to claim 2, wherein the method further comprises:

calculating billing information corresponding to the designated area according to the recorded period of time in which the designated outline intersects with the designated area.

4. The method according to claim 1, wherein

in case that the designated outline at the location on the display screen intersects with the designated area displayed on the display screen, the triggered operation corresponding to the designated area comprises:

detecting whether a designated motion form of the designated outline occurs; and

triggering an instruction operation bound with the designated area if the designated motion form of the designated outline occurs.

5. The method according to claim 4, wherein the process of detecting whether a designated motion form of the designated outline occurs comprises

creating a template of the designated outline;

detecting images in each frame in the template of the designated outline; and

determining whether a change between the acquired images in each frame satisfies the designated motion form, and triggering the instruction operation bound with the designated area if the change between the acquired images in each frame satisfies the designated motion form.

6. The method according to claim 4, wherein

the designated outline is an eye outline;

and

the process of detecting whether a designated motion form of the designated outline occurs comprises;

detecting whether the eye outline blinks.

7. The method according to claim 5, wherein

the designated outline is an eye outline;

and

the process of detecting whether a designated motion form of the designated outline occurs comprises;

detecting whether the eye outline blinks.

8. The method according to claim 6, wherein the process of detecting whether the eye outline blinks comprises:

detecting boundary values of the eye outline;

detecting a maximum value and a minimum value of the boundary values; and

detecting whether a difference between the maximum value and the minimum value of the boundary values decreases and then increases, and determining that the eye outline blinks if the difference between the maximum value and the minimum value of the boundary values decreases and then increases.

9. The method according to claim 7, wherein the process of detecting whether the eye outline blinks comprises:

detecting boundary values of the eye outline;

detecting a maximum value and a minimum value of the boundary values; and

detecting whether a difference between the maximum value and the minimum value of the boundary values decreases and then increases, and determining that the eye outline blinks if the difference between the maximum value and the minimum value of the boundary values decreases and then increases.

10. The method according to claim 1, wherein

the designated area displayed on the display screen is a display area of designated media information or a designated instruction area.

11. The method according to claim 2, wherein

the designated area displayed on the display screen is a display area of designated media information or a designated instruction area.

12. The method according to claim 3, wherein

the designated area displayed on the display screen is a display area of designated media information or a designated instruction area.

13. The method according to claim 4, wherein

the designated area displayed on the display screen is a display area of designated media information or a designated instruction area.

14. The method according to claim 5, wherein

the designated area displayed on the display screen is a display area of designated media information or a designated instruction area.

15. An apparatus for controlling triggering of a human-computer interaction operation, wherein the apparatus comprises at least one processor and a memory having processor-executable instructions stored therein, and the instructions when executed by the at least one processor, configure the apparatus to:

acquire images shot by a shooting device, and display the acquired images on a display screen in a blurring manner;

detect a difference between images in each frame, recognize a designated outline on the shot image according to the difference between acquired images in each frame, and calculate a location of the recognized designated outline on the display screen; and

determine, in real time, whether the designated outline at the location on the display screen intersects with a designated area displayed on the display screen, and trigger an operation corresponding to the designated area if the designated outline at the location on the display screen intersects with the designated area displayed on the display screen.

16. The apparatus according to claim 15, wherein the apparatus is further configured to:

trigger the operation corresponding to the designated area in case it is determined that the designated outline at the location on the display screen intersects with the designated area displayed on the display screen,

wherein the triggered operation comprises the apparatus configured to;

record a period of time in the designated outline intersects with the designated area;

determine whether the designated outline on the display screen moves out of the designated area;

stop recording the period of time if the designated outline on the display screen moves out of the designated area, and continue recording the period of time if the designated outline on the display screen does not move out of the designated area.

17. The apparatus according to claim 15, wherein the apparatus is further configured to:

trigger the operation corresponding to the designated area in case it is determined that the designated outline at the location on the display screen intersects with the designated area displayed on the display screen;

wherein the triggered operation comprises the apparatus configured to;

detect whether a designated motion form of the designated outline occurs, and

trigger an instruction operation bound with the designated area, if the designated motion form of the designated outline occurs.

18. The apparatus according to claim 17, wherein the designated outline is an eye outline;

and the apparatus is further configured to detect whether the eye outline blinks.

19. The apparatus according to claim 15, wherein the designated area displayed on the display screen comprises a display area of designated media information or a designated instruction area.

20. The apparatus according to claim 16, wherein the designated area displayed on the display screen comprises a display area of designated media information or a designated instruction area.