Method and apparatus for marking electronic document

• US 9,207,762 B2
• Filed: 04/14/2014
• Issued: 12/08/2015
• Est. Priority Date: 10/25/2013
• Status: Active Grant

• Current Assignee: Utechzone Co., Ltd.
• Original Assignee: Utechzone Co., Ltd.
• Inventors: Tsou, Chia-Chun, Lin, Po-Tsung, Kao, Chia-Wen
• Primary Examiner(s): Yodichkas, Aneeta
• Assistant Examiner(s): Wills-Burns, Chineyere
• Application Number:
  US14/251,648
• Publication Number:
  US 20150116201A1
• Time in Patent Office: 603 Days
• Field of Search: 345/156
• US Class Current: 1/1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A method and an apparatus for marking electronic document are provided. An image sequence of a user is captured by an image capturing unit, and an eye tracking procedure, a nod detecting procedure and a blink detecting procedure are executed for analyzing the image sequence. A first position on the electronic document where a current sightline of the user falls when the user performs a first specific action is obtained, and a marking action is executed for marking the electronic document along with a movement trajectory of the user'"'"'s eyes starting from the first position. A second position on the electronic document where the current sightline falls when the user performs a second specific action is obtained, and the marking action is finished at the second position.

Claims

1. A method for marking electronic document, comprising:

performing a correction procedure before an eye tracking procedure is executed, wherein the correction procedure includes sequentially receiving a plurality of correction images, and generating a coordinate transformation matrix according to the correction images;

displaying an electronic document in a display unit;

capturing an image sequence of a user by an image capturing unit;

executing the eye tracking procedure, a nod detecting procedure and a blink detecting procedure to analyze the image sequence, wherein the eye tracking procedure is configured to track a movement trajectory of eyes of the user, and the nod detecting procedure and the blink detecting procedure are configured to detect whether the user performs a first specific action or a second specific action;

wherein the eye tracking procedure includes;

detecting an eye area of a current image in the image sequence to obtain a pupil position in the current image;

obtaining a comparison correction parameter according to two bright spot positions of the eye area of the current image;

obtaining a dynamic compensation parameter according to a reference correction parameter and the comparison correction parameter, wherein the reference correction parameter is obtained based on the two bright spot positions of the eye area in each of the correction images;

calculating eye movement coordinates according to the pupil position in the current image, the dynamic compensation parameter and one of the two bright spot positions of the current image;

transforming the eye movement coordinates into sightline falling coordinates corresponding to the display unit by using the coordinate transformation matrix; and

recording the sightline falling coordinates;

obtaining a first position on the electronic document where a current sightline of the user falls through the eye tracking procedure when detecting the user performs the first specific action;

executing a marking action for marking the electronic document along with the movement trajectory of the user'"'"'s eyes starting from the first position;

obtaining a second position on the electronic document where the current sightline of the user falls through the eye tracking procedure when detecting the user performs the second specific action; and

finishing the marking action at the second position.

2. The method for marking electronic document as claimed in claim 1, wherein

the correction images are respectively obtained as the user views a plurality of correction points of the display unit;

the two bright spot positions are formed when the image capturing unit captures each of the correction images; and

the coordinate transformation matrix is used for transforming a coordinate position of the eye area into a coordinate position of the display unit.

3. The method for marking electronic document as claimed in claim 1, wherein the first specific action and the second specific action respectively comprise at least one of a nod action and a blink action, and the second specific action is the same or different to the first specific action.

4. The method for marking electronic document as claimed in claim 1, wherein the marking action comprises at least one of actions of adjusting brightness, adding a bottom line and adding a background color.

5. The method for marking electronic document as claimed in claim 1, wherein the electronic document comprises a plurality of segments, and the method further comprises:

executing the corresponding marking action to each of the segments according to a viewing frequency of each of the segments.

6. An apparatus for marking electronic document, comprising:

a display unit, displaying an electronic document

an image capturing unit, capturing an image sequence of a user;

a storage unit, storing the electronic document and the image sequence; and

a processing unit, coupled to the display unit, the image capturing unit and the storage unit, and executing a correction module, an eye tracking module, a nod detecting module and a blink detecting module to analyze the image sequence, wherein the correction module is configured to sequentially receive a plurality of correction images, and generate a coordinate transformation matrix according to the correction images;

the eye tracking module is configured to track a movement trajectory of eyes of the user, and the nod detecting module and the blink detecting module are configured to detect whether the user performs a first specific action or a second specific action,

wherein the eye tracking module detects an eye area of a current image in the image sequence to obtain a pupil position in the current image;

obtains a comparison correction parameter according to two bright spot positions of the eye area of the current image;

obtains a dynamic compensation parameter according to a reference correction parameter and the comparison correction parameter, wherein the reference correction parameter is obtained based on the two bright spot positions of the eye area in each of the correction images;

calculates eye movement coordinates according to the pupil position in the current image, the dynamic compensation parameter and one of the two bright spot positions of the current image;

transforms the eye movement coordinates into sightline falling coordinates corresponding to the display unit by using the coordinate transformation matrix; and

records the sightline falling coordinates;

wherein the eye tracking module obtains a first position on the electronic document where a current sightline of the user falls when detecting that the user performs the first specific action, and the processing unit activates a marking module to execute a marking action for marking the electronic document along with the movement trajectory of the user'"'"'s eyes starting from the first position;

the eye tracking module obtains a second position on the electronic document where the current sightline of the user falls when detecting that the user performs the second specific action, and the marking module finishes the marking action at the second position.

7. The apparatus for marking electronic document as claimed in claim 6, wherein

the correction images are respectively obtained as the user views a plurality of correction points of the display unit, the two bright spot positions are formed when the image capturing unit captures each of the correction images, and the coordinate transformation matrix is used for transforming a coordinate position of the eye area into a coordinate position of the display unit.

8. The apparatus for marking electronic document as claimed in claim 6, wherein the first specific action and the second specific action respectively comprise at least one of a nod action and a blink action, and the second specific action is the same or different to the first specific action.

9. The apparatus for marking electronic document as claimed in claim 6, wherein the marking action comprises at least one of actions of adjusting brightness, adding a bottom line and adding a background color.

10. The apparatus for marking electronic document as claimed in claim 6, wherein the electronic document comprises a plurality of segments, and the marking module executes the corresponding marking action to each of the segments according to a viewing frequency of each of the segments.