Gesture recognition

• US 7,372,993 B2
• Filed: 07/21/2004
• Issued: 05/13/2008
• Est. Priority Date: 07/21/2004
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Gonzalez, Andreu, Lagardere, Xavier
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US10/895,740
• Publication Number:
  US 20060018546A1
• Time in Patent Office: 1,392 Days
• Field of Search: 382186-189, 382309-310, 382/176, 715/530
• US Class Current: 382/186
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A gesture related to portions of handwritten text is recognized. Furthermore, an action is performed against the portions of the handwritten text. The action is associated with the recognized gesture.

Claims

1. A method comprising:

recognizing a plurality of gestures related to portions of previously recorded handwritten text supplied by a user at a first time, wherein the gestures and the handwritten text are maintained separately from one another and the gestures maintained as metadata for the portions of the handwritten text to which it relates and the gestures supplied at a second time; and

performing, in a batch fashion, actions associated with the gestures against the portions of the handwritten text.

2. The method of claim 1 further comprising, performing intelligent character recognition against at least one of the portions of the handwritten text and remaining portions of the handwritten text.

3. The method of claim 1, wherein recognizing further includes capturing the gestures with a digital pen.

4. The method of claim 1, wherein recognizing further includes identifying the gestures from a library of configurable symbols associated with a writer.

5. The method of claim 1, wherein performing further includes performing at least one of:

deleting the portions of the handwritten text;

altering the appearance of the portions of the handwritten text;

altering font characteristics of the portions of the handwritten text;

altering color or texture characteristics of the portions of the handwritten text;

inserting other handwritten text before or after the portions of the handwritten text;

associating a comment with the portions of the handwritten text; and

processing a custom application against the portions of the handwritten text.

6. The method of claim 1, wherein recognizing further includes capturing the portions of the handwritten text before recognizing gestures, and wherein the gestures can overlap some of the portions of the handwritten text.

7. A method comprising:

identifying a writer by a writer identification once a writer has logged in;

receiving one or more symbols from the writer;

receiving one or more actions from the writer for each of the one or more symbols; and

recording the one or more symbols and the one or more actions in a library associated with the writer identification, and wherein the library includes other symbols associated with other writers and selection of proper symbols is made based on a specific writer identification, and wherein symbols from different writers can be used in connection with a piece of handwritten text to identify the actions to take against that handwritten text.

8. The method of claim 7 further comprising, accessing the library from a remote device which is external to a device that is processing the method.

9. The method of claim 7 further comprising, accessing the library within a device that is processing the method.

10. The method of claim 7, wherein receiving the one or more symbols further include receiving input identifying a gesture for each of the one or more symbols from an input device controlled by the writer.

11. The method of claim 7, wherein receiving the one or more actions further include identifying the one or more actions as at least one of instructions to alter the appearance of handwritten text, instructions to delete or insert the handwritten text, instructions to process custom applications against the handwritten text, and instructions to add comments to the handwritten text.

12. The method of claim 7, wherein receiving the one or more actions further include receiving selections from a list of available actions from the writer.

13. The method of claim 7, wherein receiving the one or more actions further include receiving at least one of the one or more actions as a compound action that includes a plurality of the one or more actions.

14. A system, comprising:

a symbol library;

a gesture recognizer; and

a gesture controller, wherein the gesture recognizer, in a batch fashion, detects gestures of a writer and locates the gestures in the symbol library for that writer to acquire actions, the gesture controller processes the actions against previously recorded handwritten text associated with the detected gestures, and wherein the gestures are maintained separately from the handwritten text and carried as metadata associated with the handwritten text.

15. The system of claim 14 further comprising, an input device interfaced to the gesture recognizer, wherein the input device includes at least one of a digital pen, a digital stylus, and a digital camera.

16. The system of claim 14 further comprising, at least one of an Intelligent Character Recognition (ICR) engine and an Optical Character Recognition (OCR) engine that translates the handwritten text into machine readable characters after the gesture controller processes the actions against the handwritten text.

17. The system of claim 14, wherein at least some portions of the gestures overlay portions of the handwritten text.

18. The system of claim 14, wherein at least some portions of the gestures are adjacent to or surround portions of the handwritten text.

19. The system of claim 14, wherein the symbol library includes writer-defined symbols that correspond to the gestures based on identities of writers.

20. A computer-readable medium encoded with computer executable instructions for recognizing gestures, the instructions when executed performing the method comprising:

capturing handwriting text at a first time;

recognizing a plurality of gestures in a batch fashion and associated with a second time and that are associated with portions of the handwritten text, and wherein the gestures are maintained separately from the handwritten text as metadata for the handwritten text; and

processing actions in a batch fashion, wherein the actions are associated with the gestures and processed against the portions of the handwritten text.

21. The medium of claim 20, wherein recognizing further includes instructions for searching symbols that substantially matches the gestures.

22. The medium of claim 21, wherein recognizing further includes instructions for acquiring the actions from data associated with the symbols in the symbol library.

23. The medium of claim 21, wherein searching further includes instructions for searching the symbol library with a writer'"'"'s identification and the gestures.

24. The medium of claim 20, further including instructions for recoding writer-defined gestures based on identities of writers.

25. The medium of claim 20, further including instructions for maintaining versions for the handwritten text, wherein one version is maintained before processing the actions and another version is maintained after processing the actions.

26. The medium of claim 25, wherein maintaining further includes maintaining additional metadata that permits each of the versions to be reproduced, wherein the additional metadata identifies changes that need to be applied or undone to an existing state of the handwritten text.