Recognizing interactive media input

• US 9,207,765 B2
• Filed: 12/31/2009
• Issued: 12/08/2015
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ma, Lei, Huo, Qiang
• Primary Examiner(s): EURICE, MICHAEL
• Application Number:
  US12/651,305
• Publication Number:
  US 20110157012A1
• Time in Patent Office: 2,168 Days
• Field of Search: 345/156, 345/158, 345/157, 345/619, 345/642, 715/863, 715/856, 715/702, 715/716
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/798 for assessing skills or for ranking players, e.g. for generating a hall of fame computing the game score A63F13/46
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4753 for user identification, e.g. by entering a PIN or password cryptographic authentication protocols H04L9/32; networks authentication protocols H04L63/08
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72

Abstract

Techniques and systems for inputting data to interactive media devices are disclosed herein. In some aspects, a sensing device senses an object as it moves in a trajectory indicative of a desired input to an interactive media device. Recognition software may be used to translate the trajectory into various suggested characters or navigational commands. The suggested characters may be ranked based on a likelihood of being an intended input. The suggested characters may be displayed on a user interface at least in part based on the rank and made available for selection as the intended input.

Claims

1. A method comprising:

calibrating an interactive media device to identify and visually track;

an object moved through space by a user;

recording a trajectory of a movement of the object through space extracted from a multi-frame video sequence, the trajectory directly representing a desired handwritten character;

displaying the trajectory on a display device;

based at least in part on a shape of the trajectory, translating the trajectory into two or more suggested characters;

ranking each of the two or more suggested characters based on a likelihood that each of the two or more suggested characters are the desired handwritten character;

simultaneously displaying the two or more suggested characters on the display device so that display positions of each of the two or more suggested characters relative to an endpoint of the trajectory are based on the ranking of each of the two or more suggested characters, wherein the ranking is based on the likelihood that each of the two or more suggested characters are the desired handwritten character; and

receiving selection information indicating which of the two or more suggested characters represents the desired handwritten character.

2. The method as recited in claim 1, wherein calibrating the interactive media device comprises:

receiving a calibration trigger event;

displaying a calibration zone on the display device in response to the receipt of the calibration trigger event; and

locking-in on the object when the object is substantially placed inside of the calibration zone.

3. The method as recited in claim 1, wherein translating the trajectory into two or more suggested characters includes at least a static recognition element, a dynamic recognition element, and a combiner of multiple recognition elements.

4. The method as recited in claim 1, further comprising:

recording a gesture performed via the object, the gesture indicative of a navigational command;

translating the gesture into a navigational input; and

communicating the navigational input to control a navigational feature of the interactive media device.

5. The method as recited in claim 1, wherein translating the trajectory comprises one or more of smoothing the trajectory or enhancing the trajectory.

6. The method as recited in claim 1, further comprising calibrating the interactive media device to identify and visually track at least a portion of the user.

7. The method as recited in claim 1, further comprising displaying a live video stream of the at least a portion of the user with the trajectory on the display device.

8. One or more computer-readable memory devices comprising processor-executable instructions that, when executed, direct a device to perform actions comprising:

receiving, from a visual sensing device, a video stream of a trajectory of an object moved through space by a user;

receiving an input mode directive to designate object tracking as a handwriting mode;

receiving a start input signal to start recording the trajectory of the object through space;

receiving a stop input signal to stop the recording of the trajectory while continuing to receive the input signal of the trajectory of the object through space from the visual sensing device or the inertial sensing device, wherein a shape of the recorded trajectory of the object through space between the start input signal and the stop input signal represents a handwritten character;

based at least in part on a shape of the recorded trajectory, translating the recorded trajectory into one or more suggested characters; and

displaying the one or more suggested characters on a display device.

9. The one or more computer-readable memory devices of claim 8 wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising

individually ranking the one or more suggested characters based on a likelihood of the individual one or more suggested characters being an intended input such that a display area of a highest ranked suggested character is larger than a display area of a non-highest ranked suggested character on the display device.

10. The one or more computer-readable memory devices of claim 8, wherein a camera records a movement of the object through space as a multi-frame video sequence from which the trajectory is extracted.

11. The one or more computer-readable memory devices of claim 8, wherein the object includes the inertial sensing device and wherein the inertial sensing device records a movement of the object through space as a sequence of inertial sensor signals from which the trajectory is extracted.

12. The one or more computer-readable memory devices of claim 8, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising

displaying a real-time image of the trajectory on the display device.

13. The one or more computer-readable memory devices of claim 8, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising:

receiving selection information indicating which one of the one or more suggested characters is a command input recognizable by an interactive media device.

14. The one or more computer-readable memory devices of claim 13, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising

inserting the one or more suggested characters into an input box of the interactive media device.

15. A method comprising:

recording a trajectory of a movement of an object moved through space by a user, wherein the trajectory is extracted from a multi-frame video sequence via a camera, and wherein the trajectory directly represents a desired handwritten character;

displaying the trajectory on a display device;

based at least in part on a shape of the trajectory, translating the trajectory into a plurality of suggested inputs comprising characters;

simultaneously displaying the plurality of suggested inputs on a display device so that display positions of the plurality of suggested inputs relative to an endpoint of the trajectory are based on a likelihood of each of the plurality of suggested inputs being the desired handwritten character;

receiving a selection of the desired handwritten character, the desired handwritten character being one of the plurality of suggested inputs; and

controlling an interactive media device via the desired handwritten character.

16. The method as recited in claim 15, wherein controlling the interactive media device comprises placing the desired handwritten character into an input box of the interactive media device.

17. The method as recited in claim 16, wherein the desired handwritten character is an East Asian character in a traditional form.