Motion-based text input

• US 7,355,583 B2
• Filed: 08/10/2004
• Issued: 04/08/2008
• Est. Priority Date: 08/10/2004
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: van Baar, Jeroen, Beardsley, Paul A.
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US10/915,223
• Publication Number:
  US 20060033702A1
• Time in Patent Office: 1,337 Days
• Field of Search: 345156-157, 345/168, 345/169, 345/419, 341/22, 348/744, 348/43, 353/42, 353/30, 353/69, 356/602, 715/773, 715/727, 715/729, 382/154, 349/6
• US Class Current: 345/156
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks

Abstract

A method generates input text with a pointing device at a distance from a writing surface. Motion information of a mobile coordinate frame of the pointing device is sensed relative to a stationary coordinate frame of the writing surface using a motion sensor configured to track motion of the pointing device. The input text is then determined from the motion information. A projection of a track of the motion of the pointing device on the writing surface can also be determined from the motion information, and the track can be displayed on the writing surface as the input text.

Claims

1. A method for generating input text with a pointing device, in which the pointing device is at a distance from a writing surface, comprising:

sensing motion information of a mobile coordinate frame of a pointing device relative to a stationary coordinate frame of a writing surface at a distance from the pointing device using a motion sensor fixed to the pointing device and configured to track motion of the pointing device;

defining a track of the pointing device from the motion information;

inputting the track into a script recognizer, in which the script recognizer determines input text corresponding to the motion information.

2. The method of claim 1, further comprising:

determining a projection of a track of the motion of the pointing device on the writing surface from the motion information; and

displaying the track on the writing surface.

3. The method of claim 2, in which the input text is determined by recognizing a shape of the track.

4. The method of claim 2, in which processing of the input text depends on a location of the displayed input text on the writing surface.

5. The method of claim 2, in which processing of the input text depends on characteristics of the writing surface.

6. The method of claim 2, in which the input text is displayed on the writing surface according to a selected font.

7. The method of claim 1, further comprising:

displaying the input text on the writing surface.

8. The method of claim 1, further comprising:

storing the input text in a memory.

9. The method of claim 8, further comprising:

storing a location of the pointing device relative to the writing surface; and

associating the location with the stored input text.

10. The method of claim 1, further comprising:

transmitting the input text.

11. The method of claim 1, in which the pointing device includes a projector, and further comprising:

displaying the input text on the writing surface using the projector.

12. The method of claim 11, in which a direction of pointing the pointing device is defined by a particular pixel on an image plane of the projector.

13. The method of claim 1, in which the pointer includes an on-off button, and in which the determining is done only during a period of time when the on-off button is on.

14. The method of claim 1, in which the determining of the input text is modified according to a 3D position of the pointing device relative to the writing surface.

15. The method of claim 14, further comprising:

partitioning an area in front of the writing surface into multiple regions; and

modifying the determining of the input text according to a particular region occupied by the pointing device.

16. The method of claim 1, further comprising:

modifying the determining of the input text according to forward or backward motion of the pointing device relative to the writing surface.

17. The method of claim 1, in which the pointing device includes a camera, and further comprising:

sensing the motion information using the camera.

18. The method of claim 17, in which the input text is determined according to a color of the writing surface as sensed by the camera.

19. The method of claim 1, in which the input text is determined using a cursive script recognizer.

20. The method of claim 1, in which the input text is determined using a uni-stroke alphabet recognizer.

21. The method of claim 1, further comprising:

sensing the motion information of a plurality of mobile coordinate frames of a plurality of pointing devices relative to the stationary coordinate frame of the writing surface at the distance from the plurality of pointing devices, each of the plurality of pointing device including a particular motion sensor; and

determining input text from the motion information of each of the plurality of pointing devices.

22. The method of claim 21, further comprising:

displaying the input text of each of the plurality of pointing devices the writing surface.

23. The method of claim 1, in which the distance is one meter or greater.

24. An apparatus for generating input text, comprising:

a pointing device including a motion sensor;

means for sensing motion information of a mobile coordinate frame of the pointing device relative to a stationary coordinate frame of a writing surface at a distance from the pointing device using the motion sensor;

means for defining a track of the pointing device from the motion information; and

a script recognizer configured to determine input text corresponding to the motion information.

25. The apparatus of claim 24, in which the pointing device further comprises:

a projector configured to display the input text on the writing surface.