Gestural Techniques with Wireless Mobile Phone Devices

• US 20080009310A1
• Filed: 05/25/2007
• Published: 01/10/2008
• Est. Priority Date: 11/03/1999
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey, Rodriguez, Tony
• Granted Patent: US 8,391,851 B2
• Time in Patent Office: Days
• US Class Current: 455/550.1
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

A user gestures with a wireless mobile phone device to control some aspect of its operation, or the operation of a remote system with which the device communicates. Application programs can solicit gestural information through a standardized set of interface protocols (e.g., API functions). One function, for example, solicits a running set of gesture data, and a complementary function discontinues the first function. Other functions solicit device movement data in particular directions. In some arrangements gesture data is sent by the wireless device to a remote system, which may issue a response that is dependent on the received gesture data. This response may also depend on other data obtained by the wireless device (e.g., obtained from an electronic or physical object, or sensed from the environment) and sent to the remote system. The wireless device may have different modes of interacting with users, to accommodate users with greater or lesser proficiency in gestural techniques. A variety of other features and arrangements are also detailed.

Claims

1. A wireless mobile phone device characterized by a user interface that is responsive to motion of said device, through which a user can signal to the mobile phone device by a corresponding movement of said device, the wireless mobile phone device including a processor running a program, and an application programming interface (API) comprising plural functions, wherein said API allows the program to issue queries soliciting wireless mobile phone device movement data.

2. The wireless mobile phone device of claim 1 that further includes a module which provides said movement data to the program in response to such queries.

3. The wireless mobile phone device of claim 2 in which said module includes a 2D image sensor.

4. The wireless mobile phone device of claim 1 in which at least one API function queries for data indicating movement of the wireless mobile phone device to the left or right.

5. The wireless mobile phone device of claim 1 in which at least one API function queries for data indicating movement of the wireless mobile phone device forwards or backwards.

6. The wireless mobile phone device of claim 1 in which at least one API function queries for data indicating movement of the wireless mobile phone device in a rotational state.

7. The wireless mobile phone device of claim 1 in which at least one API function queries for data indicating movement of the wireless mobile phone device in plural directions including:

left/right;

forward/backwards; and

rotational state.

8. A wireless mobile phone device including a processor running a program, and an application program interface (API), wherein said API enables the program to issue queries soliciting wireless mobile phone device gestural data.

9. The wireless mobile phone device of claim 8 wherein said API allows the program to solicit current movement data.

10. The wireless mobile phone device of claim 8 wherein said API includes a first function that solicits a running set of movement data, and a complementary function that discontinues the first function.

11. The wireless mobile phone device of claim 8 that further includes a module which provides gesture data to the program in response to queries through the API.

12. The wireless phone device of claim 8 in which said module includes a 2D image sensor.

13. A method of operating a wireless mobile phone device, comprising:

receiving a running set of gesture-related data;

formatting the received gesture-related data into a running set of packets; and

wirelessly transmitting said running set of packets from the wireless mobile phone device to a distinct device that can respond thereto.

14. The method of claim 13 that further includes generating said stream of gesture-related data.

15. The method of claim 14 wherein said generating includes tracking movement of a feature in plural frames of image data.

16. The method of claim 13 that further includes sensing data from an environment in which the wireless mobile phone device is located, and transmitting information related to said sensed data to said distinct device.

17. The method of claim 16 in which said sensing comprises sensing machine readable data from an object.

18. The method of claim 13 that further includes obtaining data corresponding to an object, and transmitting information related to said obtained data to said distinct device.

19. The method of claim 18 that includes obtaining data from an electronic object, by processing object data to yield an identifier.

20. The method of claim 19 in which said processing object data comprises applying a watermark decoding operation to said object data.

21. The method of claim 18 that includes obtaining data corresponding to the object by using a sensor.

22. The method of claim 21 that includes sensing machine readable data using said sensor.

23. A wireless mobile phone device that includes a user interface allowing gestural interaction with the device, said interface having first and second modes of operation, the second mode of operation supporting a richer set of gestures than the first mode, and being better suited for use by relatively more proficient users.

24. The method of claim 23 wherein the wireless mobile phone device provides a function that may be invoked by a user, wherein, in the first mode, said function is invoked by a first gesture, and in the second mode, said function is invoked by a second, different gesture.

25. The wireless mobile device of claim 23, wherein selection between said first and second modes is made in accordance with data sensed by a sensor included with said wireless mobile phone device.

26. The wireless mobile device of claim 23, wherein selection between said first and second modes is made in accordance with data sensed by a 2D optical sensor included with said wireless mobile phone device.