SWITCHING BETWEEN A FIRST OPERATIONAL MODE AND A SECOND OPERATIONAL MODE USING A NATURAL MOTION GESTURE

• US 20120324213A1
• Filed: 06/23/2010
• Published: 12/20/2012
• Est. Priority Date: 06/23/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ho, Casey Kwok Ching, Nanavati, Sharvil
• Granted Patent: US 8,581,844 B2
• Time in Patent Office: Days
• US Class Current: 713/100
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A mobile device is operative to change from a first operational mode to a second or third operational mode based on a user'"'"'s natural motion gesture. The first operational mode may include a voice input mode in which a user provides a voice input to the mobile device. After providing the voice input to the mobile device, the user then makes a natural motion gesture and a determination is made as to whether the natural motion gesture places the mobile device in the second or third operational mode. The second operational mode includes an augmented reality display mode in which the mobile device displays images recorded from a camera overlaid with computer-generated images corresponding to results output in response to the voice input. The third operational mode includes a reading display mode in which the mobile device displays, without augmented reality, results output in response to the voice input.

Claims

1. A system for switching between the operational mode of a mobile device from a first operational mode to a second operational mode using a natural motion gesture comprising:

a storage device operative to;

store a first set of baseline data representative of a first state of a mobile device; and

,

store a second set of baseline data representative of a second state of the mobile device;

an orientation sensor operative to;

detect a first orientation of the mobile device and output a first set of orientation data representative of the mobile device being in the first orientation; and

,

detect a second orientation of the mobile device and output a second set of orientation data representative of the mobile device being in the second orientation; and

,

a processor operative to;

change the operational mode of the mobile device from a first mode to a second mode when;

the first set of orientation data approximates

the first set of baseline data; and

,

the second set of orientation data approximates

the second set of baseline data;

wherein;

the first operational mode comprises an input receiving mode operative to receive an input from a user; and

,

the second operational mode comprises a visual display mode operative to;

display a selected portion of the real-world environment surrounding the mobile device overlaid with at least one computer-generated graphic element.

2. The system of claim 1, wherein:

the first state of the mobile device comprises the mobile device being in the first operational mode; and

,

the input receiving mode comprises receiving a voice input from the user.

3. The system of claim 1, wherein the second state of the mobile device comprises the mobile device being in the second operational mode.

4. The system of claim 1, wherein the first set of orientation data indicates that a front surface of the mobile device is positioned approximately against the head of the user.

5. The system of claim 1, wherein the second set of orientation data indicates that a front surface of the mobile device is approximately perpendicular to a ground surface.

6. The system of claim 1, wherein the second set of orientation data indicates that a front surface of the mobile device forms an approximate acute angle to a ground surface.

7. The system of claim 1, wherein:

the memory storage device is further operative to store a third set of baseline data representative of a transition of the mobile device from the first state to the second state;

the orientation sensor is further operative to detect transitional orientations of the mobile device changing from first orientation to the second orientation, and output a third set of transitional data representative of the transitional orientations; and

,

the processor is further operative to change the operational mode of the mobile device from the first mode to the second mode when the third set of transitional data approximates the third set of baseline data.

8. The system of claim 1, wherein:

the memory storage device is further operative to store transitional timing data representative of a duration of the mobile device changing from the first state to the second state; and

,

the processor is further operative to;

acquire timing data representative of the mobile device changing from the first orientation to the second orientation; and

,

change the operational mode of the mobile device from the first mode to the second mode when the timing data approximates the transitional timing data.

9. The system of claim 1, wherein:

the processor is further operative to output a result in response to the input received during the first operational mode; and

,

the at least one computer-generated graphic element comprises a portion of the result output by the processor.

10. The system of claim 1, wherein the selected portion of the real-world environment surrounding the mobile device is recorded in real-time.

11. The system of claim 1, wherein the selected portion of the real-world environment surrounding the mobile device was previously recorded and stored in the storage device.

12. A method for switching between the operational mode of a mobile device from a first operational mode to a second operational mode using a natural motion gesture comprising:

detecting, with an orientation sensor, a first orientation of the mobile device;

outputting, with the orientation sensor, a first set of orientation data representative of a mobile device being in a first orientation; and

,

detecting, with the orientation sensor, a second orientation of the mobile device;

outputting, with the orientation sensor, a second set of orientation data representative of the mobile device being in the second orientation; and

,

changing, with a processor, the operational mode of the mobile device from a first mode to a second mode when;

the first set of orientation data approximates a first set of baseline data representative of a first state of the mobile device; and

,

the second set of orientation data approximates a second set of baseline data representative of a second state of the mobile device, wherein;

the first operational mode comprises an input receiving mode operative to receive an input from a user; and

,

the second operational mode comprises a visual display mode operative to display a selected portion of the real-world environment surrounding the mobile device overlaid with at least one computer-generated graphic element.

13. The method of claim 12, wherein:

the first state of the mobile device comprises the mobile device being in the first operational mode; and

,

the input receiving mode comprises receiving a voice input from the user.

14. The method of claim 12, wherein the second state of the mobile device comprises the mobile device being in the second operational mode.

15. The method of claim 12, wherein the first set of orientation data indicates that a front surface of the mobile device is positioned approximately against the head of the user.

16. The method of claim 12, wherein the second set of orientation data indicates that a front surface of the mobile device is approximately perpendicular to a ground surface.

17. The method of claim 12, wherein the second set of orientation data indicates that a front surface of the mobile device forms an approximate acute angle to a ground surface.

18. The method of claim 12, further comprising:

detecting, with the orientation sensor, transitional orientations of the mobile device changing from the first orientation to the second orientation;

outputting, with the orientation sensor, a third set of transitional data representative of the transitional orientations; and

,

changing, with the processor, the operational mode of the mobile device from the first mode to the second mode when the third set of transitional data approximates a third set of baseline data representative of a transition of the mobile device from the first state to the second state.

19. The method of claim 12, further comprising:

acquiring, with the processor, timing data representative of the mobile device changing from the first orientation to the second orientation; and

,

changing, with the processor, the operational mode of the mobile device from the first mode to the second mode when the timing data approximates transitional timing data representative of a duration of the mobile device changing from the first state to the second state.

20. The method of claim 12, further comprising outputting, with the processor, a result in response to the input received during the first operational mode;

and, wherein;

the at least one computer-generated graphic element comprises a portion of the result output by the processor.

21. The method of claim 12, wherein the selected portion of the real-world environment surrounding the mobile device is recorded in real-time.

22. The method of claim 12, wherein the selected portion of the real-world environment surrounding the mobile device was previously recorded and stored in the storage device.