Virtual/augmented reality transition system and method

• US 10,254,826 B2
• Filed: 05/25/2017
• Issued: 04/09/2019
• Est. Priority Date: 04/27/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Clement, Manuel Christian, Faaborg, Alexander James
• Primary Examiner(s): Eurice, Michael J
• Application Number:
  US15/605,582
• Publication Number:
  US 20170262046A1
• Time in Patent Office: 684 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/012 Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system and method of operating an audio visual system generating a virtual immersive experience may include an electronic user device in communication with a tracking device that may track a user'"'"'s physical movement in a real world space and translate the tracked physical movement into corresponding movement in the virtual world generated by the user device. The system may detect when a user and the user device are approaching a boundary of a tracking area and automatically initiate a transition out of the virtual world and into the real world. A smooth, or graceful, transition between the virtual world and the real world as the user encounters this boundary may avoid disorientation which may occur as a user continues to move in the real world, while motion appears to have stopped upon reaching the tracking boundary.

Claims

1. A method, comprising:

tracking, by a tracking device, a position of an electronic device in an ambient environment, the electronic device providing a virtual environment for a user of the electronic device;

performing a transition process to transition between the virtual environment provided by the electronic device and the ambient environment in response to one of a plurality of previously defined transition triggers; and

outputting, from the electronic device, an external indicator indicating the performing of the transition process,

the plurality of previously defined transition triggers including;

detecting at least one of an external condition or an external input corresponding to one of the plurality of previously defined transition triggers.

2. The method of claim 1, wherein the outputting the external indicator includes displaying an external visual indicator that is visible external to the electronic device.

3. The method of claim 1, wherein the outputting the external indicator includes outputting an external audible indicator that is audible external to the electronic device.

4. The method of claim 1, wherein the outputting the external indicator includes generating an external physical indicator that is detectable external to the electronic device.

5. The method of claim 1, wherein an area of the ambient environment is less than a tracking area of the tracking device, such that the boundary corresponds to a physical boundary of the ambient environment, the plurality of previously defined transition triggers including:

comparing a distance between the electronic device and the physical boundary of the ambient environment; and

initiating the transition process when the distance between the electronic device and the physical boundary of the ambient environment is less than or equal to the previously defined threshold distance.

6. The method of claim 1, wherein the performing the transition process includes transitioning from the virtual environment to the ambient environment, including at least one of:

fading out virtual image content of the virtual environment displayed on a display of the electronic device and fading in transition image content displayed on the display of the electronic device;

or

fading out of audio content of the virtual environment output by the electronic device and fading in of transition audio content by the electronic device.

7. The method of claim 6, wherein the fading out virtual image content and fading in transition image content includes at least one of:

replacing virtual video content of the virtual environment displayed on the display with a monochromatic display;

replacing the virtual video content of the virtual environment displayed on the display with pass through image content of the ambient environment captured by an imaging device of the electronic device;

or

transitioning the display from a non-transparent state to a transparent state such that the ambient environment is visible through the display upon completion of the fade out of the virtual image content displayed on the display of the electronic device.

8. The method of claim 1, wherein the detecting the external condition includes:

detecting a physical obstacle in a movement path of the electronic device in the ambient environment, within a tracking area of the tracking device;

comparing a distance between the electronic device and the detected physical obstacle; and

initiating the transition process when the distance between the electronic device and the detected obstacle is less than or equal to a previously defined threshold distance.

9. The method of claim 8, wherein the detecting the physical obstacle includes:

intermittently scanning the tracking area and intermittently capturing an image of the tracking area;

comparing a current image of the tracking area to at least one previous image of the tracking area; and

detecting the physical obstacle in the movement path of the electronic device based on at least one change detected when comparing the current image of the tracking area to the at least one previous image of the tracking area.

10. The method of claim 1, wherein the detecting the external input includes at least one of:

detecting a physical gesture implemented by an external party,

detecting an audio command generated by an external party,

detecting an approach of an external party, or

detecting an eye gaze of an external party; and

wherein the outputting the external indicator includes outputting at least one of a visual indicator, an audible indicator, or a physical indicator directed to the external party indicating completion of the transition process.

11. The method of claim 10, the detecting the audio command including at least one of detecting a voice command including at least one previously set keyword, or detecting a previously defined audio tone.

12. The method of claim 1, the detecting the external condition includes detecting a distance between the electronic device and a boundary of the tracking device that is less than or equal to a previously defined threshold distance, including:

comparing a current position of the electronic device to at least one previous position of the electronic device; and

determining at least one of a direction of movement of the electronic device, a speed of movement of the electronic device, or an acceleration of movement of the electronic device relative to at least one boundary of a tracking area tracked by the tracking device based on the comparison of the current position to the at least one previous position.

13. The method of claim 12, wherein the performing the transition process includes:

selecting a transition profile based on at least one of the determined direction, speed or acceleration of movement of the electronic device, a brightness level or a sound level associated with the virtual environment provided by the electronic device, or a previously defined user preference.

14. The method of claim 12, wherein the performing the transition process includes performing the transition process such that the transition between the virtual environment and the ambient environment is complete at or before the electronic device reaches the at least one boundary of the tracking area.

15. The method of claim 14, wherein the performing the transition process includes transitioning from the virtual environment to the ambient environment at or before the electronic device crosses the at least one boundary and is outside of the tracking area of the tracking device, the transition process also including:

detecting that the electronic device has re-entered the tracking area; and

transitioning back into the virtual environment in response to detecting the electronic device in the tracking area.

16. A system, comprising:

an electronic device displaying a virtual environment, the electronic device operating in and being movable within an ambient environment;

a tracking device tracking a position and movement of the electronic device in the ambient environment;

a processor; and

a computer program product embodied on a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by the processor, cause the processor to;

perform a transition process to transition between the virtual environment displayed by the electronic device and the ambient environment in response to detection of one of a plurality of previously defined transition triggers; and

output an external indicator indicating the performing of the transition process,

the plurality of previously defined transition triggers including detection of at least one of an external condition or an external input corresponding to one of the plurality of previously defined transition triggers.

17. The system of claim 16, wherein execution of the instructions also causes the processor to:

perform a first transition from the virtual environment displayed by the user electronic device to the ambient environment in response to detection of a first transition trigger, such that the transition out of the virtual environment is complete at or before the electronic device reaches the boundary; and

perform a second transition from the ambient environment into the virtual environment in response to detection of a second transition trigger detecting that the electronic device has crossed the boundary and has re-entered the tracking area.

18. The system of claim 16, the external input including detection of at least one of a physical gesture implemented by an external party, an audio command generated by an external party, an approach of an external party, or an eye gaze of an external party.

19. The system of claim 16, wherein execution of the instructions also causes the processor to:

select a transition profile, from a plurality of transition profiles, based on the detected transition trigger, each of the plurality of transition profiles including transition parameters for transitioning between the virtual environment and the ambient environment; and

perform the transition process to transition from the virtual environment to the ambient environment in accordance with transition parameters included in the selected transition profile.

20. The system of claim 19, wherein the plurality of transition profiles includes:

a first transition profile, including replacing virtual image content displayed on a display of the electronic device in the virtual environment with transition image content displayed on the display, the transition image content including at least one of a monochromatic display or a pass through image of the ambient environment;

a second transition profile, including replacing virtual audio content output by the electronic device in the virtual environment with transition audio content, the transition audio content including at least one of a previously defined signal or tone, or a pass through of ambient sound from the ambient environment; and

a third transition profile defining the output of the external indicator, including outputting at least one of an externally directed visual indicator, an externally directed audible indicator, or an externally directed physical indicator.

21. A method, comprising:

tracking, by a tracking device, a position of an electronic device in an ambient environment, the electronic device providing a virtual environment for a user of the electronic device;

performing a transition process to transition between the virtual environment provided by the electronic device and the ambient environment in response to one of a plurality of previously defined transition triggers; and

generating an external indicator indicating the transition process, including at least one of displaying an external visual indicator that is visible external to the electronic device or outputting an external audible indicator that is audible external to the electronic device,

the plurality of previously defined transition triggers including;

detecting a distance between the electronic device and a boundary of the tracking device that is less than or equal to a previously defined threshold distance; and

detecting an external input corresponding to one of the plurality of previously defined transition triggers.