Holographic Display System with Undo Functionality

• US 20170052595A1
• Filed: 08/21/2015
• Published: 02/23/2017
• Est. Priority Date: 08/21/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lynn, Johanna Dy, Poulos, Adam Gabriel, Scavezze, Michael, McCulloch, Daniel Joseph
• Granted Patent: US 10,007,352 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 27/0103 comprising holographic elements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

Disclosed are techniques for performing undo operations on holographic objects in an immersive 3D visual environment. A display system allows the user to undo a given user operation performed on a particular selected holographic object without affecting any other holographic objects, based on a user'"'"'s gaze and/or other user input. The technique can be implemented in conjunction with a scrollable visual “timeline” in which multiple past states of the display environment are displayed to the user and are selectable by the user as the target state of the revert operation. Also disclosed is a technique for partially undoing a single continuous user action in a holographic display system.

Claims

1. A method comprising:

displaying, by a display system to a user, an immersive three-dimensional (3D) visual environment that includes a plurality of virtual 3D objects;

detecting a gaze direction of the user;

identifying a particular virtual 3D object of the plurality of virtual 3D objects as an intended target object of a revert operation for reverting object state, based on the gaze direction of the user; and

in response to a first user input, reverting a state of the intended target object to a past state without altering a state of any other object of the plurality of virtual 3D objects.

2. The method of claim 1, wherein the immersive 3D visual environment is an augmented reality environment in which the plurality of virtual 3D objects are superimposed over a real view of a physical environment of the user.

3. The method of claim 1, further comprising:

prior to detecting the first user input, modifying a state of at least one of the plurality of virtual 3D objects according to a single user action that continuously spans a time period from a first time point to a second time point, the time period including at least one additional time point ascertainable by the display system between the first time point and the second time point; and

in response to a second user input, partially reversing the single user action to revert a state of said at least one of the plurality of virtual 3D objects to correspond to a time point between the first time point and the second time point.

4. The method of claim 1, further comprising, prior to reverting the state of the intended target object to the past state, displaying, by the display system, a user interface feature for performing the revert operation.

5. The method of claim 4, wherein displaying the user interface feature comprises:

displaying a plurality of summary images indicative of a plurality of past states of the particular virtual 3D object, in an arrangement indicative of a chronological sequence of the plurality of past states; and

enabling the user to select any one of the summary images to trigger said revert operation;

wherein the first input includes a selection by the user of one of the summary images.

6. The method of claim 5, wherein displaying the user interface feature comprises displaying a virtual joystick operable by the user to scroll chronologically forward or backward among the summary images.

7. The method of claim 5, wherein displaying the user interface feature comprises displaying a substantially circular chronology indicator, and displaying the plurality of summary images around the substantially circular chronology indicator.

8. The method of claim 5, wherein displaying the user interface feature comprises displaying a guide indicator that visually associates a particular one of the summary images with a displayed 3D virtual 3D object that was modified at a time to which said particular one of the summary images corresponds.

9. The method of claim 1, wherein the first user input comprises at least one of a gesture or speech of the user.

10. A head-mounted display (HMD) device for augmented reality visualization, comprising:

a head fitting to enable the head-mounted display device to be worn on the head of a user;

a display device coupled to the head fitting and disposed to be positioned in front of the eyes of the user when the HMD device is worn by the user, the display device being at least partially transparent;

a plurality of input devices, including an eye tracking camera; and

a processor coupled to the plurality of input devices and the display device and configured to

cause the display device to display to the user an augmented reality environment that includes a plurality of virtual three-dimensional (3D) objects overlaid on a real-world view of a 3D physical space in which the user is located,

modify states of the plurality of virtual 3D objects in response to user inputs from the user;

display a user interface feature for performing a revert operation for reverting an object state to a past state;

detect a gaze direction of the user by use of the eye tracking camera;

detect a first user input directed to the user interface feature;

identify a particular virtual 3D object of the plurality of virtual 3D objects as an intended target object of the revert operation based on the gaze direction of the user; and

in response to the first user input, revert a state of the intended target object without altering a state of any other object of the plurality of virtual 3D objects.

11. The HMD device of claim 10, wherein the processor is further configured to:

prior to detecting the first user input, modify a state of at least one of the plurality of virtual 3D objects according to a single user action that continuously spans a time period from a first time point to a second time point, the time period including at least one additional time point ascertainable by the display system between the first time point and the second time point; and

in response to a second user input, partially reverse the single user action to revert a state of said at least one of the plurality of virtual 3D objects to correspond to a time point between the first and second time points.

12. The HMD device of claim 10, wherein displaying the user interface feature comprises:

displaying a plurality of summary images indicative of a plurality of past states of the particular virtual 3D object, in an arrangement indicative of a chronological sequence of the plurality of past states; and

enabling the user to select any one of the summary images to trigger said revert operation;

wherein the first input includes a selection by the user of one of the summary images.

13. A method comprising:

displaying, by a display system to a user, an immersive three-dimensional (3D) visual environment that includes a plurality of virtual 3D objects;

modifying, by the display system, a state of at least one of the plurality of virtual 3D objects according to a single user action that continuously spans a time period from a first time point to a second time point, the time period including at least one additional time point ascertainable by the display system between the first and second time points;

identifying, by the display system, a particular virtual 3D object of the plurality of virtual 3D objects as an intended target object of a revert operation for reverting object state; and

in response to a first user input, partially reversing the single user action to revert the state of the intended target object to correspond to a time point between the first and second time points.

14. The method of claim 13, wherein reverting the state of the intended target object to correspond to a time point between the first and second time points is done without altering a state of any other object of the plurality of virtual 3D objects.

15. The method of claim 13, further comprising:

detecting a gaze direction of the user, wherein identifying the particular virtual 3D object of the plurality of virtual 3D objects as the intended target object of the revert operation is based on the gaze direction of the user.

16. The method of claim 13, wherein the immersive 3D visual environment is an augmented reality environment in which the plurality of virtual 3D objects are superimposed over a real view of a physical environment of the user.

17. The method of claim 13, further comprising, prior to reverting the state of the intended target object to the past state, displaying, by the display system, a user interface feature for performing the revert operation, wherein displaying the user interface feature comprises:

displaying a plurality of summary images indicative of a plurality of past states of the particular virtual 3D object, in an arrangement indicative of a chronological sequence of the plurality of past states; and

enabling the user to select any one of the summary images to trigger said revert operation.

18. The method of claim 17, wherein displaying the user interface feature comprises displaying a virtual joystick operable by the user to scroll chronologically forward or backward among the summary images.

19. The method of claim 17, wherein displaying the user interface feature comprises displaying a substantially circular chronology indicator, and displaying the plurality of summary images around the substantially circular chronology indicator.

20. The method of claim 13, wherein the first user input comprises at least one of a gesture or speech of the user.