System and method for spawning drawing surfaces

• US 9,898,865 B2
• Filed: 06/22/2015
• Issued: 02/20/2018
• Est. Priority Date: 06/22/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Thompson, Joe, Osborn, Dan, Hefny, Tarek, Latta, Stephen G., Gouin, Forest Woodcroft, Nakashima, James, Saunders, Megan, Gavriliuc, Anatolie, Cerriteno, Alberto E., Wright, Shawn Crispin
• Primary Examiner(s): Jansen, II, Michael J
• Application Number:
  US14/746,789
• Publication Number:
  US 20160371886A1
• Time in Patent Office: 974 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

A method for operating a computing device is described herein. The method includes determining a user'"'"'s gaze direction based on a gaze input, determining an intersection between the user'"'"'s gaze direction and an identified environmental surface in a 3-dimensional environment, and generating a drawing surface based on the intersection within a user interface on a display.

Claims

1. A method for operating a head mounted computing device, the method comprising:

determining a user'"'"'s gaze direction based on a gaze input;

determining an intersection between the user'"'"'s gaze direction and an identified environmental surface in a real-world 3-dimensional environment;

generating a first drawing surface based on the intersection within a user interface on a see-through holographic display configured to present holographic images, wherein the first drawing surface is arranged at a predetermined angle relative to the user'"'"'s gaze direction, wherein the predetermined angle is determined at least in part based on a characteristic of the 3-dimensional environment;

receiving a first drawing surface input that includes a path drawn by a user; and

in response to receiving the first drawing surface input, rendering a first drawing indicia on the first drawing surface within the user interface on the display that includes one or more lines that denote the path drawn by the user based on the first drawing surface input.

2. The method of claim 1, where the drawing surface is generated within a predetermined distance from the intersection.

3. The method of claim 1, where the computing device is a portable computing device comprising a camera configured to capture the 3-dimensional environment of the real-world external to the portable computing device and a display configured to present the 3-dimensional environment.

4. The method of claim 1, further comprising receiving a drawing activation input and where the user'"'"'s gaze direction is determined in response to receiving the drawing activation input.

5. The method of claim 1, further comprising:

determining a change in the user'"'"'s gaze direction;

receiving a second drawing surface input; and

if change in the user'"'"'s gaze direction does not exceed a predetermined value, rendering a second drawing indicia on the first drawing surface based on the second drawing surface input.

6. The method of claim 5, further comprising:

if the change in the user'"'"'s gaze direction exceeds the predetermined value, generating a second drawing surface based on an intersection between the second gaze direction and a second surface within the user interface; and

rendering the second drawing indicia on the second drawing surface based on the second drawing surface input.

7. The method of claim 5, where the step of rendering the second drawing indicia is only implemented when the second drawing surface input is received within a predetermined time interval after the first drawing surface input is received.

8. The method of claim 1, where at least a portion of the first drawing surface is highlighted.

9. The method of claim 1, where the first drawing surface is planar.

10. The method of claim 1, further comprising adjusting a geometry of the first drawing surface based on a geometric characteristic of the identified environmental surface.

11. The method of claim 1, further comprising removing the first drawing surface from the user interface after a predetermined time interval has been surpassed subsequent to generation of the first drawing surface.

12. The method of claim 1, further comprising removing the drawing surface from the display in response to a change in the user'"'"'s gaze direction exceeding a predetermined value.

13. The method of claim 1, wherein the predetermined angle is limited to a range that includes 60 degrees to 90 degrees relative to the gaze direction.

14. The method of claim 1, where the user'"'"'s gaze direction is visually displayed as a line having a predetermined thickness.

15. A head-mounted computing device comprising:

a see-through holographic display configured to present a user interface and holographic images;

instructions stored in memory executable by a processor to;

determine a user'"'"'s gaze direction based on a gaze input;

identify an environmental surface in a real-world 3-dimensional environment;

determine an intersection between the user'"'"'s gaze direction and the environmental surface;

generate a first drawing surface based on the intersection within the user interface on the display, wherein the drawing surface is arranged at a predetermined angle relative to the user'"'"'s gaze direction, wherein the predetermined angle is determined at least in part based on a characteristic of the 3-dimensional environment; and

render a first drawing indicia in the drawing surface responsive to receiving a first drawing surface input that includes a path drawn by a user and based on the first drawing surface input, wherein the first drawing indicia is rendered within the user interface on the display and includes one or more lines that denote the path drawn by the user.

16. The computing device of claim 15, where the first drawing surface is planar.

17. A method for operating a head mounted computing device, the method comprising:

receiving a drawing activation input;

generating a user'"'"'s gaze direction in response to receiving the drawing activation input and based on a gaze input;

determining an intersection between the user'"'"'s gaze direction and an identified environmental surface in a 3-dimensional environment;

generating a drawing surface based on the intersection within a user interface on a display, wherein the first drawing surface is arranged at a predetermined angle relative to the user'"'"'s gaze direction, wherein the predetermined angle is determined at least in part based on characteristics of the 3-dimensional environment;

receiving a drawing surface input that includes a path drawn by a user; and

in response to receiving the drawing surface input, rendering a drawing indicia on the drawing surface within the user interface on the display that includes one or more lines that denote the path drawn by the user based on the drawing surface input.

18. The method of claim 17, where the drawing activation input and the drawing surface input are jointly implemented as a continuous input.