COLLABORATION ENVIRONMENT USING SEE THROUGH DISPLAYS

• US 20130293468A1
• Filed: 05/04/2012
• Published: 11/07/2013
• Est. Priority Date: 05/04/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Perez, Kathryn Stone, Clavin, John, Geisner, Kevin A., Latta, Stephen G., Mount, Brian J., Tomlin, Arthur C., Poulos, Adam G.
• Granted Patent: US 9,122,321 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/2141 Access rights, e.g. capability lists, access control lists, access tables, access matrices
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment

Abstract

A see-through, near-eye, mixed reality display device and system for collaboration amongst various users of other such devices and personal audio/visual devices of more limited capabilities. One or more wearers of a see through head mounted display apparatus define a collaboration environment. For the collaboration environment, a selection of collaboration data and the scope of the environment are determined. Virtual representations of the collaboration data in the field of view of the wearer, and other device users are rendered. Persons in the wearer'"'"'s field of view to be included in collaboration environment and who are entitled to share information in the collaboration environment are defined by the wearer. If allowed, input from other users in the collaboration environment on the virtual object may be received and allowed to manipulate a change in the virtual object.

Claims

1. A method for presenting a collaboration experience using a see through head mounted display apparatus, comprising:

determining a three dimensional location of the apparatus, the apparatus includes one or more sensors and a see-through display;

determining an orientation of the apparatus;

determining a gaze of a wearer looking through the see-through display of the apparatus;

determining a three dimensional location of at one or more users in the field of view of the user through the see-through display, the determining of the three dimensional location of the movable object is performed using the one or more sensors;

receiving a selection of collaboration data and a selection of a collaboration environment within the field of view from the wearer;

rendering virtual representations of the collaboration data in the field of view;

determining persons in the wearer'"'"'s field of view to be included in collaboration environment and who are entitled to share information in the collaboration environment;

outputting shared collaboration data in the form of virtual objects to users in the collaboration environment having audio/visual devices in the collaboration environment;

receiving input from users in the collaboration environment on the virtual object, the input manipulating a change in the virtual object; and

displaying the change in the virtual object to the wearer.

2. The method of claim 1 further including:

determining the location of real world objects in the field of view of a user;

rendering the virtual objects in relation to the real world objects in the collaboration environment.

3. The method of claim 1 further including defining a first set of users and a second set of users in the collaboration environment, the first set of users receiving a version of the collaboration objects and the second set of users receiving a more limited version of the collaboration objects.

4. The method of claim 1 wherein receiving a selection includes presenting the user with a set of virtual objects for the collaboration environment and receiving a selection of the virtual objects.

5. The method of claim 1 further including presenting a set of object controls in the field of view of the wearer, the object controls including at least collaboration permission controls for users in the collaboration environment.

6. The method of claim 1 further including displaying collaboration permissions for each person in the field of view of the wearer.

7. The method of claim 1 further including determining an associated audio/visual device for each user in the collaboration environment, including determining the audio/visual capabilities of the audio visual device associated with each user.

8. The method of claim 7 further including displaying the audio/visual capabilities of each device associated with each user and wherein the step of outputting includes outputting objects suitable for the audio visual capability of each device.

9. The method of claim 8 further including translating changes for a virtual object in received input from users to a format suitable for the audio/visual capability of each device.

10. A method for presenting a collaboration experience, comprising:

creating a collaboration environment defined by at least one wearer of a see through head mounted display apparatus by

receiving from the wearer a selection of collaboration data and a selection of a collaboration environment within the field of view and a definition of which users in the collaboration environment are entitled to share information in the collaboration environment;

determining three dimensional locations of users in the collaboration environment including, for users having see through head mounted display apparatuses;

automatically determining an orientation of the apparatus;

automatically determining a gaze of a user looking through the see-through display of the apparatus;

receiving from users in the collaboration environment shared virtual objects

outputting the shared virtual objects to users having audio/visual devices in the collaboration environment;

receiving input from users in the shared environment on the virtual object, the input manipulating a change in the virtual object; and

outputting the change in the virtual object in the apparatus.

11. The method of claim 10 further including:

determining an associated audio/visual device for each user in the collaboration environment, including determining the audio/visual capabilities of the audio visual device associated with each user;

translating changes for a virtual object in received input from users to a format suitable for the audio/visual capability of each device.

12. The method of claim 11 wherein translating includes providing a representation of a virtual object manipulated in a see through head mounted display to a two dimensional display of an audio/visual device.

13. The method of claim 11 further including authenticating users and associated audio/visual devices in the collaboration environment by a combination of biometric factors.

14. The method of claim 11 further including managing permissions for shared experience by allowing or blocking applications or use of particular functions of the audio/visual device.

15. An see through head mounted display apparatus presenting a collaboration environment to a wearer'"'"'s field of view, comprising:

a see-through, near-eye, augmented reality display that is worn by a wearer;

one or more processing devices in communication with apparatus, the one or more processing devices automatically determine that the wearer is at a location, the one or more processing devices determine real objects and other persons in the field of view of the wearer, the one or more processing devices determine a collaboration environment in the field of view of the wearer, the environment including users authorized to see collaboration information in the form of virtual objects in the environment, the one or more processing devices rendering virtual objects and user actions in the collaboration environment including managing information provided to users in the collaboration environment;

the collaboration environment including a plurality of audio/visual devices capable of rendering information, including see through head mounted display devices and other devices.

16. The apparatus of claim 15 wherein the one or more processing devices render an auditory or visual representation of collaboration permissions assigned to each users relative to the collaboration information.

17. The apparatus of claim 16 wherein the one or more processing devices render a an auditory or visual representation of user identity and associated processing devices and device capability.

18. The apparatus of claim 17 wherein the one or more processing devices render a an auditory or visual representation of user sharing links between users and user devices.

19. The apparatus of claim 18 wherein the one or more processing devices render a an auditory or visual representation of a set of virtual objects for the collaboration environment and receiving a selection of the virtual objects.

20. The apparatus of claim 19 wherein the one or more processing devices render a set of object controls in the field of view of the wearer, the object controls including at least collaboration permission controls for users in the collaboration environment.