PROVIDING ELECTRONIC COMMUNICATIONS IN A PHYSICAL WORLD

• US 20130016033A1
• Filed: 07/12/2011
• Published: 01/17/2013
• Est. Priority Date: 07/12/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Latta, Stephen G., Small, Sheridan Martin, Vaught, Benjamin I., Bennett, Darren, Liu, James C.
• Granted Patent: US 9,146,398 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/147 using display panels

Abstract

Techniques are provided for displaying electronic communications using a head mounted display (HMD). Each electronic communication may be displayed to represent a physical object that indentifies it as a specific type or nature of electronic communication. Therefore, the user is able to process the electronic communications more efficiently. In some aspects, computer vision allows a user to interact with the representation of the physical objects. One embodiment includes accessing electronic communications, and determining physical objects that are representative of at least a subset of the electronic communications. A head mounted display (HMD) is instructed how to display a representation of the physical objects in this embodiment.

Claims

1. A method comprising:

accessing electronic communications;

determining a plurality of physical objects that are representative of at least a subset of the electronic communications; and

instructing a head mounted display (HMD) to display a representation of the physical objects.

2. The method of claim 1, further comprising:

tracking actions of a user wearing the HMD;

determining how the user is intending to interact with a first of the physical objects that corresponds to a first of the electronic communications based on the actions; and

determining how to alter presentation of the first physical object on the HMD based on the actions in order accomplish the user'"'"'s intent to interact.

3. The method of claim 1, wherein the determining a plurality of physical objects includes determining a first of the physical objects that reflects content within a first of the electronic communications.

4. The method of claim 1, wherein the determining a plurality of physical objects includes determining a first of the physical objects based on a source of a first of the electronic communications.

5. The method of claim 1, further comprising:

receiving a selection of a first of the physical objects; and

providing content of a first of the electronic communications that corresponds to the first physical object to the HMD in response to the selection.

6. The method of claim 1, further comprising:

determining an arrangement of the physical objects in response to a command to sort the electronic communications; and

instructing the HMD to display the arrangement of the physical objects.

7. The method of claim 1, further comprising:

determining a location of a user; and

determining which of the electronic communications to present physical objects for in the HMD based on the location of the user.

8. The method of claim 1, further comprising:

displaying the plurality of physical objects on the HMD such that the physical objects appear to be on or near an actual physical surface.

9. The method of claim 1, wherein the determining a plurality of physical objects that are representative of at least subset of the electronic communications includes:

determining a set of the electronic communications that are to be represented in the HMD; and

determining one of the physical objects for each of the electronic communications in the set.

10. A system comprising:

a processor; and

computer storage media having instructions stored thereon which, when executed on the processor, cause the processor to perform the following;

accessing electronic communications;

determining a plurality of physical objects that are representative of the electronic communications, each of the physical objects identifies the corresponding electronic communication as a specific type of communication; and

providing an image to a head mounted display (HMD), the image includes a representation of the plurality of physical objects.

11. The system of claim 10, wherein the system further includes an image capture device, and wherein the instructions further cause the processor to perform:

tracking actions of a user wearing the HMD based on a depth image from the image capture device;

determining how a user is intending to interact with a first of the physical objects that corresponds to a first of the electronic communications based on the actions; and

determining how to alter presentation of the first physical object on the HMD based on the actions in order accomplish the user'"'"'s intent to interact.

12. The system of claim 10, wherein the instructions further cause the processor to perform:

determining how to re-arrange the plurality of physical objects in response to a command to sort or index the plurality of physical objects; and

proving an updated image to the HMD to affect the re-arrangement of the plurality of physical objects.

13. The system of claim 10, further comprising the HMD which displays the plurality of physical objects.

14. Computer storage media having instructions stored thereon which, when executed on a processor, cause the processor to perform the following:

accessing electronic communications having content;

generating one or more images for representing the electronic communications as physical objects that are representative of the electronic communications;

providing the one or more images to a head mounted display (HMD);

receiving a selection of a first of the physical objects; and

providing content of a first of the electronic communications that corresponds to the first physical object to the HMD in response to the selection.

15. The computer storage media of claim 14, wherein the instructions further cause the processor to perform:

tracking actions of a user wearing the HMD;

determining how the user is intending to interact with a second of the physical objects for a second of the electronic communications based on the actions;

updating the one or more images based on the actions; and

providing the updated one or more images to the head mounted display (HMD).

16. The computer storage media of claim 14, wherein the instructions further cause the processor to perform:

receiving a request to sort the electronic communications;

determining an arrangement for the physical objects to be displayed in the HMD in response to the request;

generating one or more additional images that contain the arrangement of the physical objects; and

providing the one or more additional images to the HMD.

17. The computer storage media of claim 14, wherein the instructions that cause the processor to generate one or more images for representing the electronic communications as physical objects that are representative of the electronic communications further cause the processor to perform:

determining how to render the first physical object such that outward appearance of the first physical object indicates content within the first electronic communication.

18. The computer storage media of claim 14, wherein the instructions that cause the processor to generate one or more images for representing the electronic communications as physical objects that are representative of the electronic communications further cause the processor to perform:

determining how to render the first physical object in order to indicate a source of the first electronic communication.

19. The computer storage media of claim 14, wherein the instructions further cause the processor to perform:

determining a local surface on which to present the physical objects.

20. The computer storage media of claim 14, wherein the instructions further cause the processor to perform:

determining a location of a user; and

determining which of the electronic communications to present physical objects for in the HMD based on the location of the user.