EVENT AUGMENTATION WITH REAL-TIME INFORMATION

• US 20120293548A1
• Filed: 05/20/2011
• Published: 11/22/2012
• Est. Priority Date: 05/20/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Perez, Kathryn Stone, Tardif, John A.
• Granted Patent: US 9,330,499 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling

Abstract

A system and method to present a user wearing a head mounted display with supplemental information when viewing a live event. A user wearing an at least partially see-through, head mounted display views the live event while simultaneously receiving information on objects, including people, within the user'"'"'s field of view, while wearing the head mounted display. The information is presented in a position in the head mounted display which does not interfere with the user'"'"'s enjoyment of the live event.

Claims

1. A computer implemented method providing supplemental information to a user with a head mounted display viewing a live event, comprising:

receiving an indication that a user is attending a live event, the live event having an event duration;

determining a field of view of the user through the head mounted display, and objects within the field of view at the live event;

retrieving supplemental information describing at least the objects in the field of view of the user during a portion of the live event;

determining elements of said supplemental information to present to the user in the head mounted display; and

displaying the supplemental information in one or more display elements in the head mounted display.

2. The method of claim 1 wherein the step of displaying the supplemental information in the head mounted display includes providing information regarding an object or action in the live event in the field of view of the user, the information and including a display location for the supplemental information in the head mounted display.

3. The method of claim 1 wherein the step of receiving an indication includes detecting a user'"'"'s location at the live event during the event duration.

4. The method of claim 1 wherein the step of receiving includes retrieving supplemental information regarding to scripted events likely to occur at scripted times in the live event.

5. The method of claim 1 wherein the step of receiving includes receiving supplemental information describing objects likely to be in the field of view of the user at a future portion of the event.

6. The method of claim 1 wherein the step of displaying the supplemental information includes transmitting supplemental information and a display position within a display field in the head mounted display.

7. The method of claim 1 wherein the method further includes the step of tracking a location of the user and an orientation of the user'"'"'s head.

8. The method of claim 1 further including repeating said retrieving, determining elements, and displaying steps for each movement of the user which changes the user'"'"'s field of view.

9. A method for displaying information supplementing a live event to a user wearing a head mounted display at the live event, comprising:

registering attendance at the live event with an information provider;

detecting a user field of view of a scene and a user position;

retrieving at least first supplemental information for the field of view of the user including supplemental information associated with objects in the user'"'"'s current field of view;

determining display elements of the first supplemental information to display in a display field of the head mounted display;

displaying supplemental information in the display field within the head mounted display when the user is viewing objects in the live event during the live event; and

repeating at least the detecting and displaying steps;

upon movement of the user to define a new field of view;

upon a change in an occurrence in the live event to update information about an object; and

upon a change in the objects in the user'"'"'s field of view.

10. A method in accordance with claim 9 wherein the displaying step includes determining a location within a user'"'"'s display field within the head mounted display which does not obscure the user'"'"'s view of the live event.

11. The method of claim 9 further including the step of retrieving additional supplemental information for the same field of view.

12. The method of claim 11 further including associating supplemental information with objects in the scene of a user.

13. The method of claim 12 wherein the supplemental information is scripted supplemental information regarding a scripted event.

14. The method of claim 12 wherein the supplemental information is unscripted supplemental information provided for a semi-random live event.

15. The method of claim 12 wherein the determining step includes providing additional data concerning known occurrences within the live event likely to occur in at a future time.

16. The method of claim 9 further including providing a user interface to the display to activate a trigger to display the supplemental information when an object associated with the information is in the user view.

17. The method of claim 9 wherein the step of determining is performed by a human director.

18. The method of claim 9 wherein the step of determining is performed by a processor based on rules associating information with actions and objects in the live event.

19. A computer implemented method of presenting information to a user viewing a live event with a head mounted display device, comprising:

for at least one live event having an event duration;

providing supplemental information relevant to objects within a user'"'"'s potential field of view;

receiving an indication from a user in attendance at the live event with access to view the objects;

detecting a live event start;

monitoring objects and actions in the live event for the event duration;

receiving information from the head mounted display device of a position, orientation and view of the user for the event duration;

determining the field of view of the user through the head mounted display during the event duration;

transmitting information regarding the objects and actions. in the field of view of the user to the display device;

repeating the receiving determining and transmitting steps;

upon movement of the user to define a new field of view;

upon a change in an occurrence in the event to update information about an object; and

upon a change in the objects in the user'"'"'s field of view.

20. The computer implemented method of claim 19 further including the step of selecting a sub-set of supplemental information associated with the objects in a display field.