PRODUCT AUGMENTATION AND ADVERTISING IN SEE THROUGH DISPLAYS

• US 20130293530A1
• Filed: 05/04/2012
• Published: 11/07/2013
• Est. Priority Date: 05/04/2012
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Perez, Kathryn Stone, Clavin, John, Geisner, Kevin A., Latta, Stephen G., Mount, Brian J., Tomlin, Arthur C., Poulos, Adam G.
• Application Number:
  US13/464,944
• Publication Number:
  US 20130293530A1
• Time in Patent Office: Days
• US Class Current: 345/418
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

An augmented reality system that provides augmented product and environment information to a wearer of a see through head mounted display. The augmentation information may include advertising, inventory, pricing and other information about products a wearer may be interested in. Interest is determined from wearer actions and a wearer profile. The information may be used to incentivize purchases of real world products by a wearer, or allow the wearer to make better purchasing decisions. The augmentation information may enhance a wearer'"'"'s shopping experience by allowing the wearer easy access to important product information while the wearer is shopping in a retail establishment. Through virtual rendering, a wearer may be provided with feedback on how an item would appear in a wearer environment, such as the wearer'"'"'s home.

Claims

1. A method providing augmentation information to a wearer for a product in the field of view of a wearer, comprising:

receiving input data from a wearer of a see through head mounted display device;

determining a gaze direction in a field of view of the wearer from the input data;

determining a location of the wearer;

retrieving personal information of the wearer;

identifying real world objects in the field of view of a wearer in the see through head mounted display device;

retrieving augmentation data for the real world objects and matching objects in the field of view of the wearer to the augmentation data provided by a third party data source;

presenting the augmentation information to a wearer associated with the identified products in the field of view.

2. The method of claim 1 wherein the augmentation information is advertising presented to the wearer as visual information in the field of view or as audible information.

3. The method of claim 1 wherein the augmentation information is targeted to the wearer based on the personal information of the wearer.

4. The method of claim 1 wherein the augmentation information is rendered to a wearer when the wearer is gazing at the matched products.

5. The method of claim 1 wherein the augmentation information is based on the wearer'"'"'s location relative to the matched product and the information is displayed when the wearer is gazing at a matched product.

6. The method of claim 1 further including the step of monitoring wearer gaze at the product or augmentation information to infer wearer interest in the product or augmentation based on time spent by the wearer gazing at the information.

7. The method of claim 6 wherein the method further includes updating the augmentation information based on the attention of the wearer determined by the monitoring step.

8. A method of augmenting a view of a wearer in a see through head mounted display to provide information regarding a product to the field of view of a wearer, comprising:

determining a location a wearer;

retrieving personal information of the wearer;

retrieving virtual object models of objects in the wearer inventory;

rendering in the see through head mounted display a portion of the wearer environment model and a virtual object based on the object model which was selected by a wearer from objects presented in the field of view of the wearer;

matching objects in the field of view of the wearer to augmentation data provided by a third party data source;

presenting augmentation information to a wearer associated with the object, the augmentation information targeted to the wearer based on the personal information retrieved on the wearer.

9. The method of claim 8 wherein the step of retrieving virtual objects includes determining a real world object within the gaze of a wearer in the see through head mounted display, retrieving the virtual object matching the real world object, and

rendering the virtual object matching the real world object in the virtual environment of the wearer.

10. The method of claim 8 further including the step of determining whether the location of the wearer is proximate to real world objects;

determining a real world object within the gaze of a wearer and

providing augmentation information for the real world object.

11. The method of claim 10 wherein the augmentation information comprises advertising relating to the product or similar products and is presented in the field of view of the wearer.

12. The method of claim 10 wherein the advertising is an interactive presentation in the field of view of the wearer.

13. The method of claim 10 wherein the augmentation information comprises inventory and pricing information for a real world object or a virtual object within the gaze of the wearer.

14. An see through head mounted display apparatus presenting augmentation information to a wearer'"'"'s field of view, comprising:

a see through, near-eye, augmented reality display that is worn by a wearer;

one or more processing devices in communication with apparatus, the one or more processing devices automatically determine that the wearer is at a location, the one or more processing devices access a wearer profile for the wearer, the one or more processing devices determine real world objects in the field of view of the wearer and a real world object within the gaze of a wearer, to present augmentation information regarding the real world objects to the wearer for the object;

the augmentation information including third party information comprising one of advertising, inventory, alternative on-line sellers, alternative local sellers, pricing information or product reviews presented in the field of view of the wearer by the augmented reality display.

15. The apparatus of claim 14 wherein the one or more processing devices present the augmentation information based on targeting information specific to the wearer based on the personal information retrieved on the wearer.

16. The apparatus of claim 14 wherein the apparatus includes a rule set comprising regulating at least one rule blocking augmentation information presented to the wearer when such presentation is dangerous.

17. The apparatus of claim 16 wherein advertising is presented for a wearer location relative to a retail establishment to advertising for a location where the wearer is present.

18. The apparatus of claim 14 wherein the augmentation information is presented by determining a real world object within the field of view of a wearer is on a list of a wearer in the wearer profile, retrieving augmentation information regarding the real world object;

and presenting the augmentation in association with the real world object when the wearer gaze is directed at the object.

19. The apparatus of claim 14 wherein the augmentation information is presented by retrieving augmentation information regarding a real world object proximate to the wearer which matches an item on a wearer list and presenting augmentation to encourage the wearer to purchase the real world object.

20. The apparatus of claim 14 wherein the augmentation information is presented by retrieving augmentation information regarding a real world object proximate to the wearer which matches an item on a wearer list and directing the wearer to the location of a real world object.