Personal augmented reality

• US 9,240,059 B2
• Filed: 12/29/2011
• Issued: 01/19/2016
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Zises, Matthew Scott
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US13/340,141
• Publication Number:
  US 20130170697A1
• Time in Patent Office: 1,482 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A method, article of manufacture and system for receiving, from a user device located at an environment at which a user is viewing an event both with and without the use of the user device, image data containing an image of the environment; receiving a request from the user device for an item of information that relates to the event, for placement into the image of the environment; responsive to receiving the request, accessing a database to retrieve the item of information; generating a scaled image of the item of information based on dimensions of the environment; and transmitting the scaled image for placement into the image of the environment on the user device to generate an augmented reality image.

Claims

1. A method comprising:

receiving, from a mobile video device located at an environment, the mobile video device operated by a user who is viewing locations of the environment, the viewing both with and without the use of the mobile video device, image data containing an image of a marker in the environment that has not been placed in the environment for electronic navigation purposes;

receiving a request from the mobile video device for an item of information that provides specific location information that locates a place in the environment, the request generated in accordance with the locations of the environment that the mobile video device is focused on as the user moves in the environment and focuses the mobile video device on locations of the environment, the item of information for placement into an image of the environment on the mobile video device;

responsive to receiving the request, accessing a database to retrieve the item of information;

generating a scaled image of the item of information based on dimensions of the environment and the location of the place in the environment; and

transmitting the scaled image of the item of information that provides specific location information that locates the place in the environment, for placement into the image of the environment on the mobile video device to generate an augmented reality image, the scaled image of the item of information changing in accordance with user movement and the locations of the environment that the mobile video device is focused on.

2. The method of claim 1, further comprising determining the location where the scaled image is to be placed into the image of the environment.

3. The method of claim 1, further comprising determining the location of placement of the scaled image into the image of the environment.

4. The method of claim 1, further comprising determining a location in the environment using the marker located in the environment.

5. The method of claim 1, further comprising:

determining an orientation of the placement of the scaled image; and

orienting the scaled image to the determined orientation.

6. The method of claim 1 wherein the image of the environment comprises a video of the environment.

7. The method of claim 6, wherein the placement of the scaled image is being repeatedly performed for the video.

8. The method of claim 1, further comprising providing the augmented reality image to the mobile device operated by the user.

9. The method of claim 1, further comprising providing additional information with the item of information in conjunction with the augmented reality image.

10. The method of claim 9, wherein the additional information comprises an image of a direction indicator.

11. The method of claim 10 wherein the image of the direction indicator includes further image information.

12. The method of claim 1 wherein the image of the environment includes recognizable objects, and identity of at least one of the objects is recognized by one of RFID detection, GPS, triangulation, or facial recognition.

13. The method of claim 1 wherein the environment is a ball park, and the item of information comprises one of the location in the ball park of:

a seat, a person, a selected vendor store, a desired facility, a desired path to a parking area, or a route from the ballpark to a desired location separate from the location of the environment.

14. The method of claim 1, further including targeting an advertisement to the mobile video device based on the user'"'"'s purchase history, interest, or location.

15. One or more non-transitory computer-readable hardware storage device having embedded therein a set of instructions which, when executed by one or more processors of a computer, causes the computer to execute operations comprising:

receiving, from a mobile video device located at an environment, the mobile video device operated by a user who is viewing locations of the environment, the viewing both with and without the use of the mobile video device, image data containing an image of a marker in the environment that has not been placed in the environment for electronic navigation purposes;

receiving a request from the mobile video device for an item of information that provides specific location information that locates a place in the environment, the request generated in accordance with the locations of the environment that the mobile video device is focused on as the user moves in the environment and focuses the mobile video device on locations of the environment, the item of information for placement into an image of the environment on the mobile video device;

responsive to receiving the request, accessing a database to retrieve the item of information;

generating a scaled image of the item of information based on dimensions of the environment and the location of the place in the environment; and

transmitting the scaled image of the item of information that provides specific location information that locates the place in the environment, for placement into the image of the environment on the mobile video device to generate an augmented reality image, the scaled image of the item of information changing in accordance with user movement and the locations of the environment that the mobile video device is focused on.

16. The one or more computer-readable hardware storage device of claim 15, the operations further comprising determining the location where the scaled image is to be placed into the image of the environment.

17. The one or more computer-readable hardware storage device of claim 15, the operations further comprising determining the location of placement of the scaled image into the image of the environment.

18. The one or more computer-readable hardware storage device of claim 15, the operations further comprising determining a location in the environment using the marker located in the environment.

19. The one or more computer-readable hardware storage device of claim 15, the operations further comprising:

determining an orientation of the placement of the scaled image; and

orienting the scaled image to the determined orientation.

20. The one or more computer-readable hardware storage device of claim 15 wherein the image of the environment comprises a video of the environment.

21. The one or more computer-readable hardware storage device of claim 20, the placement of the scaled image being repeatedly performed for the video.

22. The one or more computer-readable hardware storage device of claim 15, the operations further comprising providing the augmented reality image to the mobile device operated by the user.

23. The one or more computer-readable hardware storage device of claim 15, the operations further comprising providing additional information with the item of information in conjunction with the augmented reality image.

24. The one or more computer-readable hardware storage device of claim 23, the additional information comprising an image of a direction indicator.

25. The one or more computer-readable hardware storage device of claim 24 wherein the image of the direction indicator includes further image information.

26. The one or more computer-readable hardware storage device of claim 15 wherein the image of the environment includes recognizable objects, and identity of at least one of the objects is recognized by one of RFID detection, GPS, triangulation, or facial recognition.

27. The one or more computer-readable hardware storage device of claim 15 wherein the environment is a ball park, and the item of information comprises one of the location in the ball park of:

a seat, a person, a selected vendor store, a desired facility, a desired path to a parking area, or a route from the ballpark to a desired location separate from the location of the environment.

28. The one or more computer-readable hardware storage device of claim 15, the operations further comprising targeting an advertisement to the mobile video device based on the user'"'"'s purchase history, interest, or location.

29. A system comprising at least one hardware processor configured to include:

a receiving module to receive, from a mobile video device located at an environment, the mobile video device operated by a user who is viewing locations of the environment, the viewing both with and without the use of the mobile video device, image data containing an image of the marker in the environment that has not been placed in the environment for electronic navigation purposes, the receiving module also to receive a request from the mobile video device for an item of information that provides specific location information that locates a place in the environment, the request generated in accordance with the locations of the environment that the mobile video device is focused on as the user moves in the environment and focuses the mobile video device on locations of the environment, the item of information for placement into a location of image of the environment on the mobile video device;

a database accessing module to, responsive to receipt of the request, access a database to retrieve the item of information;

an image generating module to generate a scaled image of the item of information based on dimensions of the environment and the location of the place in the environment; and

a transmitting module to transmit the scaled image of the item of information that provides specific location information that locates the place in the environment, for placement into the image of the environment on the mobile video device to generate an augmented reality image, the scaled image of the item of information changing in accordance with user movement and the locations of the environment that the mobile video device is focused on.

30. The system of claim 29 further including a targeting module to target an advertisement to the mobile video device based on the user'"'"'s purchase history, interest, or location.