Augmented reality image offset and overlay

• US 8,542,906 B1
• Filed: 05/21/2008
• Issued: 09/24/2013
• Est. Priority Date: 05/21/2008
• Status: Active Grant

• Current Assignee: T-Mobile Innovations LLC (Deutsche Telekom AG)
• Original Assignee: Sprint Communications Company LP (Deutsche Telekom AG)
• Inventors: Persson, Carl J., Wilson, Thomas H.
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): Li, Ruiping
• Application Number:
  US12/124,826
• Time in Patent Office: 1,952 Days
• Field of Search: 382/154, 382/275, 382/289, 382/103, 382/255, 382291-292, 382/294
• US Class Current: 382/154
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A method is provided to implement augmented reality using markers. An image is captured of an environment. An image of a marker is detected in the image of the environment. A virtual image is displayed overlaid on the image of the environment at an offset from the marker, wherein the virtual image is based on the marker.

Claims

1. A method to implement augmented reality using markers, comprising:

capturing an image of an environment that comprises a first marker and a second marker, wherein the first marker and the second marker are separate markers;

displaying the captured image of the environment, wherein displaying the captured image of the environment comprises displaying the first marker and the second marker;

detecting a first image of the first marker and a second image of the second marker in the displayed captured image of the environment;

displaying a first virtual image overlaid on the displayed captured image of the environment at an offset from the first image of the first marker, wherein the first virtual image is based on the detection of the first image of the first marker;

displaying a second virtual image overlaid on the displayed captured image of the environment, wherein the second virtual image is based on the detection of the second image of the second marker; and

displaying a third virtual image overlaid on the displayed captured image of the environment, wherein the third virtual image is based on the detection of a combination of the first image of the first marker and the second image of the second marker, wherein the third virtual image is different from the first virtual image and the second virtual image.

2. The method of claim 1, further comprising displaying another virtual image overlaid on the displayed captured image of the environment in response to a user selection, wherein the other virtual image is based on the first marker.

3. The method of claim 1, further comprising:

detecting another image of another marker that is indirectly associated with the first marker based on at least one of compatibility with an item directly associated with the first marker and customers purchasing pattern; and

displaying another virtual image associated with the other marker.

4. The method of claim 1, wherein at least one of the first marker, the second marker, or the third marker comprises at least one of an asymmetrical pattern, a moving marker, a two dimensional bar code, a color light pattern, an updateable digital marker, a primary marker assigned to a marker hierarchy, a secondary marker assigned to the marker hierarchy, or a dependent marker assigned to the marker hierarchy.

5. The method of claim 1, wherein at least one of the first marker, the second marker, or the third marker is at least one of navigational information, an advertisement, information about an item in the image of the environment, specifications of a product, a schematic drawing, an operational status, an image associated with the item, an outline of the item, or a plurality of colors that indicate information about the item.

6. The method of claim 1, wherein an amount of the offset is based on the first marker, and wherein the first virtual image is displayed at least partially over the first marker and over another portion of the displayed captured image of the environment.

7. The method of claim 1, wherein displaying the first virtual image comprises displaying an animated image based on the first marker.

8. The method of claim 1, wherein the second image of the second marker is situated in the displayed captured image of the environment at a non-parallel angle from the plane of the display, and wherein a plane of the second virtual image is displayed parallel to the plane of the display to compensate for the second image of the second marker in the displayed captured image of the environment being situated at the non-parallel angle from the plane of the display.

9. The method of claim 1, wherein the second virtual image is displayed at least partially over the second image of the second marker and over another portion of the displayed captured image of the environment, and wherein the second virtual image depicts an existing item in the captured image of the environment that is blocked from view by another item in the captured image of the environment.

10. The method of claim 1, further comprising:

accessing information about an item that is associated with the first marker;

determining a size of the item based on the information;

evaluating the displayed captured image of the environment based on a size of the first image of the first marker in the displayed captured image of the environment; and

determining a relative position for the first virtual image based on the size of the item in relation to the displayed captured image of the environment.

11. The method of claim 10, further comprising:

determining whether the first virtual image matches the size of the item in the displayed captured image of the environment; and

outputting an error message in response to a determination that the first virtual image does not match the size of the item in the displayed captured image of the environment.

12. The method of claim 1, further comprising:

analyzing the displayed first virtual image; and

adjusting the displayed first virtual image based on at least one of readability and color perception.

13. The method of claim 9, wherein the second virtual image depicts an existing below-floor portion of the environment that is blocked from view by the other item in the environment, and wherein the existing below-floor portion of the environment comprises one of plumbing beneath the floor or an electrical schematic of wires and cables beneath the floor.

14. A system to implement augmented reality using markers, comprising:

a processor;

a camera associated with a mobile device configured to capture an image of an environment that comprises a first marker and a second marker, wherein the first marker and the second marker are separate markers;

a plurality of instructions in a memory, when executed by the processor detect a first image of the first marker and a second image of the second marker in a displayed captured image of the environment; and

a display associated with the mobile device, wherein the display is configured to;

display the captured image of the environment, wherein displaying the captured image of the environment comprises displaying the first marker and the second marker,

display a first virtual image overlaid on the displayed captured image of the environment at an offset from the first image of the first marker, wherein the first virtual image is based on the detection of the first image of the first marker,

displaying a second virtual image overlaid on the displayed captured image of the environment, wherein the second virtual image is based on the detection of the second image of the second marker, and

displaying a third virtual image overlaid on the displayed captured image of the environment, wherein the third virtual image is based on the detection of a combination of the first image of the first marker and the second image of the second marker, and wherein the third virtual image is different from the first virtual image and the second virtual image.

15. The system of claim 14, wherein the plurality of the instructions in the memory detect the first image of the first marker based on a perspective analysis of at least one of the first marker or the second marker in the environment.

16. The system of claim 14, further comprising a digital compass and an accelerometer to promote the instructions adjusting the display of at least one of the first virtual image, the second virtual image, or the third virtual image overlaid on the displayed captured image of the environment.

17. The system of claim 14, wherein the memory is associated with at least one of the mobile device and a server in communication with the mobile device.

18. The system of claim 14, further comprising a server that comprises a plurality of other virtual images, wherein each of the plurality of other virtual images is associated with at least one marker, and wherein the display is further configured to display one of the plurality of other virtual images based on accessing the server.