Systems and methods for an augmented reality platform

• US 8,963,957 B2
• Filed: 07/13/2012
• Issued: 02/24/2015
• Est. Priority Date: 07/15/2011
• Status: Active Grant

• Current Assignee: PicPocket-Labs, Inc.
• Original Assignee: Mark Skarulis
• Inventors: Skarulis, Mark
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US13/549,081
• Publication Number:
  US 20130016123A1
• Time in Patent Office: 956 Days
• Field of Search: None
• US Class Current: 345/633
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Systems and methods for augmenting a view of reality. In an embodiment, a first medium is superimposed over a first view of reality. One or more changes to the superimposed medium are received, such as a change in transparency, change in size, and change in position. A first marker, comprising at least a portion of the first view of reality, is generated. First metadata related to the first medium and/or the first marker are also generated. The first medium, the first marker, and the first metadata are sent to a depository. In a further embodiment, a second medium, second marker, and second metadata are received from the depository. The second marker is matched to a least a portion of a second view of reality, and the second medium is superimposed over the at least a portion of the second view of reality to generate an augmented view of reality.

Claims

1. A system for augmenting a view of reality, the system comprising:

a client perspective module stored on a client device, the client device comprising at least one processor, the client perspective module configured to, when executed by the at least one processor,

superimpose a first medium over a first view of reality,

receive one or more of a change in transparency of the superimposed first medium, a change in size of the superimposed first medium, and a change in position of the superimposed first medium,

generate a first marker, the first marker comprising at least a portion of the first view of reality,

generate first metadata related to at least one of the first medium and the first marker wherein the first metadata comprises first position data including at least one of a location and orientation of the client device at a time when the first marker was generated, and

send the first medium, the first marker, and the first metadata to a depository; and

a client viewer module stored on the client device, the client viewer module configured to, when executed by the at least one processor,

receive a second medium, a second marker, and second metadata from the depository, wherein the second metadata comprises second position data including at least one of a location and orientation of a second device at a time when the second marker was generated,

match the second marker to at least a portion of a second view of reality,

superimpose the second medium over the at least a portion of the second view of reality to generate an augmented view of reality, and

provide instructions to a user of the client device on how to position the client device based on the second position data.

2. The system of claim 1, wherein the second metadata comprises a location of a second device at a time when the marker was generated, and the client viewer module is further configured to automatically retrieve at least the second medium and the second marker from the depository when the client device is within a predetermined range of the location.

3. The system of claim 1, wherein the second metadata comprises a relationship between the second medium and the second marker, and wherein the client viewer module is configured to superimpose the second medium over the at least a portion of the second view of reality based on the relationship between the second medium and the second marker.

4. The system of claim 1, wherein the client viewer module is configured to match the second marker to the at least a portion of the second view of reality by matching one or more patterns in the second marker to one or more patterns in the second view of reality.

5. The system of claim 1, wherein the client viewer module is configured to match the second marker to the at least a portion of the second view of reality by matching one or more patterns in the second marker to one or more patterns in the second view of reality and matching the second position data to at least one of a location and orientation of the client device.

6. The system of claim 5, wherein the client viewer module is configured to:

attempt to match one or more patterns in the second marker to one or more patterns in the second view of reality; and

if it cannot match one or more patterns in the second marker to one or more patterns in the second view of reality, attempt to match the second position data to at least one of a location and orientation of the client device.

7. The system of claim 5, wherein the client viewer module is configured to attempt to match the second position data to at least one of a location and orientation of the client device by providing instructions to a user of the client device, the instructions based on the position data and the at least one of the location and orientation of the client device.

8. The system of claim 1, wherein the client viewer module is further configured to store an image of the augmented view of reality in a memory.

9. The system of claim 1, wherein generating the first marker comprises generating a high-density image using one or more omni-directional cameras, wherein the second marker comprises a high-density image, and wherein matching the second marker to at least a portion of the second view of reality comprises approximating the relative position between the client device and the second marker using inverse photogrammetry.

10. The system of claim 1, further comprising an embeddable code, wherein the embeddable code, when executed:

superimposes the first medium over a captured portion of the first view of reality; and

changes a transparency of the superimposed first medium in response to one or more user interactions.

11. A method for augmenting a view of reality, the method comprising, by a client device comprising at least one processor and at least one memory:

superimposing a medium over a view of reality;

receiving one or more of a change in transparency of the superimposed medium, a change in size of the superimposed medium, and a change in position of the superimposed medium,

generating a marker, the marker comprising at least a portion of the view of reality;

generating metadata related to at least one of the medium and the marker, wherein the metadata comprises position data including at least one of a location and orientation of the client device at a time when the marker was generated;

sending the medium, the marker and the metadata to a depository, and

instructing a user of a second client device on how to position the second client device based on the position data.

12. A method for augmenting a view of reality, the method comprising, by a client device comprising at least one processor and at least one memory:

receiving a medium, a marker and metadata from a depository, the marker comprising at least a portion of a view of reality and the metadata comprising position data including at least one of a location and orientation of a second device at a time when the marker was generated;

matching the marker to at least a portion of the view of reality;

superimposing the medium over the matched at least a portion of the view of reality to generate an augmented view of reality, and

instructing a user of the client device on how to position the client device based on the position data.

13. The method of claim 12, wherein the instructions comprise an outline of the marker.

14. The method of claim 12, wherein the metadata comprises a relationship between the medium and the marker, and wherein superimposing the medium over the matched at least a portion of the view of reality is based on the relationship between the medium and the marker.

15. The method of claim 12, further comprising, once the marker is matched to the at least a portion of the view of reality:

generating a new marker based on the at least a portion of the view of reality; and

matching the new marker to at least a portion of the view of reality.

16. The method of claim 12, wherein the matching comprises:

attempting to match one or more patterns in the marker to one or more patterns in the view of reality; and

if unable to match one or more patterns in the marker to one or more patterns in the view of reality, attempting to match the position data to at least one of a location and orientation of the client device.

17. The method of claim 16, wherein attempting to match the position data to at least one of a location and orientation of the client device comprises providing instructions to a user of the client device, the instructions based on the position data and the at least one of the location and orientation of the client device.

18. The method of claim 12, further comprising storing an image of the augmented view of reality in a memory.