Systems and methods for an augmented reality platform

• US 9,665,986 B2
• Filed: 02/24/2015
• Issued: 05/30/2017
• Est. Priority Date: 07/15/2011
• Status: Active Grant

• Current Assignee: PicPocket-Labs, Inc.
• Original Assignee: Mark Skarulis
• Inventors: Skarulis, Mark
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US14/630,251
• Publication Number:
  US 20150178993A1
• Time in Patent Office: 826 Days
• Field of Search: None
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Systems, methods, and media for augmenting a view of reality. In an embodiment, a medium, a first marker, and metadata are received from a depository. The first marker comprises at least a portion of a view of reality, and is matched to at least a first portion of the view of reality. Subsequently, a second marker is generated based on at least a second portion of the view of reality, and uploaded to the depository. In addition, the medium is superimposed over at least a portion of the view of reality, based on at least one of the first marker and the second marker, to generate an augmented view of reality.

Claims

1. A system comprising:

at least one hardware processor; and

one or more modules configured to, when executed by the at least one hardware processor,

receive a medium, a first marker, and metadata from a depository, wherein the first marker comprises at least a portion of a view of reality,

match the first marker to at least a first portion of the view of reality,

after the first marker is matched to the at least a first portion of the view of reality, generate a second marker based on at least a second portion of the view of reality, and upload the second marker to the depository, and

superimpose the medium over at least a portion of the view of reality, based on at least one of the first marker and the second marker, to generate an augmented view of reality, wherein the metadata comprises position data, the position data comprising at least one of a location and orientation of a device that generated the marker at a time when the marker was generated.

2. The system of claim 1, wherein the system comprises a client device, and wherein the one or more modules are further configured to provide instructions to a user of the client device on how to position the client device based on the position data.

3. The system of claim 1, wherein the system comprises a client device, wherein the metadata comprises a location of a device that generated the marker at a time when the marker was generated, and wherein the one or more modules are further configured to automatically retrieve at least the medium and the first marker from the depository when the client device is within a predetermined range of the location.

4. The system of claim 1, wherein the metadata comprises a relationship between the medium and the first marker, and wherein the one or more modules are further configured to superimpose the medium over the at least a first portion or the at least a second portion of the view of reality based on the relationship between the medium and the first marker.

5. The system of claim 1, wherein the one or more modules are further configured to match the first marker to the at least first portion of the view of reality by matching one or more patterns in the first marker to one or more patterns in the view of reality.

6. The system of claim 1, wherein the system comprises a client device, and wherein the one or more modules are further configured to match the first marker to the at least a first portion of the view of reality by matching one or more patterns in the first marker to one or more patterns in the view of reality and matching the position data to at least one of a location and orientation of the client device.

7. The system of claim 6, wherein the one or more modules are configured to:

attempt to match one or more patterns in the first marker to one or more patterns in the view of reality; and

,

when it cannot match one or more patterns in the first marker to one or more patterns in the view of reality, attempt to match the position data to at least one of a location and orientation of the client device.

8. The system of claim 6, wherein the one or more modules are configured to attempt to match the position data to at least one of a location and orientation of the client device by providing instructions to a user of the client device, wherein the instructions are based on the position data and the at least one of the location and orientation of the client device.

9. The system of claim 1, wherein the client viewer module is further configured to store an image of the augmented view of reality in a memory.

10. The system of claim 1, wherein the system comprises a client device, wherein generating the second marker comprises generating a high-density image using one or more omni-directional cameras, wherein the second marker comprises a high-density image, and wherein matching the second marker to the least a second portion of the view of reality comprises approximating the relative position between the client device and the second marker using inverse photogrammetry.

11. The system of claim 1, further comprising an embeddable code, wherein the embeddable code, when executed:

superimposes the medium over a captured portion of the view of reality; and

changes a transparency of the superimposed medium in response to one or more user interactions.

12. The system of claim 1, wherein the at least a first portion of the view of reality and the at least a second portion of the view of reality are identical.

13. The system of claim 1, wherein the at least a first portion of the view of reality and the at least a second portion of the view of reality are different.

14. A method comprising, by a client device comprising at least one processor and at least one memory:

receiving a medium, a first marker, and metadata from a depository, wherein the first marker comprises at least a portion of a view of reality;

matching the first marker to at least a first portion of the view of reality;

after the first marker is matched to the at least a first portion of the view of reality, generating a second marker based on at least a second portion of the view of reality, and uploading the second marker to the depository; and

superimposing the medium over at least a portion of the view of reality, based on at least one of the first marker and the second marker, to generate an augmented view of reality, wherein the metadata comprises position data, the position data comprising at least one of a location and orientation of a second device at a time when the marker was generated.

15. The system of claim 14, further comprising instructing a user of the client device on how to position the client device based on the position data.

16. The method of claim 15, wherein the instructions comprise an outline of the marker.

17. The method of claim 14, wherein the metadata comprises a relationship between the medium and the marker, and wherein superimposing the medium over the at least a first portion or the at least a second portion of the view of reality is based on the relationship between the medium and the marker.

18. The system of claim 14, wherein the matching comprises:

attempting to match one or more patterns in the first marker to one or more patterns in the view of reality; and

if unable to match one or more patterns in the first marker to one or more patterns in the view of reality, attempting to match the position data to at least one of a location and orientation of the client device.

19. The method of claim 18, wherein attempting to match the position data to at least one of a location and orientation of the client device comprises providing instructions to a user of the client device based on the position data and the at least one of the location and orientation of the client device.

20. The method of claim 14, further comprising storing an image of the augmented view of reality in a memory.

21. A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to:

receive a medium, a first marker, and metadata from a depository, wherein the first marker comprises at least a portion of a view of reality;

match the first marker to at least a first portion of the view of reality;

after the first marker is matched to the at least a first portion of the view of reality, generate a second marker based on at least a second portion of the view of reality, and upload the second marker to the depository; and

superimpose the medium over at least a portion of the view of reality, based on at least one of the first marker and the second marker, to generate an augmented view of reality.