View of a physical space augmented with social media content originating from a geo-location of the physical space

• US 9,619,489 B2
• Filed: 04/15/2016
• Issued: 04/11/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Tai Technologies Inc. (Crisis24)
• Original Assignee: Geofeedia, Inc.
• Inventors: Mulroy, Michael J., Harris, Philip B., Mitchell, Scott K.
• Primary Examiner(s): SOMERS, MARC S
• Application Number:
  US15/130,289
• Publication Number:
  US 20160232182A1
• Time in Patent Office: 361 Days
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/18 Commands or executable codes
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

The disclosure relates to systems and methods for augmenting a view of a physical space of one or more geographically definable locations (“geo-locations”) with social media and/or other content originating from the one or more geo-locations. Generally speaking, the system may include a computing device having one or more processors programmed to augment (e.g., add to, overlay, embed, etc.) the view of the physical space of a geo-location with social media content, thereby allowing a user to view a physical space at a geo-location along with social media content that was created from the geo-location.

Claims

1. A system for augmenting a view of a physical space at a geographically definable location with social media content originating from the geographically definable location, wherein the augmented view includes an augmented space that is coincident with the physical space, the system comprising:

one or more processors programmed by one or more computer program instructions that, when executed by the one or more processors, program the one or more processors to;

obtain (i) a real-time or stored image of the physical space corresponding to the geographically definable location and (ii) reality contextual information that describes a context of the physical space;

identify the geographically definable location associated with the physical space;

obtain (i) social media content that was created at the geographically definable location, and (ii) content contextual information that describes a context for the social media content, wherein the social media content comprises at least a first social media content item and the content contextual information comprises at least first content contextual information that describes a context in which the first social media content item was created;

compare the reality contextual information with the content contextual information;

determine a match between at least some of the reality contextual information and the content contextual information;

obtain a second orientation of a second device used to create the first social media content item based on the first content contextual information;

determine that the second orientation is different than an orientation of a first device to which the augmented view is provided such that a graphical representation of the first social media content item would not be in the augmented view;

cause an indication to be provided on the augmented view associated with the physical space that indicates a direction in which to move the first device to match the second orientation to bring the graphical representation of the first social media content item into the augmented view;

determine that the graphical representation of the first social media content item is in the augmented view;

cause the graphical representation of the first social media content item to be included in the augmented space that is coincident with the physical space such that the augmented view is augmented with the augmented space; and

cause the augmented view to be provided to the first device.

2. The system of claim 1, wherein the one or more processors are programmed to obtain the reality contextual information from one or more sensors that provide one or more sensor measurements with respect to the physical space.

3. The system of claim 2, wherein the one or more sensor measurements relate to the orientation of the first device that includes the one or more sensors, an ambient condition around the first device, and/or an altitude of the first device.

4. The system of claim 3, wherein to obtain the social media content that was created at the geographically definable location and the content contextual information that describes the context for the social media content, the one or more processors are further programmed to:

generate a first request to be provided to a first social media content provider, wherein the first request comprises a first specification of the geographically definable location in a first format specific to the first social media content provider;

generate a second request to be provided to a second social media content provider, wherein the second request comprises a second specification of the geographically definable location in a second format specific to the second social media content provider, the second format being different than the first format;

obtain at least the first social media content item relevant to the geographically definable location based on the first request, and the first content contextual information that describes the context in which the first social media content item was created; and

obtain at least a second social media content item relevant to the geographically definable location based on the second request, and second content contextual information that describes a context in which the second social media content item was created.

5. The system of claim 4, wherein to determine a match between at least some of the reality contextual information and the content contextual information, the one or more processors are further programmed to:

determine that there exists a first match between at least some of the reality contextual information and the first content contextual information, wherein the first match indicates a match between the context of the physical space and the context in which the first social media content item was created; and

determine that there exists a second match between at least some of the reality contextual information and the second content contextual information, wherein the second match indicates a match between the context of the physical space and the context in which the second social media content item was created.

6. The system of claim 5, wherein the one of more processors are further programmed to:

cause a second graphical representation of the second social media content item to be included in the augmented space responsive to a determination that there exists the second match between at least some of the reality contextual information and the second content contextual information.

7. The system of claim 2, wherein the one or more sensor measurements relate to an altitude of the first device that includes the one or more sensors, and wherein the one or more processors are further programmed to:

determine that the social media content was created at an altitude that is the same as the altitude of the first device.

8. The system of claim 2, wherein the one or more sensor measurements relate to an ambient condition of the first device that includes the one or more sensors, and wherein the one or more processors are further programmed to:

determine that the social media content was created at an ambient condition that is the same as the ambient condition of the first device.

9. The system of claim 2, wherein the one or more sensor measurements relate to an altitude of the first device that includes the one or more sensors and an ambient condition related to the device, and wherein the one or more processors are programmed to:

determine that the social media content was created at an altitude that is the same as the altitude of the first device; and

determine that the social media content was created at an ambient condition that is the same as the ambient condition of the first device.

10. The system of claim 1, wherein to identify the geographically definable location associated with the physical space, the one or more processors are further programmed to:

recognize an object from the image of the physical space; and

associate the recognized object with a recognized location, wherein the geographically definable location is determined based on the recognized object.

11. A method for augmenting a view of a physical space at a geographically definable location with social media content originating from the geographically definable location, wherein the augmented view includes an augmented space that is coincident with the physical space, the method being implemented on a computer having one or more processors programmed with one or more computer program instructions that, when executed by the one or more processors, cause the computer to perform the method, the method comprising:

obtaining, by the computer, (i) a real-time or stored image of the physical space corresponding to the geographically definable location and (ii) reality contextual information that describes a context of the physical space;

identifying, by the computer, the geographically definable location associated with the physical space;

obtaining, by the computer, (i) social media content that was created at the geographically definable location, and (ii) content contextual information that describes a context for the social media content, wherein the social media content comprises at least a first social media content item and the content contextual information comprises at least first content contextual information that describes a context in which the first social media content item was created;

comparing, by the computer, the reality contextual information with the content contextual information;

determining, by the computer, a match between at least some of the reality contextual information and the content contextual information;

obtaining, by the computer, a second orientation of a second device used to create the first social media content item based on the first content contextual information;

determining, by the computer, that the second orientation is different than an orientation of a first device to which the augmented view is provided such that a graphical representation of the first social media content item would not be in the augmented view;

causing, by the computer, an indication to be provided on the augmented view associated with the physical space that indicates a direction in which to move the first device to match the second orientation to bring the graphical representation of the first social media content item into the augmented view;

determining, by the computer, that the graphical representation of the first social media content item is in the augmented view;

causing, by the computer, the graphical representation of the first social media content item to be included in the augmented space that is coincident with the physical space such that the augmented view is augmented with the augmented space; and

causing, by the computer, the augmented view to be provided to the first device.

12. The method of claim 11, wherein the one or more processors are programmed to obtain the reality contextual information from one or more sensors that provide one or more sensor measurements with respect to the physical space.

13. The method of claim 12, wherein the one or more sensor measurements relate to the orientation of the first device that includes the one or more sensors, an ambient condition around the first device, and/or an altitude of the first device.

14. The method of claim 13, wherein obtaining the social media content that was created at the geographically definable location and the content contextual information that describes the context for the social media content comprises:

generating, by the computer, a first request to be provided to a first social media content provider, wherein the first request comprises a first specification of the geographically definable location in a first format specific to the first social media content provider;

generating, by the computer, a second request to be provided to a second social media content provider, wherein the second request comprises a second specification of the geographically definable location in a second format specific to the second social media content provider, the second format being different than the first format;

obtaining, by the computer, at least the first social media content item relevant to the geographically definable location based on the first request, and the first content contextual information that describes the context in which the first social media content item was created; and

obtaining, by the computer, at least a second social media content item relevant to the geographically definable location based on the second request, and second content contextual information that describes a context in which the second social media content item was created.

15. The method of claim 14, wherein determining a match between at least some of the reality contextual information and the content contextual information comprises:

determining, by the computer, that there exists a first match between at least some of the reality contextual information and the first content contextual information, wherein the first match indicates a match between the context of the physical space and the context in which the first social media content item was created; and

determining, by the computer, that there exists a second match between at least some of the reality contextual information and the second content contextual information, wherein the second match indicates a match between the context of the physical space and the context in which the second social media content item was created.

16. The method of claim 15, the method further comprising:

causing, by the computer, a second graphical representation of the second social media content item to be included in the augmented space responsive to a determination that there exists the second match between at least some of the reality contextual information and the second content contextual information.

17. The method of claim 12, wherein the one or more sensor measurements relate to an altitude of the first device that includes the one or more sensors, the method further comprising:

determining, by the computer, that the social media content was created at an altitude that is the same as the altitude of the first device.

18. The method of claim 12, wherein the one or more sensor measurements relate to an ambient condition of the first device that includes the one or more sensors, the method further comprising:

determining, by the computer, that the social media content was created at an ambient condition that is the same as the ambient condition of the first device.

19. The method of claim 12, wherein the one or more sensor measurements relate to an altitude of the first device that includes the one or more sensors and an ambient condition related to the device, the method further comprising:

determining, by the computer, that the social media content was created at an altitude that is the same as the altitude of the first device; and

determining, by the computer, that the social media content was created at an ambient condition that is the same as the ambient condition of the first device.

20. The method of claim 11, wherein identifying the geographically definable location associated with the physical space comprises:

recognizing, by the computer, an object from the image of the physical space; and

associating, by the computer, the recognized object with a recognized location, wherein the geographically definable location is determined based on the recognized object.