Event Determination From Photos

• US 20120251011A1
• Filed: 04/04/2011
• Published: 10/04/2012
• Est. Priority Date: 04/04/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hua, Xian-Sheng, Gao, Mingyan
• Granted Patent: US 8,831,352 B2
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Events may be determined based on an image and context data associated with the image. An event type associated with the image may be determined based on a concept of the image. A list of events may be retrieved from an event database based on the context data. The retrieved list of events may then be ranked based on the determined event type and the context data. Through this event determination, a user may obtain information of one or more events happening at a specific location simply by capturing an image of that specific location, thereby saving the user from searching and browsing the Internet or brochure to locate the information of the one or more events at the specific location.

Claims

1. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

receiving an image;

extracting image features from the image;

determining a concept associated with image based at least on the extracted image features, the concept including an action, an object and/or a background scene shown in the image; and

determining an event type of an event shown in the image based at least on the determined concept.

2. The computer-implemented method as recited in claim 1, further comprising:

receiving context data associated with the image, the context data including spatial data and/or temporal data associated with the image; and

retrieving a first list of events from an event database based at least on the context data.

3. The computer-implemented method as recited in claim 2, further comprising obtaining a second list of events from the first list of events based on the determined event type.

4. The computer-implemented method as recited in claim 1, wherein the image comprises a scene captured using the client device, a scene captured by an application of the client device, a scene captured by the client device being pointed at the scene without snapping an image of the scene.

5. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

receiving an image;

obtaining context data associated with the image;

comparing the context data to an event database to retrieve a first list of events;

analyzing the image to determine an event type of an event shown in the image; and

comparing event types of the events in the first list with the determined event type to obtain a second list of events.

6. The computer-implemented method as recited in claim 5, wherein obtaining context data associated with the image includes receiving the context data along with the image in a query, or generating the context data based at least on content of the image.

7. The computer-implemented method as recited in claim 5, wherein the context data includes spatial data and/or temporal data associated with an event shown in the image.

8. The computer-implemented method as recited in claim 7, wherein the spatial data associated with the event shown in the image includes a location at which the image was captured, and the temporal data associated with the event shown in the image includes a time at which the image was captured.

9. The computer-implemented method as recited in claim 7, further comprising ranking a first event in the second list at a position higher than a second event in the second list in response to determining that the location associated with the event shown in the image is closer to a location associated with the first event than a location associated with the second event.

10. The computer-implemented method as recited in claim 7, further comprising ranking a first event in the second list at a position higher than a second event in the second list in response to determining that the time associated with the event shown in the image is closer to a time associated with the first event than a time associated with the second event.

11. The computer-implemented method as recited in claim 5, further comprising ranking a first event in the second list at a position higher than a second event in the second list in response to determining that an event type of the first event matches the determined event type and an event type of the second event does not matches the determined event type.

12. The computer-implemented method as recited in claim 5, further comprising returning the second list of events to a client from which the image is received.

13. The computer-implemented method as recited in claim 5, wherein the event database includes planned events and emergent events, the planned events including events that are pre-scheduled to occur and the emergent events including events which occurrences are impromptu.

14. The computer-implemented method as recited in claim 5, further comprising:

determining a point of focus in the image; and

estimating a location of the point of focus based on the context data and the image.

15. The computer-implemented method as recited in claim 14, wherein comparing the context data to the event database includes comparing the location of the point of focus to the event database to retrieve the first list of events, the events in the first list being occurred within a predetermined distance from the location of the point of focus.

16. One or more computer-readable media configured with computer-executable instructions that, when executed by one or more processors, configure the one or more processors to perform acts comprising:

receiving an image and context data associated with the image, the context data including spatial information and temporal information associated with the image;

analyzing content of the image to detect an event type shown in the image;

retrieving a first list of events from an event database based on the spatial information and the temporal information associated with the image; and

ranking the first list of events to obtain a ranked list of events based on the detected event type of the image, the spatial information and the temporal information.

17. The one or more computer-readable media as recited in claim 16, wherein the spatial information associated with the image includes a location at which the image was captured, and the temporal information associated with the image includes a date and/or time at which the image was captured.

18. The one or more computer-readable media as recited in claim 17, wherein retrieving the first list of events from an event database includes retrieving one or more events that occur within a predetermined distance from the location at which the image is captured and within a predetermined time range before and after the time at which the image is captured.

19. The one or more computer-readable media as recited in claim 16, wherein the acts further comprise:

determining a point of focus in the image; and

estimating a location of the point of focus based on the context data and the image.

20. The one or more computer-readable media as recited in claim 19, wherein retrieving the first list of events from an event database is further based on comparing the location of the point of focus to the event database to retrieve the first list of events, the events in the first list being occurred within a predetermined distance from the location of the point of focus.