CHANGE INVARIANT SCENE RECOGNITION BY AN AGENT

• US 20120209514A1
• Filed: 02/14/2011
• Published: 08/16/2012
• Est. Priority Date: 02/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chrysanthakopoulos, Georgios
• Granted Patent: US 8,942,917 B2
• Time in Patent Office: Days
• US Class Current: 701/431
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A technology can be provided for change invariant scene recognition of an environment by an agent that uses image sensors to capture images. An example method can include grouping the images into situation image sets to form an image database. A navigation image can be captured by the agent at the agent'"'"'s current location. Another operation can be searching through images by separately loaded situation image set to identify images that are a probable match for the navigation image. Error values can be computed between the navigation image and situation image sets. A probability can be calculated that situation image sets are to be used as a relevant situation image set based on the error values. Then the relevant situation image set can be searched for matching purposes with the navigation image while the error value is below a defined threshold.

Claims

1. A method for change invariant scene recognition of an environment by an agent that uses image sensors to capture images, comprising:

grouping the images into situation image sets for locations and edges of a navigation graph to form an image database;

obtaining a navigation image captured by the agent at the agent'"'"'s current location;

searching through images by separately loaded situation image set from the image database to identify images that are a probable match for the navigation image;

computing error values for matches between the navigation image and situation image sets;

calculating a probability that situation image sets are to be used as a relevant situation image set based on the error values; and

searching the relevant situation image set for matching purposes with the navigation image while the error value is below a defined threshold.

2. The method as in claim 1, further comprising creating a situation image set by a user initiating a situation image set capture.

3. The method as in claim 1, further comprising discarding the situation image sets that have had consistently high error matching results over a defined period of time.

4. The method as in claim 1, further comprising tracking a most relevant situation image set for locations and edges of the navigation graph.

5. The method as in claim 4, further comprising tracking a state that represents the most relevant situation image set for the navigation images currently being captured by the agent.

6. The method as in claim 1, wherein calculating a probability further comprises, computing a transition probability that determines whether the navigation image match is searched for in the current situation image set or whether the agent will change which the situation image set is searched for a match.

7. The method as in claim 1, further comprising using a global image signature for a situation image set for comparing against the navigation image.

8. The method as in claim 1, wherein a situation image set is created during a training session where a human trainer identifies nodes that are locations to the agent and edges linking the nodes.

9. The method as in claim 1, wherein the image sensors are selected from the group consisting of a color camera, grayscale camera, or a depth image camera.

10. The method as in claim 1, wherein the agent is a robotic device, a cell phone, web camera, or a game controller sensor.

11. A system for change invariant scene recognition of an environment by an agent, comprising:

an image camera using image sensors to capture images for the agent;

a grouping module to group the images into situation image sets that are identified as being at locations and edges of a navigation graph;

an image database containing the situation image sets;

a query module to query the image database by obtaining an image signature from a navigation image that is compared with image signatures for situation image sets loaded as separate sets from the image database;

a probability module to calculate a probability that a selected situation image set is to be used as a most relevant situation image set; and

a feature matching module to identify images in the most relevant situation image set that are similar to the navigation image and to provide a navigation location for the agent.

12. The system as in claim 11, wherein the images are grayscale depth images or visual images.

13. The system as in claim 11, further comprising using a global image signature for a situation image set for comparing against the navigation image.

14. The system as in claim 11, wherein the agent is a robotic agent having a navigation module to navigate from a first navigation location to a second navigation location.

15. The system as in claim 11, further comprising a pruning module to remove situation image sets that have not been used for a period of time due to a reduced amount of matching.

16. A method for light invariant scene recognition in an environment by an agent, comprising:

capturing depth images of a scene using a camera of the agent;

capturing visual images, using a visual camera of the agent, from the scene that are correlated to the depth images;

identifying interest points in depth images based on depth gradient;

extracting pixel patches from the visual images that correspond to interest points in the depth images;

matching interest points for agent navigation by using pixel patches from the visual image for the interest points in the depth map.

17. The method as in claim 16, wherein matching interest points further comprises matching interest points in a navigation image to interest points in a visual image using pixel patches extracted from the visual image to identify the agent'"'"'s current location within a navigation graph.

18. The method as in claim 16, further comprising storing the depth images and visual images in separate situation image sets.

19. The method as in claim 18, further comprising matching based on depth images or visual images depending on which type of situation image set has an error value below a defined threshold when a scene illumination has changed.

20. The method as in claim 16, further comprising avoiding light invariance in images using only the depth images to avoid changes in time of day, weather, or artificial lighting.