System and method for effectively performing a scene rectification procedure

• US 20120219221A1
• Filed: 02/25/2011
• Published: 08/30/2012
• Est. Priority Date: 02/25/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yu, Liangyin, Liu, Ming-Chang
• Granted Patent: US 8,379,979 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/38 Outdoor scenes

Abstract

A system and method for effectively performing a scene rectification procedure comprises an image manager that includes a segmentation module, a label module, and a rectification module. The segmentation module initially performs a segmentation procedure upon an image to produce corresponding sub-scenes. The label module then categorizes the sub-scenes by assigning initial labels without utilizing context information from other sub-scenes in the image. The rectification module performs a semantic grouping procedure upon the sub-scenes to produce semantic group nodes corresponding to pairs of the sub-scenes that have a predefined semantic relationship. The rectification module converts a sub-scene graph of the sub-scenes into a semantic graph that includes the semantic group nodes. The rectification module then performs a rectification procedure to convert the initial labels of the sub-scenes into rectified labels. A processor of an electronic device typically controls the image manager for performing the scene rectification procedure.

Claims

1. A system for performing an image recognition procedure, comprising:

an image manager with a rectification module that performs a semantic grouping procedure upon sub-scenes of an image to produce semantic group nodes corresponding to pairs of said sub-scenes that have a semantic relationship, said rectification module converting a sub-scene graph of said sub-scenes into a semantic graph that includes said semantic group nodes, said rectification module performing a rectification procedure to convert initial labels of said sub-scenes into rectified labels; and

a processor of an electronic device that controls said image manager to perform said image recognition procedure.

2. The system of claim 1 wherein said image manager analyzes said rectified labels to perform said image recognition procedure on said image.

3. The system of claim 1 wherein a segmentation module of said image manager initially performs a segmentation procedure upon said image to produce said sub-scenes.

4. The system of claim 3 wherein said segmentation module identifies sub-scene nodes corresponding to locations of said sub-scenes in said image.

5. The system of claim 3 wherein a label module of said image manager performs a labeling procedure upon said sub-scenes to produce said initial labels.

6. The system of claim 3 wherein said label module identifies said initial labels without utilizing context information from other sub-scenes in said image.

7. The system of claim 5 wherein said rectification module creates said semantic group nodes for said pairs of sub-scenes by utilizing predefined grouping criteria.

8. The system of claim 7 wherein said predefined grouping criteria include adjacency characteristics of said pairs of sub-scenes.

9. The system of claim 7 wherein said predefined grouping criteria include class dependency characteristics of said pairs of sub-scenes.

10. The system of claim 7 wherein said predefined grouping criteria include semantic co-occurence characteristics of said pairs of sub-scenes.

11. The system of claim 7 wherein said rectification module creates said sub-scene graph as a sub-scene adjacency graph that connects said pairs of sub-scenes that have predefined adjacency characteristics.

12. The system of claim 11 wherein said rectification module creates said semantic graph to include sub-scene nodes and semantic group nodes for each of said pairs of sub-scenes.

13. The system of claim 11 wherein said rectification module performs said rectification procedure by utilizing linear optimization techniques.

14. The system of claim 11 wherein said rectification module performs said rectification procedure by utilizing a Bayesian framework.

15. The system of claim 14 wherein said Bayesian framework includes probability values for said rectified labels of said pairs of sub-scenes and a corresponding one of said semantic group nodes from said semantic graph.

16. The system of claim 15 wherein said probability values are derived by performing a machine learning procedure upon images in an image database.

17. The system of claim 15 wherein said rectification module calculates a belief network function for one of said pairs of said sub-scenes by multiplying probability values for said rectified labels of one of said pairs of sub-scenes and a corresponding one of said semantic group nodes.

18. The system of claim 17 wherein said rectification module calculates a global belief function for all of said pairs of said sub-scenes by multiplying all of said belief network functions, a maximum value of said global belief function indicating said rectified labels.

19. The system of claim 18 wherein said rectification module calculates said global belief function as a context-sensitive global belief function by also including a context condition.

20. A method for performing an image recognition procedure with an image manager, comprising the steps of:

performing a semantic grouping procedure upon sub-scenes of an image to produce semantic group nodes corresponding to pairs of said sub-scenes that have a semantic relationship;

converting a sub-scene graph of said sub-scenes into a semantic graph that includes said semantic group nodes;

performing a rectification procedure to convert initial labels of said sub-scenes into rectified labels; and

controlling said image manager with a processor of an electronic device to perform said image recognition procedure.