IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD AND PROGRAM

• US 20150227798A1
• Filed: 10/18/2013
• Published: 08/13/2015
• Est. Priority Date: 11/02/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Nagano, Hidetoshi
• Granted Patent: US 9,785,839 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

The present invention relates to an image processing device, an image processing method, and a program, which combine an image with less sense of incongruity.

A high accuracy marker detector and a high sensitivity marker detector each detects a marker on an image by a different detection method. The high accuracy marker detector detects the position and the posture of the marker with higher accuracy, and the high sensitivity marker detector has a lower accuracy to detect the marker than the high accuracy marker detector, but detects the position of the marker with high sensitivity by prioritizing detecting the marker. A detection result selector selects the optimal detection result from a plurality of the detection results. The present technique is applicable to, for example, an image processing device which replaces the marker with another image.

Claims

1. An image processing device comprising:

a plurality of detectors configured to each detect a marker appearing on an image by a different detection method; and

a selector configured to select, from detection results of the marker by the detectors, the detection result to be used in an image process using the marker.

2. The image processing device according to claim 1, further comprising an image combiner configured to combine, on the image, a replacement image to be replaced and displayed, based on the detection result of the marker selected by the selector, on an area where the marker appears on the video.

3. The image processing device according to claim 2, wherein

the detectors include at least a high accuracy detector configured to detect the marker with high accuracy and a high sensitivity detector configured to detect the marker with high sensitivity, and

the image processing device further comprises an image transformer configured to transform the replacement image and provide the image combiner with the transformed replacement image according to a position and a posture of the marker based on the detection result by the high accuracy detector, when the detection result by the high accuracy detector is selected by the selector.

4. The image processing device according to claim 3, wherein the image combiner performs a process of painting a position of the marker in a basic color based on that of the replacement image, when the detection result by the high sensitivity detector is selected by the selector.

5. The image processing device according to claim 3, further comprising an estimator configured to estimate an area where the marker appears based on the detection result by the high sensitivity detector.

6. The image processing device according to claim 5, wherein the image transformer transforms the replacement image so as to be inscribed within the area, where the marker appears, estimated by the estimator.

7. The image processing device according to claim 5, wherein

the estimator detects a vertex position of the estimated area where the marker appears, and

the image transformer transforms the replacement image based on the vertex position detected by the estimator.

8. The image processing device according to claim 5, wherein

the estimator detects sides of the estimated area where the marker appears, and

the image transformer transforms the replacement image based on a point of intersection of the sides detected by the estimator.

9. The image processing device according to claim 3, wherein when the detection results by the high accuracy detector and by the high sensitivity detector indicate both of the detectors have not detected the marker, the selector determines that the marker does not appear on the image.

10. The image processing device according to claim 3, wherein when the detection results by the high accuracy detector and by the high sensitivity detector indicate both of the detectors have detected the marker, the selector selects the detection result by the high accuracy detector.

11. The image processing device according to claim 3, wherein when the detection results by the high accuracy detector and by the high sensitivity detector indicate either of the detectors has detected the marker, the selector selects the detection result by the detector which has detected the marker.

12. The image processing device according to claim 3, wherein the high sensitivity detector detects the marker by a detection method for extracting a designated color from the image.

13. The image processing device according to claim 3, wherein the high accuracy detector detects the marker by a detection method for performing a pattern recognition to a known pattern used for the marker.

14. The image processing device according to claim 13, wherein after transforming the marker to an image on which the marker seems to be captured from a front using a component for rough detection, the high accuracy detector performs the pattern recognition using a component for detail detection.

15. An image processing method comprising the steps of:

detecting a marker on an image by a plural of different detection methods; and

selecting, from a plurality of detection results of the marker, the detection result to be used in an image process using the marker.

16. A program for causing a computer to execute an image process including the steps of:

detecting a marker on an image by a plural of different detection methods; and

selecting, from a plurality of detection results of the marker, the detection result to be used in an image process using the marker.