IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20140225919A1
• Filed: 09/04/2012
• Published: 08/14/2014
• Est. Priority Date: 10/27/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kaino, Akihiko, Fukuchi, Masaki, Kashitani, Tatsuki, Ooi, Kenichiro
• Granted Patent: US 9,292,974 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An information processing system that acquires video data captured by an image pickup unit; detects an object from the video data; detects a condition corresponding to the image pickup unit; and controls a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

Claims

1. An information processing system comprising:

one or more processing units that;

acquire video data captured by an image pickup unit;

detect an object from the video data;

detect a condition corresponding to the image pickup unit; and

control a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

2. The information processing system of claim 1, wherein

the object detected from the video data is at least one of a shape, symbol, character string, design, object, part of an object and an image displayed on a display.

3. The information processing system of claim 1, wherein

detecting a condition corresponding to the image pickup unit includes detecting at least one of a position and posture of the image pickup unit.

4. The information processing system of claim 3, wherein

the one or more processing units detect at least one of a position and a posture of the detected object.

5. The information processing system of claim 4, wherein

the one or more processors control the position on the display at which the content is displayed based on the at least one of a position and posture of the image pickup unit and the at least one of a position and a posture of the detected object.

6. The information processing system of claim 1, further comprising:

a sensor unit that detects the condition corresponding to the image pickup unit.

7. The information processing system of claim 6, wherein

the sensor unit includes at least one of a positioning sensor, an acceleration sensor and a gyrosensor.

8. The information processing system of claim 1, further comprising:

a memory that stores information corresponding to at least one object in association with information corresponding to the content, wherein

the information corresponding to the content includes at least one of a marker identifier, a related content identifier and a size corresponding to the content to be displayed.

9. The information processing system of claim 1, further comprising:

a memory that stores information corresponding to the detection of the object, wherein

the information corresponding to the detection of the object includes at least one of an object identifier, an object position, an object posture, an object detection time and a time corresponding to when the object was no longer detected in the acquired video data.

10. The information processing system of claim 9, wherein

the one or more processing units control the position on the display at which the content is displayed based on the condition corresponding to the image pickup unit and the information corresponding to the detection of the object.

11. The information processing system of claim 1, wherein

the one or more processing units control the display to move the content on the display based on a change in at least one of a position and a posture of the image pickup unit.

12. The information processing system of claim 1, wherein

the one or more processing units control the display to display the content on the display when the object is no longer detected in the acquired video data.

13. The information processing system of claim 1, wherein

the detected condition corresponding to the image pickup unit includes at least one of a position and a posture of the image pickup unit, and

the one or more processing units control the display to stop displaying the content when at least one of the position and the posture of the image pickup unit satisfies a predetermined condition.

14. The information processing system of claim 13, wherein

the one or more processing units controls the display to display an indication that the display is to stop displaying the content when at least one of the position and the posture of the image pickup unit satisfies the predetermined condition.

15. The information processing system of claim 13, wherein

the predetermined condition corresponds to a distance between the image pickup unit and the detected object, and

the one or more processing units control the display to stop displaying the content when the distance between image pickup unit and the detected object exceeds a predetermined threshold value.

16. The information processing system of claim 13, wherein

the predetermined condition corresponds to a difference in an angle between an optical axis of the image pickup unit and an axis extending from the image pickup unit to the detected object, and

the one or more processing units control the display to stop displaying the content when the difference in angle exceeds a predetermined threshold value.

17. The information processing system of claim 13, wherein

the predetermined condition corresponds to a time elapsed from when the object was first detected in the acquired video data, and

the one or more processing units control the display to stop displaying the content when time elapsed exceeds a predetermined threshold value.

18. The information processing system of claim 13, wherein

the predetermined condition corresponds to a time elapsed from when the object was last detected in the acquired video data, and

the one or more processing units control the display to stop displaying the content when time elapsed exceeds a predetermined threshold value.

19. An information processing method performed by an information processing system, the method comprising:

acquiring video data captured by an image pickup unit;

detecting an object from the video data;

detecting a condition corresponding to the image pickup unit; and

controlling a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

20. A non-transitory computer-readable medium including computer program instructions, which when executed by an information processing system, cause the information processing system to perform a method, the method comprising:

acquiring video data captured by an image pickup unit;

detecting an object from the video data;

detecting a condition corresponding to the image pickup unit; and

controlling a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.