IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20160133057A1
• Filed: 01/13/2016
• Published: 05/12/2016
• Est. Priority Date: 10/27/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ooi, Kenichiro, Kaino, Akihiko, Fukuchi, Masaki, Kashitani, Tatsuki, Guo, Jingjing
• Granted Patent: US 9,704,297 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An information processing system that acquires video data captured by an image pickup unit; detects an object from the video data; detects a condition corresponding to the image pickup unit; and controls a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

Claims

1. (canceled)

2. An information processing system comprising:

circuitry configured to;

acquire video data captured by an image pickup unit;

detect at least one of a position and a posture of the image pickup unit corresponding to the video data;

determine one or more parameters of a virtual content based on at least one of the position and the posture of the image pickup unit following a predetermined rule; and

control display of the virtual content based on the one or more parameters of the virtual content,

wherein

the virtual content is capable of expressing a line of sight; and

the one or more parameters of the virtual content include a line-of-sight parameter.

3. The information processing system of claim 2, wherein the predetermined rule is related to at least one of the position and the posture of the image pickup unit relative to a position of the virtual content in a virtual space.

4. The information processing system of claim 3, wherein the predetermined rule is pointing the line of sight of the virtual content to the image pickup unit.

5. The information processing system of claim 2, wherein the one or more parameters of the virtual content further include a transparency parameter.

6. The information processing system of claim 2, wherein the one or more parameters of the virtual content further include a resolution parameter.

7. The information processing system of claim 6, wherein the predetermined rule includes setting the resolution parameter of the virtual content based on a distance between the position of the image pickup unit and a position of the virtual content in a virtual space.

8. The information processing system of claim 2, wherein the predetermined rule is related to a threshold value of a distance between the position of the image pickup unit and a position of the virtual content in a virtual space.

9. The information processing system of claim 2, wherein the predetermined rule is related to a threshold value of an angle corresponding to the posture of the image pickup unit relative to a position of the virtual content in a virtual space.

10. The information processing system of claim 2, wherein the predetermined rule is related to a user operation of the image pickup unit.

11. The information processing system of claim 2, wherein the circuitry is further configured to:

identify an object from the captured video data; and

select the virtual content from a virtual content database based on the identified object.

12. The information processing system of claim 2, wherein the predetermined rule is related to a threshold value of a distance between the position or posture of the image pickup unit and the identified object.

13. The information processing system of claim 2, wherein the circuitry is configured to manage an analyzing result of the video data for detecting at least one of the position and the posture of the image pickup unit.

14. An information processing method performed by an information processing system, the method comprising:

acquiring video data captured by an image pickup unit;

detecting a position or a posture of the image pickup unit corresponding to the video data;

determining one or more parameters of a virtual content based on the position or the posture of the image pickup unit following a predetermined rule; and

controlling display of the virtual content based on the one or more parameters of the virtual content,

wherein

the virtual content is capable of expressing a line of sight; and

the one or more parameters of the virtual content include a line-of-sight parameter.

15. The information processing method of claim 14, wherein

the predetermined rule is related to at least one of the position and the posture of the image pickup unit relative to a position of the virtual content in a virtual space;

and

the predetermined rule is pointing the line of sight of the virtual content to the image pickup unit.

16. The information processing method of claim 14, wherein

the one or more parameters of the virtual content further include a resolution parameter;

and

the predetermined rule includes setting the resolution parameter of the virtual content based on a distance between the position of the image pickup unit and a position of the virtual content in a virtual space.

17. The information processing method of claim 14, further comprising:

identifying an object from the captured video data; and

selecting the virtual content from a virtual content database based on the identified object.

18. A non-transitory computer-readable medium including computer program instructions, which when executed by an information processing system, cause the information processing system to perform a method, the method comprising:

acquiring video data captured by an image pickup unit;

detecting a position or a posture of the image pickup unit corresponding to the video data;

determining one or more parameters of a virtual content based on the position or posture of the image pickup unit following a predetermined rule; and

controlling display of the virtual content based on the one or more parameters of the virtual content,

wherein

the virtual content is capable of expressing a line of sight; and

the one or more parameters of the virtual content include a line-of-sight parameter.

19. The non-transitory computer-readable medium of claim 18, wherein

the predetermined rule is related to at least one of the position and the posture of the image pickup unit relative to a position of the virtual content in a virtual space;

and

the predetermined rule is pointing the line of sight of the virtual content to the image pickup unit.

20. The non-transitory computer-readable medium of claim 18, wherein

the one or more parameters of the virtual content further include a resolution parameter;

and

the predetermined rule includes setting the resolution parameter of the virtual content based on a distance between the position of the image pickup unit and a position of the virtual content in a virtual space.

21. The non-transitory computer-readable medium of claim 18, wherein the method further comprises:

identifying an object from the captured video data; and

selecting the virtual content from a virtual content database based on the identified object.