Image processing apparatus, image processing method, and control program

• US 20020154070A1
• Filed: 03/12/2002
• Published: 10/24/2002
• Est. Priority Date: 03/13/2001
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Sato, Hiroaki, Iizuka, Yoshio, Morita, Kenji, Aso, Takashi, Kawai, Tomoaki, Shimoyama, Tomohiko, Matsui, Taichi
• Granted Patent: US 7,038,699 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

This invention has as its object to provide a high-quality composite image of a real image and virtual image to a viewer or the like. To achieve this object, the position/posture of photographing means for photographing a performer or the like is measured, the position/posture of an HMD, which the performer or the like wears, and which allows the viewer to see through the external world, is measured, a CG at the viewpoint of the photographing means and image data photographed by the photographing means are aligned and composited on the basis of the measurement result of the position/posture of the photographing means, CG at the viewpoint of the HMD and image data photographed by a camera built in the HMD are aligned and composited on the basis of the measurement result of the position/posture of the HMD, and image data is displayed on the HMD.

Claims

1. An image processing apparatus characterized by comprising:

first measurement means for measuring a position/posture of photographing means for photographing an object such as a performer or the like;

second measurement means for measuring a position/posture of the object such as the performer or the like;

first image composition means for aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of said first measurement means;

second composition means for aligning and compositing CG data at a viewpoint of a display means and image data of real image based on the position/posture of the object on the basis of a measurement result of said second measurement means; and

display control means for displaying the image data formed by said second composition means on the display means.

3. The apparatus according to claim 1, characterized in that the display means comprises an HMD (Head Mount Display) which can see through an external world.

4. The apparatus according to claim 1, characterized by further comprising CG management means for managing the CG data, and scenario management means for managing a scenario, and in that said CG management means designates CG data to be composited or displayed on the basis of the scenario managed by said scenario management means.

5. The apparatus according to claim 1, characterized by further comprising prohibition means for setting a prohibited region in association with a layout position of the CG data to be composited or displayed.

6. The apparatus according to claim 1, characterized by further comprising server means for receiving a request from a viewer, and operator means for reflecting the request from the viewer received by said server means in the scenario managed by said scenario management means.

7. The apparatus according to claim 6, characterized in that the request from the viewer is assigned to a plurality of CG data or a predetermined building component of sole CG data for each viewer.

8. The apparatus according to claim 6, characterized in that said server means comprises limitation means for limiting a viewer from which a request is received.

9. The apparatus according to claim 1, further comprising introduction means for externally introducing the CG data.

10. The apparatus according to claim 1, characterized by further comprising an encoder for encoding image data composited by said first and second composition means to be accessible via an Internet.

11. The apparatus according to claim 1, characterized in that CG data associated with an advertisement is included as one of the CG data.

12. The apparatus according to claim 11, characterized by further comprising change means for changing the CG data associated with the advertisement.

13. The apparatus according to claim 1, characterized by further comprising multi-channel control means for converting image data composited by said first and second composition means into multi-channel data which can be simultaneously broadcasted in a plurality of channels of an Internet, satellite broadcast, ground wave broadcast, and the like.

14. The apparatus according to claim 13, characterized in that said multi-channel control means comprises selection means for selecting image data to be converted into the multi-channel data.

15. The apparatus according to claim 14, characterized in that said selection means selects pre-set image data from image data composited by said first and second composition means.

16. The apparatus according to claim 14, characterized in that said selection means selects image data to be converted into the multi-channel data on the basis of information described in the scenario.

17. The apparatus according to claim 14, characterized in that said selection means selects image data to be converted into the multi-channel data in accordance with a request from a viewer.

18. The apparatus according to claim 1, characterized in that said image processing apparatus is built in a photographing studio apparatus.

21. The method according to claim 19, characterized in that the display means comprises an HMD (Head Mount Display) which can see through an external world.

2. An image processing apparatus characterized by comprising:

first measurement means for measuring a position/posture of photographing means for photographing an object such as a performer or the like;

second measurement means for measuring a position/posture of the object such as the performer or the like;

first image composition means for aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of said first measurement means; and

display control means for controlling a position of CG data at a viewpoint of a display means on the basis of a measurement result of said second measurement means, and displaying the CG data on the display means.

19. An image processing method characterized by comprising the steps of:

measuring a position/posture of photographing means for photographing an object such as a performer or the like;

measuring a position/posture of the object such as the performer or the like;

aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of the position/posture of the photographing means;

aligning and compositing CG data at a viewpoint of a display means and image data of real image based on the position/posture of the object on the basis of a measurement result of the position/posture of the display means; and

displaying image data on the display means.

20. An image processing method characterized by comprising the steps of:

measuring a position/posture of photographing means for photographing an object such as a performer or the like;

measuring a position/posture of the object such as the performer or the like;

aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of the position/posture of the photographing means; and

controlling a position of CG data at a viewpoint of a display means on the basis of a measurement result of the position/posture of the display means, and displaying the CG data on the display means.

22. A control program characterized by having contents of:

measuring a position/posture of photographing means for photographing an object such as a performer or the like;

measuring a position/posture of the object such as the performer or the like;

aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of the position/posture of the photographing means;

aligning and compositing CG data at a viewpoint of a display means and image data of real image base on the position/posture of the object on the basis of a measurement result of the position/posture of the display means; and

displaying image data on the display means.

24. The program according to claim 22, characterized in that the display means comprises an HMD (Head Mount Display) which can see through an external world.

23. A control program characterized by having contents of:

measuring a position/posture of photographing means for photographing an object such as a performer or the like;

measuring a position/posture of the object such as the performer or the like;

aligning and compositing CG (Computer Graphics) data at a viewpoint of the photographing means and image data photographed by the photographing means on the basis of a measurement result of the position/posture of the photographing means; and

controlling a position of CG data at a viewpoint of a display means on the basis of a measurement result of the position/posture of the display means, and displaying the CG data on the display means.