IMAGE PROCESSING SYSTEM

• US 20100134688A1
• Filed: 09/11/2009
• Published: 06/03/2010
• Est. Priority Date: 11/28/2008
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: MORIWAKE, Katsuakira
• Application Number:
  US12/558,087
• Publication Number:
  US 20100134688A1
• Time in Patent Office: Days
• US Class Current: 348/586
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A computer graphics generation system combines video images of a scene captured by a camera with one or more rendered computer generated objects. The system comprises a camera which is arranged to generate an image signal representative of a scene including a reference object of a predetermined shape, and an image processor. The image processor is arranged in operation to identify the reference object from the image signal, to detect a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and to estimate a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the luminance distribution across the surface of the reference object is detected for luminance above a threshold clipping level. Accordingly a reduction in the “wobble” of the computer generated objects in the scene can be achieved and a more stable image provided from the image light direction estimation.

Claims

1. An image processor arranged to receive an image signal representative of a scene including a reference object of a predetermined shape, the image processor being operable

to identify the reference object from the image signal,

to detect a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and

to estimate a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the luminance distribution across the surface of the reference object is detected for luminance above a threshold clipping level.

2. An image processor according to in claim 1, wherein the predetermined shape is at least partly spherical.

3. An image processor according to claim 1, wherein the reference object is identified by the reference object occupying a predefined position within the scene.

13. A system according to in claim 1, wherein the predetermined shape is at least partly spherical.

14. A system according to claim 1, wherein the reference object is identified by the reference object occupying a predefined position within the scene.

4. A system for generating composite images comprising an image processor and a camera, the camera being operable to capture an image representative of a scene including a reference object of a predetermined shape and to communicate an information signal representative of the image to the image processor, the image processor being operable

to identify the reference object from the image signal,

to detect a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object,

to estimate a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, the luminance distribution across the surface of the reference object being detected for luminance above a threshold clipping level, and

to generate a composite image comprising image data captured by the camera combined with a rendering of a computer generated object, the computer generated object rendered in accordance with the estimated direction of light.

5. A system according to claim 4, wherein the scene includes a calibration surface comprising a calibration pattern, the image processor being operable to determine an orientation and a position of the camera by reference to the calibration surface and thereby determine the direction of light relative to the calibration surface.

6. A method of estimating a direction of light incident on a reference object of predetermined shape, comprising

identifying the reference object from a received image signal representative of a scene including the reference object,

detecting a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and

estimating a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the detection of the luminance distribution across the surface of the reference object is only detected for luminance above a threshold clipping level.

7. A method according to claim 6, wherein the predetermined shape is substantially spherical.

8. A method according to claim 6, wherein the reference object is identified by the reference object occupying a predefined position within the scene.

9. A computer program having computer executable instructions, which when loaded on to a computer causes the computer to perform a method of estimating a direction of light incident on a reference object of predetermined shape, comprising

identifying the reference object from a received image signal representative of a scene including the reference object,

detecting a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and

estimating a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the detection of the luminance distribution across the surface of the reference object is only detected for luminance above a threshold clipping level.

10. A computer program according to claim 9, wherein the predetermined shape is substantially spherical.

11. A computer program according to claim 9, wherein the reference object is identified by the reference object occupying a predefined position within the scene.

12. A computer graphics generation system for combining video images of a scene captured by a camera with one or more rendered computer generated objects, the system comprising

a camera which is arranged to generate an image signal representative of a scene including a reference object of a predetermined shape, and

an image processor which is arranged in operation

to identify the reference object from the image signal,

to detect a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and

to estimate a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the luminance distribution across the surface of the reference object is detected for luminance above a threshold clipping level.

15. An apparatus for estimating a direction of light incident on a reference object of predetermined shape, the apparatus comprising

means for identifying the reference object from a received image signal representative of a scene including the reference object,

means for detecting a luminance distribution across a surface of the reference object by estimating a luminance magnitude at a plurality of surface points on the surface of the reference object, and

means for estimating a direction of light incident on the reference object derived from the detected luminance distribution across the surface of the reference object by calculating the average of a plurality of luminance vectors, each luminance vector corresponding to one of the surface points and comprising a luminance magnitude of the corresponding surface point and a luminance direction corresponding to a direction perpendicular to the surface at the corresponding surface point, wherein the detection of the luminance distribution across the surface of the reference object is only detected for luminance above a threshold clipping level.