Image processing apparatus and method, printed matter production apparatus and method, and printed matter production system

• US 20030210461A1
• Filed: 03/12/2003
• Published: 11/13/2003
• Est. Priority Date: 03/15/2002
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ashizaki, Koji, Toyoda, Takahiro, Shirakura, Akira
• Granted Patent: US 7,565,003 B2
• Time in Patent Office: Days
• US Class Current: 359/443
• CPC Class Codes: G03H 1/268 Holographic stereogram
• CPC Class Codes: G03H 2210/62 Moving object
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

An apparatus capable of converting motion image data, which is captured in an ordinarily way without intending to produce a parallax image printed matter, to data that can be printed as a parallax image printed matter is provided. The apparatus includes the steps of specifying at least one frame as a reference frame from a plurality of frames constituting the motion image data, and setting a point-of-regard on an object in the reference frame, calculating an amount of movement of the point-of-regard using a relation between the point-of-regard in the reference frame and a point corresponding thereto in another frame, obtaining a motion compensation amount for each frame based on this amount of movement thereof, and executing a motion compensation processing for the motion image data based on the motion compensation amount, thereby converting the motion image data to parallax image data.

Claims

1. An image processing apparatus comprising:

a point-of-regard setting means for selecting at least one frame as a reference frame from a plurality of frames that constitute motion image data and setting a point-of-regard on an object in said reference frame; and

a motion compensation processing means for calculating an amount of movement of the point-of-regard using a relation between the point-of-regard set on the object in said reference frame and a point corresponding to said point-of-regard in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement thereof, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax imaged printed matter, in which a perceived image varies depending on a viewing angle.

2. The image processing apparatus as claimed in claim 1, wherein

said parallax image printed matter is a holographic stereogram, and

said present image processing apparatus further comprises an image conversion processing means for converting said parallax image data obtained by said motion compensation processing means to data that can be printed as said holographic stereogram.

3. The image processing apparatus as claimed in claim 2, wherein said image conversion processing means executes a viewing point conversion processing for said parallax image data.

4. The image processing apparatus as claimed in claim 1, wherein said parallax image printed matter is a printed matter produced by utilizing a lenticular method.

5. The image processing apparatus as claimed in claim 1, wherein said motion compensation processing means calculates said amount of movement of the point-of-regard by a correlation processing.

6. The image processing apparatus as claimed in claim 1, wherein said motion compensation processing means executes a translation motion processing for each frame on the basis of said motion compensation amount.

7. The image processing apparatus as claimed in claim 6, wherein said motion compensation processing means executes, in addition to said translation motion processing, at least one of a rotation processing and an expansion/reduction processing for each frame on the basis of said motion compensation amount.

8. The image processing apparatus as claimed in claim 1, wherein said motion compensation processing means executes a motion compensation processing with using said amount of movement of the point-of-regard in each frame as said motion compensation amount.

9. The image processing apparatus as claimed in claim 1, wherein said motion compensation processing means calculates said motion compensation amount by performing a functional conversion of said amount of movement of the point-of-regard in each frame.

10. The image processing apparatus as claimed in claim 6, wherein said motion compensation processing means executes, in addition to said translation motion processing, a clip processing for extracting a common region having image information for each frame.

11. The image processing apparatus as claimed in claim 10, wherein said motion compensation processing means executes, in addition to said clip processing, an expansion processing for expanding a size of the region extracted by said clip processing to an original frame size in each frame.

12. The image processing apparatus as claimed in claim 1, wherein said motion compensation processing means calculates said motion compensation amount by executing a smoothing processing or an approximated curve processing for said amount of movement of the point-of-regard in each frame.

13. The image processing apparatus as claimed in claim 1, further comprising a display means for displaying said parallax image data.

14. A method of image processing comprising:

a step of specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame; and

a step of calculating an amount of movement of the point-of-regard using a relation between said point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement calculated, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on a viewing angle.

15. An image processing apparatus comprising:

a point-of-regard setting means for specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a compensation amount calculation means for calculating an amount of movement of said point-of-regard using a relation between said point-of-regard in said reference frame and a point corresponding thereto in another frame, and calculating a motion compensation amount for each frame on the basis of said amount of movement calculated; and

an image conversion processing means for generating a parameter for use in an image conversion to which said motion compensation amount calculated by said motion compensation amount calculation means is added, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter, on the basis of said parameter.

16. A method of image processing comprising:

a step of specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a step of calculating an amount of movement of said point-of-regard using a relation between said point-of-regard in said reference frame and a point corresponding thereto in another frame, and calculating a motion compensation amount for each frame on the basis of said amount of movement of said point-of-regard; and

a step of generating a parameter for use in an image conversion, to which said motion compensation amount is added, and converting said motion image data to data that can be printed as a parallax image printed matter on the basis of said parameter.

17. A printed matter production apparatus comprising:

a point-of-regard setting means for specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a motion compensation processing means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard set in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement calculated, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter, in which a perceived image varies depending on its viewing angle; and

a printing means for printing a parallax image printed matter on the basis of said parallax image data.

18. The printed matter production apparatus as claimed in claim 17, wherein

said parallax image printed matter is a holographic stereogram,

said present image processing apparatus further comprises an image conversion processing means for converting said parallax image data obtained by said motion compensation processing means to data that can be printed as a holographic stereogram, and

said printing means prints out said holographic stereogram as said parallax image printed matter.

19. The printed matter production apparatus as claimed in claim 18, wherein said image conversion processing means executes a viewing point conversion processing for said parallax image data.

20. The printed matter production apparatus as claimed in claim 18, further comprising an imaging means for capturing said motion image data.

21. The printed matter production apparatus as claimed in claim 17, further comprising a display means for displaying said parallax image data.

22. A method of producing a printed matter comprising:

a step of specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a step of calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on its viewing angle; and

a step of printing a parallax image printed matter on the basis of said parallax image data.

23. The method of producing a printed matter as claimed in claim 22, wherein

said parallax image printed matter is a holographic stereogram,

the present method of producing a printed matter further includes a step of executing a viewing point conversion processing for converting said parallax image data to data that can be printed as a holographic stereogram, and

said holographic stereogram is printed-as said parallax image printed matter.

24. A printed matter production apparatus comprising:

a point-of-regard setting means for specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a motion compensation amount calculation means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard set in said reference frame and a point corresponding thereto in another frame, and calculating a motion compensation amount for each frame on the basis of said amount of movement thereof;

an image conversion processing means for generating a parameter relating to image conversion, to which said motion compensation amount calculated by said motion compensation calculation means is added, thereby converting said motion image data to data that can be printed as a parallax image printed matter on the basis of said parameter calculated; and

a printing means for printing a parallax image printed matter on the basis of said data converted.

25. A method of producing a printed matter comprising:

a step of specifying at least one frame as a reference frame from a plurality of frames that constitute motion image data, and setting a point-of-regard on an object in said reference frame;

a step of calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, and calculating a motion compensation amount on the basis of said amount of movement thereof;

a step of generating a parameter relating to image conversion, to which said motion compensation amount calculated is added, thereby converting said motion image data to data that can be printed as a parallax image printed matter on the basis of said parameter generated; and

a step of printing the parallax image printed matter on the basis of said data converted.

26. A printed matter production system comprising

a communication terminal device including:

a motion image input means for inputting motion image data;

a point-of-regard setting means for specifying at least one frame as a reference frame from a plurality of frames that constitute the motion image data, and setting a point-of-regard on an object in said reference frame; and

a first communication means for transmitting data relating to said motion image data and said point-of-regard, and

a printing device including;

a second communication means for receiving data from said first communication means;

a motion compensation processing means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement thereof, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on its viewing angle; and

a printing means for printing a parallax image printed matter on the basis of said parallax image data converted.

27. The printed matter production system as claimed in claim 26, wherein said communication terminal device includes an imaging means for said motion image data.

28. A printing device comprising:

a communication means for receiving data relating to motion image data, a reference frame in said motion image data and a point-of-regard set on an object in said reference frame from an external device;

a motion compensation processing means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement calculated, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on its viewing angle; and

a printing means for printing a parallax image printed matter on the basis of said parallax image data.

29. A printed matter production system comprising

a communication terminal device including:

a motion image input means for inputting motion image data;

a point-of-regard setting means for specifying at least one frame as a reference frame from a plurality of frames that constitute the motion image data, and setting a point-of-regard on an object in said reference frame;

a motion compensation processing means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement calculated, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on a viewing angle; and

a first communication means for transmitting the data relating to said motion image data, and

a printing device including;

a second communication means for receiving the data from said first communication means; and

a printing means for printing a parallax image printed matter on the basis of the data relating to said motion image data.

30. The printed matter production system as claimed in claim 29, wherein said communication terminal device includes an imaging device for capturing said motion image data.

31. The printed matter production system as claimed in claim 29, wherein said data relating to said motion image data is said parallax image data, or data relating to said motion image data and said point-of-regard, or data relating to said parallax image data, said motion image data and said point-of-regard.

32. A communication terminal device comprising:

a motion image input means for inputting motion image data;

a point-of-regard setting means for setting at least one frame as a reference frame in a plurality of frames that constitute the motion image data, and a point-of-regard on an object in said reference frame;

a motion compensation processing means for calculating an amount of movement of said point-of-regard using a relation between the point-of-regard in said reference frame and a point corresponding thereto in another frame, calculating a motion compensation amount for each frame on the basis of said amount of movement thereof, and executing a motion compensation processing for said motion image data on the basis of said motion compensation amount, thereby converting said motion image data to parallax image data that can be printed as a parallax image printed matter in which a perceived image varies depending on a viewing angle; and

a communication means for transmitting data relating to said motion image data.