Image processing device and image capturing device

• US 20060164526A1
• Filed: 03/14/2006
• Published: 07/27/2006
• Est. Priority Date: 09/18/2003
• Status: Active Grant

• Current Assignee: Brother Kogyo Kabushiki Kaisha (Brother Industries Limited)
• Original Assignee: Brother Kogyo Kabushiki Kaisha (Brother Industries Limited)
• Inventors: Suzuki, Hiroaki, Iwasaki, Takeo
• Granted Patent: US 7,627,196 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters

Abstract

An image processing device comprises: plane conversion means which receives a subject image in which a subject having a particular shape has been captured and shape information about three-dimensional shape of the subject from the outside and converts the subject image into a plane image in which the subject is viewed from a prescribed direction based on the shape information; and in-plane position detection means which recognizes a subject area in the plane image as an area representing the subject and determines a position of the subject in the plane image.

Claims

1. An image processing device comprising:

a plane conversion unit which receives a subject image in which a subject having a particular shape has been captured and shape information about three-dimensional shape of the subject and converts the subject image into a plane image in which the subject is viewed from a prescribed direction based on the shape information; and

an in-plane position detection unit which recognizes a subject area in the plane image as an area representing the subject and determines a position of the subject in the plane image.

2. The image processing device according to claim 1, further comprising a first correction conversion unit which converts the subject image into a corrected image in which the subject is viewed from a prescribed direction and placed substantially at the center based on the shape information and the position of the subject determined by the in-plane position detection unit.

3. The image processing device according to claim 2, further comprising an in-plane angle detection unit which recognizes the subject area in the plane image and determines an angle of a prescribed side of the subject area with respect to a prescribed direction in the plane image.

4. The image processing device according to claim 3, wherein the first correction conversion unit converts the subject image into a corrected image in which the subject is viewed from a prescribed direction and placed substantially at the center and a prescribed side of the subject is oriented in a prescribed direction in the image based on the shape information, the position of the subject determined by the in-plane position detection unit, and the angle determined by the in-plane angle detection unit.

5. The image processing device according to claim 2, wherein the first correction conversion unit executes the conversion into the corrected image only for an area in the subject image that represents the subject.

6. The image processing device according to claim 2, wherein:

the subject image is a multicolor image, and

the first correction conversion unit makes the conversion into the corrected image that has the same number of colors as pixels of the subject image.

7. The image processing device according to claim 2, further comprising an image extraction unit which extracts an image by cutting out a part of the corrected image representing the subject in a prescribed shape.

13. The image processing device according to claim 1, further comprising a range input unit that inputs a range specifying an area in the subject image to be converted by the plane conversion unit, wherein:

the plane conversion unit converts the area in the subject image specified by the range inputted via the range input unit into the plane image.

14. The image processing device according claim 1, wherein the subject is in a rectangular shape when its particular surface is viewed from an orthogonal direction.

15. The image processing device according to claim 1, wherein the plane conversion unit makes the conversion into the plane image that has a lower resolution than the subject image.

16. The image processing device according to claim 1, wherein:

the subject image is a multicolor image, and

the plane conversion unit makes the conversion into the plane image that is made of monochrome pixels.

17. An image capturing device comprising:

the image processing device according to claim 1;

a projector which projects pattern light;

an image capturing unit which obtains the subject image by shooting the subject; and

a three-dimensional shape detection unit which figures out shape information about three-dimensional shape of the subject based on an image obtained by letting the image capturing unit shoot the subject on which the pattern light is projected, wherein;

the subject image obtained by the image capturing unit and the shape information figured out by the three-dimensional shape detection unit are inputted to the image processing device.

18. An image capturing device comprising:

the image processing device according to claim 2;

a projector which projects pattern light;

an image capturing unit which obtains the subject image by shooting the subject and inputs the subject image to the image processing device;

a three-dimensional shape detection unit which figures out shape information about three-dimensional shape of the subject based on an image obtained by letting the image capturing unit shoot the subject on which the pattern light is projected and inputs the shape information to the image processing device;

a display unit which displays the corrected image obtained by the first correction conversion unit;

an input unit that inputs a display status modification parameter for modifying display status of the corrected image displayed on the display unit; and

a modification unit which modifies the display status of the corrected image displayed on the display unit based on the display status modification parameter inputted through the input unit.

8. An image processing device comprising:

a plane conversion unit which receives a subject image in which a subject having a particular shape has been captured and shape information about three-dimensional shape of the subject and converts the subject image into a plane image in which the subject is viewed from a prescribed direction based on the shape information; and

an in-plane angle detection unit which recognizes a subject area in the plane image as an area representing the subject and determines an angle between a prescribed side of the subject area and a prescribed direction in the plane image.

9. The image processing device according to claim 8, further comprising a second correction conversion unit which converts the subject image into a corrected image in which the subject is viewed from a prescribed direction and a prescribed side of the subject is oriented in a prescribed direction in the image based on the shape information and the angle determined by the in-plane angle detection unit.

10. The image processing device according to claim 9, wherein the second correction conversion unit executes the conversion into the corrected image only for an area in the subject image that represents the subject.

11. The image processing device according to claim 9, wherein:

the subject image is a multicolor image, and

the second correction conversion unit makes the conversion into the corrected image that has the same number of colors as pixels of the subject image.

12. The image processing device according to claim 9, further comprising an image extraction unit which extracts an image by cutting out a part of the corrected image representing the subject in a prescribed shape.

19. An image capturing device comprising:

the image processing device according to claim 9;

a projector which projects pattern light;

an image capturing unit which obtains the subject image by shooting the subject and inputs the subject image to the image processing device;

a three-dimensional shape detection unit which figures out shape information about three-dimensional shape of the subject based on an image obtained by letting the image capturing unit shoot the subject on which the pattern light is projected and inputs the shape information to the image processing device;

a display unit which displays the corrected image obtained by the second correction conversion unit;

an input unit that inputs a display status modification parameter for modifying display status of the corrected image displayed on the display unit; and

a modification unit which modifies the display status of the corrected image displayed on the display unit based on the display status modification parameter inputted through the input unit.

20. An image capturing device comprising:

a projector which projects pattern light;

an image capturing unit which obtains a captured image of a subject by shooting the subject;

a three-dimensional shape calculation unit which figures out shape information about three-dimensional shape of the subject based on an image obtained by letting the image capturing unit shoot the subject on which the pattern light is projected;

a lane conversion unit which converts the captured image of the subject into a plane image in which the subject is viewed from a prescribed direction based on the shape information figured out by the three-dimensional shape calculation unit;

a display unit which displays the plane image converted by the plane conversion unit;

an input unit that inputs a display status modification parameter for modifying display status of the plane image displayed on the display unit; and

a first modification unit which modifies the display status of the plane image displayed on the display unit based on the display status modification parameter inputted through the input unit.

21. The image capturing device according to claim 20, wherein the input unit includes an operation unit allowing the inputting of at least one of movement and magnification for the plane image displayed on the display unit as the display status modification parameter.

22. The image capturing device according to claim 20, wherein:

the input unit includes a detector which detects movement of a body of the image capturing device in three-dimensional space as the display status modification parameter, and

the first modification unit modifies the display status of the plane image by associating the movement of the body of the image capturing device in the three-dimensional space detected by the detector with at least one of movement and magnification for the plane image displayed on the display unit.

23. The image capturing device according to claim 20, wherein the plane conversion unit converts the plane image into an image having a lower resolution than the captured image.

24. The image capturing device according to claim 20, further comprising a storage unit which stores an image modified by the first modification unit, wherein:

the image is stored in the storage unit in a higher resolution than the plane image.

25. The image capturing device according to claim 20, further comprising:

a display status modification parameter storage unit which stores the display status modification parameter used for modifying the display status of the plane image displayed on the display unit; and

a selection unit that selects a display status parameter to be used for modifying the display status of a different plane image displayed on the display unit from the display status modification parameters stored in the display status modification parameter storage unit, wherein;

the first modification unit modifies the display status of the different plane image displayed on the display unit based on the display status modification parameter selected through the selection unit.

26. The image capturing device according to claim 20, further comprising:

an in-plane position detection unit which recognizes a subject area representing the subject from the plane image and determines a position of the subject in the plane image before the plane image is displayed on the display unit; and

a second modification unit which modifies the plane image into an image in which the subject viewed from a prescribed direction is placed substantially at the center of the image based on the position determined by the in-plane position detection unit and the shape information, wherein;

the plane image displayed on the display unit is the image modified by the second modification unit.

27. The image capturing device according to claim 26, further comprising an in-plane angle detection unit which recognizes a subject area representing the subject from the plane image and determines an angle of a prescribed side of the subject area with respect to a prescribed direction in the plane image before the plane image is displayed on the display unit, wherein:

the second modification unit modifies the plane image into an image in which the subject viewed from a prescribed direction is placed substantially at the center and a prescribed side of the subject is oriented in a prescribed direction in the image based on the angle determined by the in-plane angle detection unit, the position determined by the in-plane position detection unit and the shape information.