IMAGE PROCESSING METHOD AND APPARATUS

• US 20130083221A1
• Filed: 11/27/2012
• Published: 04/04/2013
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: HORII, Hirofumi
• Granted Patent: US 8,767,096 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

An AR taking image producer produces an AR taking image of a current frame. A displaying image producer produces a displaying image of the current frame and first to fourth interpolative displaying images. An image pattern reader reads a marker on the AR taking image. CPU retrieves AR moving image that is associated with the marker from an AR image memory. An AR image overlying section produces first to fourth interpolative AR images, and overlies the images on the areas of the marker.

Claims

1. An image processing apparatus comprising:

an imaging section for capturing images in continuous succession by imaging a subject together with a marker, the captured images constituting moving images;

a displaying image producing section for producing a displaying image on the basis of the captured image each time the image is captured, the displaying image being for making the motion of the subject continuous in displaying the moving images;

an additional information reference image producing section for producing an additional information reference image on the basis of the captured image each time the image is captured, the additional information reference image representing the subject with less blurs and being used to acquire additional information that is to be overlaid on the displaying image;

a reading section for reading the marker on the additional information reference image;

an additional information recording section having additional information recorded therein in association with individual markers;

an additional information acquiring section for retrieving and acquiring additional information that is associated with the marker read by the reading section from among a plurality of additional information recorded in the additional information recording section;

an additional information overlaid image producing section for producing an additional information overlaid image which has the additional information, acquired by the additional information acquiring section, overlaid on the displaying image in the area of the marker; and

a displaying device for displaying the additional information overlaid image.

2. The image processing apparatus claimed in claim 1, wherein the imaging section captures images at a shutter speed suitable for producing the additional information reference image, and the displaying image producing section also produces a plurality of interpolative displaying images which are to be displayed in between successive ones of the displaying images.

3. The image processing apparatus claimed in claim 1, wherein

the imaging section makes imaging alternately at a first shutter speed that is suitable for producing the displaying image and a second shutter speed that is higher than the first shutter speed and suitable for producing the additional information reference image;

the displaying image producing section produces the displaying image on the basis of an image taken through the first shutter speed imaging; and

the additional information reference image producing section produces the additional information reference image on the basis of an image taken through the second shutter speed imaging.

4. The image processing apparatus claimed in claim 1, wherein

the imaging section makes imaging concurrently at a first shutter speed that is suitable for producing the displaying image and a second shutter speed that is higher than the first shutter speed and suitable for producing the additional information reference image;

the displaying image producing section produces the displaying image on the basis of an image taken through the first shutter speed imaging; and

the additional information reference image producing section produces the additional information reference image on the basis of an image taken through the second shutter speed imaging.

5. The image processing apparatus claimed in claim 3, comprising a position correcting section for correcting the position of overlying the additional information on the displaying image on the basis of a difference in time of production between the displaying image and the additional information reference image.

6. The image processing apparatus claimed in claim 1, wherein

the displaying image producing section produces the displaying image to have a high resolution and such color reproduction that is suitable for displaying moving images, and

the additional information reference image producing section produces the additional information reference image to have a higher SN ratio and a wider dynamic range than the displaying image.

7. The image processing apparatus claimed in claim 2, wherein

the displaying image producing section produces the displaying image to have a high resolution and such color reproduction that is suitable for displaying moving images, and

the additional information reference image producing section produces the additional information reference image to have a higher SN ratio and a wider dynamic range than the displaying image.

8. The image processing apparatus claimed in claim 1, comprising:

a detecting section for detecting the speed of movement of the imaging section; and

a determining section for determining whether the speed of movement detected by the detecting section is more than a predetermined threshold level or not;

wherein,

when the determining section determines that the speed of movement is not more than the threshold level, the additional information reference image producing section does not produce the additional information reference image and the reading section reads the marker in the displaying image.

9. The image processing apparatus claimed in claim 2, comprising:

a detecting section for detecting the speed of movement of the imaging section; and

a determining section for determining whether the speed of movement detected by the detecting section is more than a predetermined threshold level or not;

wherein,

when the determining section determines that the speed of movement is not more than the threshold level, the additional information reference image producing section does not produce the additional information reference image and the reading section reads the marker in the displaying image.

10. The image processing apparatus claimed in claim 1, comprising:

a marker motion speed detecting section for detecting the motion speed of the marker; and

a marker motion speed determining section for determining whether the motion speed detected by the marker motion speed detecting section is more than a predetermined threshold level or not;

wherein,

when the marker motion speed determining section determines that the motion speed is not more than the threshold level, the additional information reference image producing section does not produce the additional information reference image and the reading section reads the marker in the displaying image.

11. The image processing apparatus claimed in claim 2, comprising:

a marker motion speed detecting section for detecting the motion speed of the marker; and

a marker motion speed determining section for determining whether the motion speed detected by the marker motion speed detecting section is more than a predetermined threshold level or not;

wherein,

when the marker motion speed determining section determines that the motion speed is not more than the threshold level, the additional information reference image producing section does not produce the additional information reference image and the reading section reads the marker in the displaying image.

12. The image processing apparatus claimed in claim 1, comprising an additional information overlaid image recording section for recording the additional information overlaid image.

13. The image processing apparatus claimed in claim 2, comprising an additional information overlaid image recording section for recording the additional information overlaid image.

14. An image processing method comprising:

an imaging step for capturing images in continuous succession by imaging a subject together with a marker, the captured images constituting moving images;

a displaying image producing step for producing a displaying image on the basis of the captured image each time the image is captured, the displaying image being for making the motion of the subject continuous in displaying the moving images;

an additional information reference image producing step for producing an additional information reference image on the basis of the captured image each time the image is captured, the additional information reference image representing the subject with less blurs and being used to acquire additional information that is to be overlaid on the displaying image;

a reading step for reading the marker on the additional information reference image;

an additional information acquiring step for retrieving and acquiring additional information that is associated with the marker read by the reading step from an additional information recording section in which additional information is recorded in association with individual markers;

an additional information overlaid image producing step for producing an additional information overlaid image which has the additional information, acquired by the additional information acquiring step, overlaid on the displaying image in the area of the marker; and

a displaying step for displaying the additional information overlaid image.

15. An image processing method claimed in claim 14, wherein the displaying image producing step produces a plurality of interpolative displaying images which are to be displayed in between two of the captured displaying images.

16. An image processing method claimed in claim 15, wherein the imaging step makes imaging at a first shutter speed to produce the displaying image and at a second shutter speed that is higher than the first shutter speed to produce the additional information reference image.