Imaging apparatus and method of recognizing target object

• US 10,091,408 B2
• Filed: 04/14/2017
• Issued: 10/02/2018
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Nonaka, Shunichiro
• Primary Examiner(s): Spinks, Antoinette
• Application Number:
  US15/487,441
• Publication Number:
  US 20170280045A1
• Time in Patent Office: 536 Days
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G03B 15/0431 Rotatable mounting means
• CPC Class Codes: G03B 15/0436 Mounting means for sequential translation
• CPC Class Codes: G03B 17/56 Accessories carrying-cases A45C11/38
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

The imaging apparatus includes: an imaging optical system that is formed of a wide-angle optical system and a telephoto optical system having a common optical axis; a directional sensor; a panning/tilting mechanism that rotates an imaging section, which includes the imaging optical system and the directional sensor, in horizontal and vertical directions; and an image acquisition section that respectively acquires a wide-angle image, and a telephoto image, and a first target object is detected from a wide-angle image, the panning/tilting mechanism is controlled on the basis of information about a position of the detected first target object within the wide-angle image, and image recognition is performed on a telephoto image, when the first target object is positioned at the center of the wide-angle image, thereby recognizing the first target object or a second target object.

Claims

1. An imaging apparatus comprising:

an imaging optical system that is formed of a wide-angle optical system and a telephoto optical system having a common optical axis;

a directional sensor that has a plurality of pixels constituted of photoelectric conversion elements two-dimensionally arranged and that respectively pupil-divides rays incident through the wide-angle and telephoto optical systems so as to selectively receive the rays;

a panning/tilting mechanism that rotates an imaging section, which includes the imaging optical system and the directional sensor, in horizontal and vertical directions;

an image acquisition section that respectively acquires a wide-angle image, for which light is received from the directional sensor through the wide-angle optical system, and a telephoto image for which light is received from the directional sensor through the telephoto optical system;

a target object detection section that detects a first target object from the wide-angle image by performing image processing on the wide-angle image acquired by the image acquisition section,

wherein the target object detection section includes a moving object detection section that performs image recognition of a moving object on the basis of the wide-angle images continuously acquired by the image acquisition section, and that detects the image recognized moving object as the first target object;

a panning/tilting control section that controls the panning/tilting mechanism on the basis of information about a position of the first target object, which is detected by the target object detection section, within the wide-angle image such that the first target object is positioned at the center of the wide-angle image; and

an image holding section that holds the telephoto image for a predetermined time period, if the moving object detection section detects the first target object as a moving object.

2. The imaging apparatus according to claim 1, further comprising:

an image recognition section that performs image recognition on the telephoto image, which is acquired by the image acquisition section, when the panning/tilting control section controls the panning/tilting mechanism such that the first target object is positioned at the center of the wide-angle image, and recognizes the first target object or a second target object close to the first target object; and

wherein in a case where there is a plurality of the first target objects, the image recognition section preferentially recognizes the first target object closest to the imaging apparatus.

3. The imaging apparatus according to claim 1, wherein if a plurality of target objects is detected from the wide-angle image, the target object detection section detects a target object closest to the imaging apparatus as the first target object.

4. The imaging apparatus according to claim 2, further comprising

wherein the target object detection section includes a moving object detection section that performs image recognition of a moving object on the basis of the wide-angle images continuously acquired by the image acquisition section, and that detects the image recognized moving object as the first target object,

wherein if the target object detection section detects a plurality of target objects and the moving object detection section detects a moving object, the image recognition section preferentially recognizes the detected moving object.

5. The imaging apparatus according to claim 2, wherein the image recognition section extracts a face image of a person from the telephoto image, and performs face recognition for specifying an individual on the basis of the extracted face image.

6. The imaging apparatus according to claim 3, wherein the image recognition section extracts a face image of a person from the telephoto image, and performs face recognition for specifying an individual on the basis of the extracted face image.

7. The imaging apparatus according to claim 2, wherein the image recognition section extracts an image of a license plate of a vehicle from the telephoto image, and performs image recognition for specifying a number of the vehicle on the basis of the extracted image of the license plate.

8. The imaging apparatus according to claim 3, wherein the image recognition section extracts an image of a license plate of a vehicle from the telephoto image, and performs image recognition for specifying a number of the vehicle on the basis of the extracted image of the license plate.

9. The imaging apparatus according to claim 2, further comprising an output section that outputs recognition information of the first target object or the second target object recognized by the image recognition section.

10. The imaging apparatus according to claim 9,

wherein the image recognition section extracts a face image of a person from the telephoto image, or extracts an image of a license plate of a vehicle from the telephoto image, and

wherein if the image recognition section extracts the face image of the person from the telephoto image, or extracts the image of the license plate of the vehicle from the telephoto image, the output section trims the face image or the image of the license plate extracted from the telephoto image, and outputs only the trimmed image to a display section.

11. The imaging apparatus according to claim 9, wherein the output section outputs the recognition information to at least one of an information recording section, which records the recognition information, or a display section which displays the recognition information.

12. The imaging apparatus according to claim 1, further comprising an image recording section that records at least the telephoto image of the wide-angle and telephoto images acquired by the image acquisition section.

13. The imaging apparatus according to claim 1, wherein the target object detection section performs image recognition on the first target object on the basis of the wide-angle image acquired by the image acquisition section, and detects the first target object.

14. The imaging apparatus according to claim 1,

wherein the target object detection section performs image processing on the wide-angle images continuously acquired by the image acquisition section in real time, and continuously detects the first target object from the wide-angle images, and

wherein the panning/tilting control section continuously controls the panning/tilting mechanism on the basis of information about the position of the first target object, which is continuously detected by the target object detection section, in the wide-angle image such that the first target object is kept at the center of the wide-angle image.

15. The imaging apparatus according to claim 1, wherein the wide-angle optical system is a central optical system that has a circular shape, and the telephoto optical system is an annular optical system that is disposed concentrically around the central optical system.

16. The imaging apparatus according to claim 13, wherein the annular optical system has a reflective optical system that reflects rays twice or more.

17. The imaging apparatus according to claim 1, wherein the telephoto optical system is a central optical system that has a circular shape, and the wide-angle optical system is an annular optical system that is disposed concentrically around the central optical system.

18. The imaging apparatus according to claim 1, further comprising a focus adjustment section that performs focus adjustment of the telephoto optical system.

19. The imaging apparatus according to claim 1, wherein the directional sensor has a micro lens array or a light blocking mask that functions as pupil division means.

20. A method of recognizing a target object by using an imaging apparatus including an imaging optical system that is formed of a wide-angle optical system and a telephoto optical system having a common optical axis, a directional sensor that has a plurality of pixels constituted of photoelectric conversion elements two-dimensionally arranged and that respectively pupil-divides rays incident through the wide-angle and telephoto optical systems so as to selectively receive the rays, a panning/tilting mechanism that rotates an imaging section, which includes the imaging optical system and the directional sensor, in horizontal and vertical directions, and an image acquisition section that respectively acquires a wide-angle image, for which light is received from the directional sensor through the wide-angle optical system, and a telephoto image for which light is received from the directional sensor through the telephoto optical system, the method comprising:

detecting a first target object from the wide-angle image by performing image processing on the wide-angle image acquired by the image acquisition section;

controlling the panning/tilting mechanism on the basis of information about a position of the detected first target object within the wide-angle image such that the first target object is positioned at the center of the wide-angle image;

performing image recognition of a moving object, on the basis of the wide-angle images continuously acquired by the image acquisition section, and detecting the image recognized moving object as the first target object; and

holding the telephoto image for a predetermined time period, if the first target object is detected as a moving object.