Surround surveillance apparatus for mobile body

• US 7,295,229 B2
• Filed: 10/11/2001
• Issued: 11/13/2007
• Est. Priority Date: 10/11/2000
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Kumata, Kiyoshi, Kako, Noritoshi
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/975,678
• Publication Number:
  US 20020080017A1
• Time in Patent Office: 2,224 Days
• Field of Search: 219/203, 340/435, 340/436, 340/438, 340/903, 340/937, 340/995.13, 348142-170, 348 36- 39, 382/103, 382/102, 701/213, 73/290.1
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surround surveillance system mounted on a mobile body for surveying surroundings around the mobile body includes an omniazimuth visual system, the omniazimuth visual system includes at least one omniazimuth visual sensor including an optical system capable of obtaining an image with an omniazimuth view field area therearound and capable of central projection transformation of the image into an optical image, and an imaging section including an imaging lens for converting the optical image obtained by the optical system into image data, an image processor for transforming the image data into at least one of panoramic image data and perspective image data, a display section for displaying one of a panoramic image corresponding to the panoramic image data and a perspective image corresponding to the perspective image data and a display control section for controlling the display section.

Claims

1. A surround surveillance system mounted on a mobile body for surveying surroundings around the mobile body, comprising an omniazimuth visual system, the omniazimuth visual system including:

at least one omniazimuth visual sensor including an optical system capable of obtaining an image with an omniazimuth view field area therearound and capable of central projection transformation of the image into an optical image, and an imaging section including an imaging lens for converting the optical image obtained by the optical system into image data;

an image processor for transforming the image data into at least one of panoramic image data and perspective image data;

a display section for displaying one of a panoramic image corresponding to the panoramic image data and a perspective image which is panned or tilted corresponding to the perspective image data; and

a display control section for controlling the display section;

wherein the optical system includes a hyperboloidal mirror which has a shape of one sheet of a two-sheeted hyperboloid, an optical axis of the hyperboloidal mirror being identical with an optical axis of the imaging lens, and the principal point of the imaging lens being located at one of focal points of the hyperboloidal mirror, and

wherein the at least one omniazimuth visual sensor is stationary with respect to the mobile body, such that the perspective image, which is panned or tilted corresponding to the perspective image data, is obtained by transforming the image data obtained from the optical image taken by the at least one omniazimuth visual sensor.

2. A surround surveillance system according to claim 1, wherein the at least one omniazimuth visual sensor is located such that a bird'"'"'s-eye image of the mobile body and surroundings thereof is transformed into the image data.

3. A surround surveillance system according to claim 1, wherein the display section simultaneously or selectively displays the panoramic image and the perspective image.

4. A surround surveillance system according to claim 1, wherein the display section displays an image seen in a direction opposite to a moving direction of the mobile body.

5. A surround surveillance system according to claim 1, wherein the image processor transforms image data corresponding to a first area within the omniazimuth view field area around the optical system into first perspective image data.

6. A surround surveillance system according to claim 5, wherein in response to control by the display control section, the image processor transforms image data corresponding to a second area within the omniazimuth view field area around the optical system which does not overlap with the first area into a second perspective image data which does not coincide with the first perspective image data.

7. A surround surveillance system according to claim 6, wherein the second area is identical to an area which is obtained by performing a least one of translational transfer processing and zoom-in/zoom-out processing on the first area.

8. A surround surveillance system according to claim 1, wherein the optical system is positioned such that an optical axis of the optical system is perpendicular to a moving direction of the mobile body.

9. A surround surveillance system according to claim 1, wherein in response to control by the display control section, the display section displays an image showing the mobile body on a display screen of the display section such that the mobile body is shown at a predetermined position on a displayed image on the display screen.

10. A surround surveillance system according to claim 1, wherein the display section simultaneously displays an image seen in a direction opposite to a moving direction of the mobile body and an image seen in a direction which is not identical or opposite to the moving direction of the mobile body.

11. A surround surveillance system according to claim 1, wherein the mobile body is a vehicle.

12. A surround surveillance system according to claim 11, wherein:

the vehicle includes a first bumper provided at a moving direction side of the vehicle and a second bumper provided at a side of or the vehicle opposite to the moving direction side; and

the at least one omniazimuth visual sensor includes a first omniazimuth visual sensor placed on the first bumper and a second omniazimuth visual sensor placed on the second bumper.

13. A surround surveillance system according to claim 12, wherein:

the first omniazimuth visual sensor is placed on one of a right end and a left end of the first bumper with respect to the moving direction of the vehicle; and

the second omniazimuth visual sensor is placed on one end of the second bumper which is diagonal to the end of the first bumper where the first omniazimuth visual sensor is placed with respect to a body of the vehicle.

14. A surround surveillance system according to claim 13, wherein the display section displays an image obtained by combining a first perspective image derived from the first omniazimuth visual sensor and a second perspective image derived from the second omniazimuth visual sensor.

15. A surround surveillance system according to claim 1, wherein:

the image processor includes a storage section for storing mobile body image data;

the image processor combines the mobile body image data from the storage section with the perspective image data derived from the optical system; and

the display section displays based on the combined image data a perspective image including the image showing the mobile body.

16. A surround surveillance system according to claim 15, wherein the mobile body image data is image data created by using computer graphics software.

17. A surround surveillance system according to claim 15, wherein the mobile body image data is image data obtained by capturing an image of the mobile body.

18. A surround surveillance system according to claim 1, wherein:

the omniazimuth visual system further includes a temperature measurement section for measuring an environmental temperature of the mobile body;

when the environmental temperature measured by the temperature measurement section is equal to or lower than a predetermined temperature, the display section displays a perspective bird'"'"'s-eye image of the mobile body and surroundings thereof after the mobile body becomes movable.

19. A surround surveillance system according to claim 13, wherein, when the display section displays a perspective image of an overlapping region between a display region of a perspective bird'"'"'s-eye image of the mobile body and surroundings thereof which is obtained through the first omniazimuth visual sensor and a display region of a perspective bird'"'"'s-eye image of the mobile body and surroundings thereof which is obtained through the second omniazimuth visual sensor, the display section displays based on control by the display control section a perspective image derived from one of the first omniazimuth visual sensor and the second omniazimuth visual sensor.