Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to subsequent images

• US 5,850,254 A
• Filed: 06/01/1995
• Issued: 12/15/1998
• Est. Priority Date: 07/05/1994
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Monzi, Tatsuhiko, Ondoh, Eiryoh, Tanaka, Yasunari, Takano, Kazuaki, Shioya, Makoto
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US08/456,394
• Time in Patent Office: 1,293 Days
• Field of Search: 348/135, 348/136, 348/137, 348/140, 348/148, 348/149, 348/142, 348/118, 348/119, 348/143, 348/175, 348/176, 348/178, 348/179, 348/190, 348/208, 348/211, 340/903, 340/937, 340/435, 340/436, 180/167, 180/169, 180/270, 250/491.1, 382/103, 382/104
• US Class Current: 348/148
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/98 Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

An environment recognition device formed in such a manner that an image sensor is installed so that an image sensor mounting location recognition mark falls within an image pickup area, comparator for comparing mark locations the images of which are picked up sequentially by the image sensor with respect to the initial location of the recognition mark in the image pickup area is provided, and, when it is detected that the mark location recognized sequentially has changed with respect to the initial location, an indication signal for indicating that correction of the location of installing the image sensor is required is outputted, thereby to prevent erroneous decision in image processing when the mounting position changes by vibration, contact or the like.

Claims

1. An environment recognition device for use in a vehicle, said device comprising:

an image pickup device, mounted within an interior of said vehicle, for recognizing an environment confronted by said vehicle by signal processing of image pickup data;

a storage unit for storing a reference frame including an image of an initial location of a reference mark fixed to said vehicle and lying within a field of view of said image pickup device; and

a comparator for comparing the reference frame including the image of the initial location of said reference mark with at least one subsequent image of said image pickup device to check for deviation in aiming of said image pickup device.

2. A device according to claim 1, wherein said reference mark is one of a mark fixed at a predetermined location on a windshield of said vehicle and a hood ornament.

23. A device according to any one of claims 1, 3, 4, 5, 7, 10, 11, 12, 19, and 22, wherein said reference mark is one of a mark fixed at a predetermined location on a windshield of said vehicle and a hood ornament.

3. A vehicle comprising:

an image pickup device, mounted within an interior of said vehicle, for obtaining peripheral environmental information as image data of an environment confronted by said vehicle;

a reference mark fixed to said vehicle and lying within a field of view of said image pickup device;

a storage unit for storing a reference frame including an image of an initial location of said reference mark; and

a comparator for comparing said reference frame including the image of the initial location of said reference mark with at least one subsequent image of said image pickup device to check for deviation in aiming of said image pickup device.

4. An image pickup device for a vehicle for outputting environmental information inside and outside said vehicle as image data, said device comprising:

a storage unit for storing a reference frame including an image of an initial location of a reference mark fixed to said vehicle and lying within a field of view of said image pickup device; and

a comparator for comparing said reference frame including the image of the initial location of said reference mark with at least one subsequent image of said image pickup device to check for deviation in aiming of said image pickup device.

5. A method of recognizing an environment confronting a vehicle from image pickup data, by image processing of image pickup data picked up by an image pickup device fixed to said vehicle, said method comprising the steps of:

fixing a reference mark on said vehicles, and said reference mark lying within a field of view of said image pickup device;

storing, in a storage unit, a reference frame of an image of an initial location of said reference mark; and

comparing said reference frame including the image of the initial location of said reference mark with at least one subsequent image of said image pickup device to check for deviation in aiming of said image pickup device.

6. A method according to claim 5, wherein said reference mark is one of a hood ornament and a mark fixed to a predetermined location on a windshield of said vehicle.

7. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking up an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle based on image pickup data of said image pickup device;

warning means for alerting a driver when the predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided at a fixed position on said vehicle within an image pickup area of said image pickup device,

a memory for storing a reference frame including a reference image of said image pickup device, said reference image containing an initial location of said reference mark; and

a detecting apparatus for detecting a location of the reference mark within at least one subsequent image of said image pickup device, and

a comparator apparatus for comparing said reference frame including the reference image with said at least one subsequent image and for alerting a driver through said warning means when the reference mark location in said at least one subsequent image has changed with respect to the initial location of said reference mark in the reference image of the reference frame stored in said memory.

8. A device according to claim 7, wherein said memory for storing said initial location of said reference mark is a read-only memory.

9. A device according to claim 7, wherein said memory for storing said initial location of said reference mark is a random access memory having a back-up power source so that said memory is not erased even when an engine is stopped.

10. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking up an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle by using image pickup data of said image pickup device;

warning means for alerting a driver when the predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided at a fixed position on said vehicle within an image pickup area of said image pickup device;

a memory for storing a reference frame including a reference image of said image pickup device, said reference image containing an initial location of said reference mark; and

a detecting apparatus for detecting locations of the reference mark location within subsequent images of said image pickup device, said image pickup device continuing to transmit an image signal to said image processing apparatus until said detecting apparatus detects a deviation of one of said reference mark locations in said subsequent images from the initial location of said reference mark in the reference image of said reference frame.

11. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking up an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle by using image pickup data of said image pickup device;

warning means for alerting a driver when the predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided at a fixed position on said vehicle within an image pickup area of said image pickup device;

a memory for storing a reference frame including a reference image of said image pickup device, said reference image containing an initial location of said reference mark; and

a detecting apparatus for detecting a location of the reference mark within at least one subsequent image of said image pickup device, and for suspending an image signal transmission from said image pickup device to said image processing apparatus when the reference mark location in said at least one subseruent-image has changed with respect to the initial location of said reference mark in the reference image of said reference frame.

12. An environment recognition device for a vehicle comprising:

posture change sensors for sensing a posture change of said vehicle;

an image pickup device installed in said vehicle for picking up an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state resenting a danger to said vehicle using image data of said image pickup device;

alerting means for alerting a driver when the predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided at a fixed position on said vehicle within an image pickup area of aid image pickup device;

a storage device for storing a reference frame including an initial image including said reference mark;

a comparator for comparing the reference frame including the initial image with subsequent images of said image pickup device to check for deviation in aiming of said image pickup device; and

a suspending apparatus for suspending an image signal transmission from said image pickup device to said image processing apparatus when a posture change of said vehicle is sensed by said posture change sensors.

13. A device according to claim 12, wherein said posture change of said vehicle is sensed by outputs of at least one height sensor provided on a suspension.

14. A device according to claim 12, wherein said posture change of said vehicle is sensed by an output of an accelerometer fitted on a vehicle body.

15. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking up an image of a fixed portion of said vehicle and an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle by using image pickup data of said image pickup device, and

warning means for alerting a driver when the predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided fixed to said fixed portion of said vehicle within an image pickup area of said image pickup device;

a memory for storing a reference frame including a reference image containing an initial location of said reference mark on said fixed portion of said vehicle; and

a detecting apparatus for detecting a reference mark location and a fixed portion of said vehicle within at least one subsequent image of said image pickup device, or comparing the location of said reference mark and the fixed portion of the vehicle in said at least subsequent image with the initial location of said reference mark in the reference image of said reference frame, and for alerting a driver through said warning means when the location of said reference mark location and the fixed portion of the vehicle in said at least one subsequent image has changed with respect to die initial location of said reference mark in the reference image of said reference frame.

16. A device according to claim 15, wherein said fixed portion of said vehicle is more specifically a hood in front of a windshield.

17. A device according to claim 15, wherein said reference mark is more specifically a mark stuck on a windshield.

18. A device according to claim 15, wherein said fixed portion of said vehicle is more specifically a trunk hood in a rear windshield.

19. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking up an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle by using image pickup data of said image pickup device;

warning means for alerting a driver when said predetermined dangerous state is recognized by said image processing apparatus;

a reference mark provided fixed to said vehicle within an image pickup area of said image pickup device, said reference mark serving as a basis for checking deviation in aiming of said bray pickup device;

a memory for storing a reference frame including a reference image of said image pickup device, said reference image containing an initial location of said reference mark; and

an aiming apparatus for detecting a location of the reference mark within at least one subsequent image of said image pickup device, for comparing the location of the reference mark within said at least one subsequent image with the initial location of said reference mark in the reference image of said reference frame, and when said reference mark location in said at least one subsequent image his changed with respect to the initial location of s id reference mark in the reference image of said reference frame, for automatically adjusting an aiming direction of said image pickup device so that said reference mark location in sail at least one subsequent image and images thereafter coincide with the in trial location of said reference mark in the reference image of said reference frame.

20. A device according to claim 19, wherein the adjusting of aiming direction of the image pickup device is made by a motor.

21. A device according to claim 19, wherein the adjusting of aiming direction of the image pickup device is made by a change in an image pickup system composed of a lens and a charge coupled device element.

22. An environment recognition device for a vehicle comprising:

an image pickup apparatus having an image pickup device installed in said vehicle for picking an image of an exterior environment confronting said vehicle;

an image processing apparatus for deciding existence of a predetermined dangerous state representing a danger to said vehicle by using image pickup data of said image pickup device;

warning means for alerting a driver, when said predetermined dangerous state is recognized by said image processing means;

a reference mart provided fixed to said vehicle within an image pickup area of said image pickup device;

a memory for storing a reference frame including a reference image of said image pickup device, said reference image containing an initial location of said reference mark; and

an aiming apparatus for detecting a location of the reference mark within at least one subsequent page of said image pickup device, for comparing the location of the reference mark within said at least one subsequent image with the initial location of said reference mark in the reference image of said reference frame, and when said reference mark location n in said at least one subsequent image has changed with respect to the initial location of aid reference mark in the reference image of said reference frame, for correcting image data taken into said image processing apparatus in accordance with a quantity and a direction of the change.

24. An environment recognition device for a vehicle, comprising:

an image pickup device, mounted within said vehicle, for recognizing an environment confronted by said vehicle by signal processing of image pickup data;

a storage unit for storing a reference frame including a reference, image indicative of an initial location of a reference mark fixed to said vehicle and lying within field of view of said image pickup device; and

a comparator for comparing said reference frame including the reference image indicative of the initial location of said reference mark with at least one subsequent image from said image pickup device to check for deviation in aiming of said image pickup device.

25. A device according to claim 24, wherein said reference mark is one of a mark fixed at a predetermined location on a windshield of said vehicle and a hood ornament.