Method and apparatus for self-diagnostics of a vision system

• US 7,003,384 B2
• Filed: 10/24/2003
• Issued: 02/21/2006
• Est. Priority Date: 10/24/2003
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Luo, Yun, Dziadula, Robert, Wallace, Jon K., Khairallah, Farid, Lynch, Russell J.
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US10/692,949
• Publication Number:
  US 20050090957A1
• Time in Patent Office: 851 Days
• Field of Search: 701/45, 701/36, 701/200, 701/207, 348/148, 348/118, 348/113, 348/135, 348/139, 348/142
• US Class Current: 701/36
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method and apparatus (20) for recognizing and determining a location of an object (40) in an area of interest (31) comprises an imaging device (61) for obtaining an image of the area of interest. At least three identifying marks (104) are located at predetermined positions in the area of interest (31). The apparatus (20) also comprises a memory (78) in which the predetermined positions of the at least three identifying marks (104) are stored. A controller (80) determines whether the imaging device (61) is calibrated properly by analyzing the image obtained by the imaging device to determine whether positions of the at least three identifying marks (104) in the obtained image correspond to the predetermined positions stored in the memory (78). The controller (80) is also adapted to determine whether a view of the area of interest (31) from the imaging device (61) is obstructed.

Claims

1. An apparatus for determining a location of an object in an area of interest within a vehicle, the apparatus comprising:

an imaging device fixed relative to the vehicle for obtaining an image of the area of interest;

at least three identifying marks located at predetermined positions relative to the imaging device and in the area of interest within the vehicle;

a memory in which the predetermined positions of the at least three identifying marks are stored; and

a controller for determining whether the imaging device is calibrated properly by analyzing the image obtained by the imaging device to determine whether positions of the at least three identifying marks in the obtained image correspond to the predetermined positions stored in the memory, the controller determining a location of the object in the area of interest when the imaging device is calibrated properly.

2. The apparatus of claim 1 wherein the imaging device includes first and second cameras spaced apart from one another.

3. The apparatus of claim 2 wherein each of the first and second cameras is adapted to take a two-dimensional image of the area of interest, the controller forming a three-dimensional image from the two-dimensional images.

4. The apparatus of claim 3 wherein the controller analyzes exposure of at least one of the two-dimensional images taken by the first and second cameras, the controller determining that a view of the area of interest from the imaging device is obstructed when the analyzed exposure is outside of a predetermined exposure range, the controller, in response to determining that the view is obstructed, actuating an indicator for indicating to the object that the imaging device is obstructed.

5. The apparatus of claim 3 wherein the controller determines a distance to the object from disparity of the two-dimensional images, the controller determining that a view of the area of interest from the imaging device is obstructed when the determined distance is outside of a predetermined distance range, the controller, in response to determining that the view is obstructed, actuating an indicator for indicating to the object that the imaging device is obstructed.

6. The apparatus of claim 3 wherein the controller examines the three-dimensional image for blobs, the controller determining that a view of the area of interest from the imaging device is obstructed when a blob of the three-dimensional image has a size that is outside of a predetermined blob size range, the controller, in response to determining that the view is obstructed, actuating an indicator for indicating to the object that the imaging device is obstructed.

7. The apparatus of claim 1 wherein the at least three identifying marks are formed on interior structures of a vehicle and the area of interest is a passenger compartment of a vehicle.

8. The apparatus of claim 7 further including an inflatable air bag, inflation of the air bag being controlled in response to the location of the object in the passenger compartment.

9. The apparatus of claim 7 wherein the at least three identifying marks are formed from dye and the first and second cameras are wide spectrum cameras configured to obtain images including the at least three identifying marks.

10. The apparatus of claim 9 further including a near-infrared illuminator for illuminating the passenger compartment of the vehicle.

11. The apparatus of claim 2 wherein the at least three identifying marks are formed from dye and the first and second cameras are wide spectrum cameras configured to obtain images including the at least three identifying marks.

12. The apparatus of claim 11 further including a near-infrared illuminator for illuminating the area of interest.

13. A method of self-diagnostics of an apparatus for determining a location of an object in an area of interest within a vehicle, the method comprising the steps of:

obtaining an image of the area of interest using an imaging device that is fixed relative to the vehicle;

locating at least three identifying marks at predetermined positions relative to the imaging device and in the area of interest within the vehicle;

storing in a memory the predetermined positions of the at least three identifying marks;

determining whether the imaging device is calibrated properly by analyzing the obtained images to determine whether positions of the at least three identifying marks in the obtained images correspond to the predetermined positions stored in the memory; and

determining a location of the object in the area of interest when the imaging device is calibrated properly.

14. The method of claim 13 wherein the step of obtaining an image of the area of interest further includes the steps of:

taking first and second two-dimensional images of the area of interest; and

forming a three-dimensional image of the area of interest from the first and second two-dimensional images.

15. The method of claim 14 further including the steps of:

analyzing exposure of at least one of the first and second two-dimensional images;

determining that a view of the area of interest from the imaging device is obstructed when the analyzed exposure is outside of a predetermined exposure range; and

providing an indication when the view is obstructed.

16. The method of claim 14 further including the steps of:

determining distance to the object using disparity of the first and second two-dimensional images;

determining that a view of the area of interest from the imaging device is obstructed when the determined distance is outside of a predetermined distance range; and

providing an indication when the view is obstructed.

17. The method of claim 14 further including the steps of:

examining the three-dimensional image for blobs;

determining that a view of the area of interest from the imaging device is obstructed when a blob of the three-dimensional image has a size that is outside of a predetermined blob size range; and

providing an indication to the object when the view is obstructed.

18. The method of claim 13 further including the steps of:

determining whether a view of the area of interest from the imaging device is obstructed;

determining the location of the object in the area of interest when the imaging device is calibrated properly and the view is unobstructed; and

controlling inflation of an inflatable air bag in response to the determined location of the object in the area of interest.

19. The method of claim 13 further including the steps of:

forming the at least three identifying marks from near-infrared dye; and

forming the imaging device from first and second near-infrared cameras that are configured to obtain images including the at least three identifying marks.

20. The method of claim 19 further including the step of:

illuminating the area of interest with near-infrared light.

21. An apparatus for determining a location of an object in an area of interest, the apparatus comprising:

an imaging device for obtaining an image of the area of interest;

a controller for analyzing a feature of the obtained image to determine whether a view of the area of interest from the imaging device is obstructed; and

an indicator that is actuatable for indicating that the view is obstructed, the controller controlling actuation of the indicator,

the controller analyzing the obtained image to determine the location of the object when the view is unobstructed.

22. The apparatus of claim 21 wherein the imaging device includes first and second cameras spaced apart from one another.

23. The apparatus of claim 22 wherein each of the first and second cameras is adapted to take a two-dimensional image of the area of interest, the controller forming a three-dimensional image from the two-dimensional images taken by the first and second cameras.

24. The apparatus of claim 23 wherein the feature analyzed by the controller is exposure of at least one of the two-dimensional images taken by the first and second cameras, the controller determining that the imaging device is obstructed when the analyzed exposure is outside of a predetermined exposure range.

25. The apparatus of claim 23 wherein the feature analyzed by the controller is a distance to the object, the distance to the object being determined using disparity of the two-dimensional images taken by the first and second cameras, the controller determining that the imaging device is obstructed when the determined distance is outside of a predetermined distance range.

26. The apparatus of claim 23 wherein the feature analyzed by the controller is blobs of the three-dimensional image, the controller determining that the imaging device is obstructed when a blob of the three-dimensional image has a size that is outside of a predetermined blob size range.

27. The apparatus of claim 21 further including:

at least three identifying marks located at predetermined positions in the area of interest;

a memory in which the predetermined positions of the at least three identifying marks are stored; and

a controller for determining whether the imaging device is calibrated properly by analyzing the obtained image to determine whether positions of the at least three identifying marks in the obtained image correspond to the predetermined positions stored in the memory.

28. The apparatus of claim 27 wherein the at least three identifying marks are formed on interior structures of a vehicle and the area of interest is a passenger compartment of a vehicle.

29. The apparatus of claim 28 further including an inflatable air bag, inflation of the inflatable air bag being controlled in response to the location of the object in the passenger compartment.

30. The apparatus of claim 28 wherein the at least three identifying marks are formed from near-infrared dye and the imaging device includes first and second near infrared cameras that are configured to obtain images including the at least three identifying marks.

31. The apparatus of claim 30 further including a near-infrared illuminator for illuminating the passenger compartment of the vehicle.

32. A method of self-diagnostics of an apparatus for determining a location of an object in an area of interest, the method comprising the steps of:

obtaining an image of the area of interest with an imaging device;

analyzing a feature of the obtained image to determine whether a view of the area of interest from the imaging device is obstructed;

providing an indication when the view is obstructed; and

analyzing the obtained image to determine the location of the object when the view is unobstructed.

33. The method of claim 32 wherein the step of obtaining an image of the area of interest further includes the steps of:

taking first and second two-dimensional images of the area of interest; and

forming a three-dimensional image of the area of interest from the first and second two-dimensional images.

34. The method of claim 33 wherein the step of analyzing a feature further includes the steps of:

analyzing exposure of at least one of the first and second two-dimensional images; and

determining that the imaging device is obstructed when the analyzed exposure is outside of a predetermined exposure range.

35. The method of claim 33 wherein the step of analyzing a feature further includes the steps of:

determining a distance to the object using disparity of the first and second two-dimensional images; and

determining that the imaging device is obstructed when the determined distance is outside of a predetermined distance range.

36. The method of claim 33 wherein the step of analyzing a feature further includes the steps of:

analyzing blobs of the three-dimensional image; and

determining that the imaging device is obstructed when a blob of the three-dimensional image has a size that is outside of a predetermined blob size range.

37. The method of claim 32 further including the steps of:

locating at least three identifying marks at predetermined positions in the area of interest;

storing the predetermined positions of the at least three identifying marks in a memory; and

determining whether the imaging device is calibrated properly by analyzing the obtained image to determine whether positions of the at least three identifying marks in the obtained image correspond to the predetermined positions stored in the memory.

38. The method of claim 37 further including the steps of:

forming the at least three identifying marks from dye; and

forming the imaging device from first and second wide spectrum cameras that are configured to obtain images including the at least three identifying marks.

39. The method of claim 38 further including the step of:

illuminating the area of interest with near-infrared light.

40. The method of claim 32 further including the steps of:

determining whether the imaging device is calibrated properly;

determining the location of the object in the area of interest when the imaging device is calibrated properly and the view is unobstructed; and

controlling inflation of an inflatable air bag in response to the determined location of the object in the area of interest.