Image processing for vehicular applications

• US 7,596,242 B2
• Filed: 10/05/2006
• Issued: 09/29/2009
• Est. Priority Date: 06/07/1995
• Status: Expired due to Fees

• Current Assignee: American Vehicular Sciences LLC (Acacia Research Corporation)
• Original Assignee: Automotive Technologies International Incorporated
• Inventors: Khvastukhin, Mykhailo, Breed, David S., Johnson, Wendell C.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US11/538,934
• Publication Number:
  US 20070086624A1
• Time in Patent Office: 1,090 Days
• Field of Search: 382/103, 382/104, 382/106, 382/107, 382/108, 382/144, 382168-172, 382/181, 382/184, 382154-156, 382/209, 382/224, 382/266, 382/270, 382/286, 382/291, 345/168, 703/6, 250/208.1, 250/221, 250/559.32, 701/44, 701/45
• US Class Current: 382/103
• CPC Class Codes: B60R 21/01516 using force or pressure sensing means
• CPC Class Codes: B60R 21/0152 using strain gauges
• CPC Class Codes: B60R 21/01532 using electric or capacitive field sensors
• CPC Class Codes: B60R 21/01534 using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 using ultrasonic waves
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: E05F 15/40 Safety devices, e.g. detection of obstructions or end positions
• CPC Class Codes: E05Y 2900/50 for vehicles
• CPC Class Codes: E05Y 2900/55 Windows
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Method for obtaining information about an occupying item in a compartment in a vehicle includes obtaining a series of images of the compartment, each image being taken at a different time and at least one of which includes the occupying item, comparing a later obtained image from the series of images to at least one previously obtained image from the series of images to ascertain the presence of differences between the images, and analyzing the differences to obtain information about the occupying item. One way to compare images is to subtract the later obtained image from the previously obtained image to determine which image pixels have changed in value and analyze the changed pixels. This analysis may involve analyzing a leading edge of the changed pixels and/or analyzing a width of a field of the changed pixels.

Claims

1. A method for obtaining information about an occupying item in a compartment in a vehicle, comprising:

obtaining a series of images of the compartment using an imaging device, each image being taken at a different time and at least one of which includes the occupying item;

using a processor for comparing a later obtained image from the series of images to at least one previously obtained image from the series of images to ascertain the presence of differences between the images; and

using the processor for analyzing the differences to obtain information about the occupying item.

2. The method of claim 1, wherein the analysis of the differences is performed by the processor using a trained pattern recognition system.

3. The method of claim 1, wherein the differences are analyzed by the processor to determine position of the occupying item relative to an occupant protection system.

4. The method of claim 1, wherein the differences are analyzed by the processor to determine the presence of a head or chest of a human occupant and the position of the head or chest, when present, relative to a deployable occupant protection system, further comprising using the processor for controlling deployment of the occupant protection system based on the position of the head or chest of the occupant.

5. The method of claim 1, wherein the differences are analyzed by the processor to determine motion of the occupying item.

6. The method of claim 1, wherein the differences are analyzed by the processor to determine an identity of the occupying item.

7. The method of claim 1, wherein the comparison of images comprises using the processor for subtracting the later obtained image from the at least one previously obtained image to determine which image pixels have changed in value and analyzing the changed pixels.

8. The method of claim 7, wherein the analysis of the changed pixels comprises using the processor for analyzing a leading edge of the changed pixels.

9. The method of claim 7, wherein the analysis of the changed pixels comprises using the processor for analyzing a width of a field of the changed pixels.

10. The method of claim 1, wherein the at least one previously obtained image is obtained using the imaging device without infrared illumination and the later obtained image is obtained using the imaging device with infrared illumination.

11. The method of claim 10, wherein the comparison of images comprises using the processor for subtracting the later obtained image from the at least one previously obtained image or subtracting the at least one previously obtained image from the later obtained image.

12. The method of claim 1, further comprising arranging a single imaging device in the vehicle to obtain the entire series of images.

13. The method of claim 1, wherein the at least one previously obtained image is an image of the compartment including permanent structure in the compartment without a temporary occupying item and the comparison of images comprises using the processor for subtracting the at least one previously obtained image from the later obtained image to thereby remove the effect of the presence of permanent structure from the later obtained image.

14. The method of claim 1, wherein the series of images are taken by a rolling shutter, further comprising:

providing illumination for alternating rows of pixels using an illumination device with rows of pixels between alternating illuminated rows not being illuminated; and

using the processor for subtracting one illuminated row of pixels from an adjacent unilluminated row.

15. The method of claim 1, further comprising using the processor for determining edges of shapes in the images prior to comparison of the images such that only the determined edges of the shapes of the at least one previously obtained image and the later obtained image are compared to one another.

16. The method of claim 1, wherein the comparison of images comprises using the processor for determining which reflections of the occupying item remain static for all of the images and determining which reflections move between the images whereby analysis of the differences constitutes analyzing the reflections which move.

17. A method for determining the identification and position of objects in a compartment of a vehicle, comprising:

storing images of the compartment taken from a single imaging device with different objects in various positions in the compartment in association with data relating to the identification of the objects and the position of the objects relative to the compartment, the images being stored in a memory device;

obtaining a plurality of images of the interior of the compartment from the imaging device; and

using a processor for comparing the obtained images of the interior of the compartment with the stored images to obtain the associated data relating to the identification and position of the objects of the stored image to which the obtained images of the interior of the compartment match most closely.

18. The method of claim 17, wherein after the images are obtained and prior to their comparison with the stored images, the obtained images are compared by the processor with a series of stored images representing different unoccupied states of the compartment and each of the stored images representing unoccupied states of the compartment is subtracted from the obtained images.

19. The method of claim 17, wherein the step of comparing the obtained images of the interior of the compartment with stored images comprises:

using the processor for analyzing the total number of pixels in each obtained image reduced below a threshold level; and

using the processor for analyzing the minimum number of remaining pixels.

20. The method of claim 17, further comprising using the processor for generating a library of images of the compartment with different objects in various positions by

positioning an object on the seat,

transmitting electromagnetic waves into the compartment from an electromagnetic wave transmitter,

obtaining images of the interior of the compartment using the single imaging device,

using the processor for associating the images with the identification and position of the object, and

repeating the positioning step, transmitting step, image obtaining step and associating step for the same object in different positions and for different objects in different positions.

21. The method of claim 17, further comprising after the identification and position of the objects are obtained, using a processor to affect another system in the vehicle based on the obtained identification and position of at least one of the objects.

22. The method of claim 17, further comprising:

measuring the weight applied onto the seat; and

affecting another system in the vehicle based on the measured weight applied onto the seat and the identification and position of at least one of the objects in the compartment.

23. A method for determining the identification and position of objects in a compartment of a vehicle, comprising:

storing images of radiation emanating from different objects in various positions in the compartment taken from a single location in association with data relating to the identification of the objects and the position of the objects relative to the compartment, the images being stored in a memory device;

obtaining a plurality of images of the interior of the compartment using an imaging device; and

using a processor for comparing the obtained images with the stored of images to obtain the associated data relating to the identification and position of the object of the stored image which the obtained images of the interior of the compartment matches most closely.

24. The method of claim 23, further comprising using the processor for generating a library of images of the compartment with different objects in various positions by

positioning an object on the seat,

obtaining images of the object using the imaging device,

using the processor for associating the images with the identification and position of the object, and

repeating the positioning step, image obtaining step and associating step for the same object in different positions and for different objects in different positions.

25. The method of claim 23, further comprising after the identification and position of the objects are obtained, using a processor to affect another system in the vehicle based on the obtained identification and position of at least one of the objects.

26. The method of claim 23, further comprising:

measuring the weight applied onto the seat; and

affecting another system in the vehicle based on the measured weight applied onto the seat and the identification and position of at least one of the objects in the compartment.

27. A method of developing a system for determining the occupancy state of a seat in a passenger compartment of a vehicle, comprising:

mounting at least one imaging device in the vehicle;

storing an image of the compartment obtained from each of the at least one imaging device with a seat in the compartment being empty to thereby obtain an empty seat image analysis;

obtaining at least one additional image of the compartment from each of the at least one imaging device, each of the at least one additional image including the seat and the seat having one of a plurality of different occupancy states;

using a processor for pre-processing each of the at least one additional image to remove pixels from each of the at least one additional images based on the empty seat image analysis to thereby obtain image data; and

using the processor for creating a trained pattern recognition algorithm from the image data capable of producing an output indicative of the occupancy state of the seat upon inputting image data.