Image Processing for Vehicular Applications

• US 20070086624A1
• Filed: 10/05/2006
• Published: 04/19/2007
• Est. Priority Date: 06/07/1995
• Status: Active Grant

• Current Assignee: American Vehicular Sciences LLC (Acacia Research Corporation)
• Original Assignee: Automotive Technologies International Incorporated
• Inventors: Breed, David, Johnson, Wendell, Khvastukhin, Mykhailo
• Granted Patent: US 7,596,242 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60R 21/01516 using force or pressure sensing means
• CPC Class Codes: B60R 21/0152 using strain gauges
• CPC Class Codes: B60R 21/01532 using electric or capacitive field sensors
• CPC Class Codes: B60R 21/01534 using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 using ultrasonic waves
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: E05F 15/40 Safety devices, e.g. detection of obstructions or end positions
• CPC Class Codes: E05Y 2900/50 for vehicles
• CPC Class Codes: E05Y 2900/55 Windows
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Method for obtaining information about an occupying item in a compartment in a vehicle includes obtaining a series of images of the compartment, each image being taken at a different time and at least one of which includes the occupying item, comparing a later obtained image from the series of images to at least one previously obtained image from the series of images to ascertain the presence of differences between the images, and analyzing the differences to obtain information about the occupying item. One way to compare images is to subtract the later obtained image from the previously obtained image to determine which image pixels have changed in value and analyze the changed pixels. This analysis may involve analyzing a leading edge of the changed pixels and/or analyzing a width of a field of the changed pixels.

Claims

1. A method for obtaining information about an occupying item in a compartment in a vehicle, comprising:

obtaining a series of images of the compartment, each image being taken at a different time and at least one of which includes the occupying item;

comparing a later obtained image from the series of images to at least one previously obtained image from the series of images to ascertain the presence of differences between the images; and

analyzing the differences to obtain information about the occupying item.

2. The method of claim 1, wherein the analysis of the differences is performed using a trained pattern recognition system.

3. The method of claim 1, wherein the differences are analyzed to determine position of the occupying item relative to an occupant protection system.

4. The method of claim 1, wherein the differences are analyzed to determine the presence of a head or chest of a human occupant and the position of the head or chest, when present, relative to a deployable occupant protection system, further comprising controlling deployment of the occupant protection system based on the position of the head or chest of the occupant.

5. The method of claim 1, wherein the differences are analyzed to determine motion of the occupying item.

6. The method of claim 1, wherein the differences are analyzed to determine an identity of the occupying item.

7. The method of claim 1, wherein the comparison of images involves subtracting the later obtained image from the previously obtained image to determine which image pixels have changed in value and analyzing the changed pixels.

8. The method of claim 7, wherein the analysis of the changed pixels involves analyzing a leading edge of the changed pixels.

9. The method of claim 7, wherein the analysis of the changed pixels involves analyzing a width of a field of the changed pixels.

10. The method of claim 1, wherein the previously obtained image is obtained without infrared illumination and the later obtained image is obtained with infrared illumination.

11. The method of claim 10, wherein the comparison of images involves subtracting the later obtained image from the previously obtained image or subtracting the previously obtained image from the later obtained image.

12. The method of claim 1, further comprising arranging a single imaging device in the vehicle to obtain the entire series of images.

13. The method of claim 1, wherein the at least one previously obtained image is an image of the compartment including permanent structure in the compartment without a temporary occupying item and the comparison of images involves subtracting the at least one previously obtained image from the later obtained image to thereby remove the effect of the presence of permanent structure from the later obtained image.

14. The method of claim 1, wherein the series of images are taken by a rolling shutter, further comprising:

providing illumination for alternating rows of pixels with rows of pixels between said alternating illuminated rows not being illuminated; and

subtracting one illuminated row of pixels from an adjacent unilluminated row.

15. The method of claim 1, further comprising determining edges of shapes in the images prior to comparison of the images such that only the determined edges of the shapes of the previously obtained image and the later obtained image are compared to one another.

16. The method of claim 1, wherein the comparison of images involves determining which reflections of the occupying item remain static for all of the images and determining which reflections move between the images whereby analysis of the differences constitutes analyzing the reflections which move.

17. A method for determining the identification and position of objects in a compartment of a vehicle, comprising:

storing images of the compartment taken from a single imaging device with different objects in various positions in the compartment in association with data relating to the identification of the objects and the position of the objects relative to the compartment;

obtaining a plurality of images of the interior of the compartment from the imaging device; and

comparing the obtained images of the interior of the compartment with the stored images to obtain the associated data relating to the identification and position of the objects of the stored image to which the obtained images of the interior of the compartment match most closely.

18. The method of claim 17, wherein after the images are obtained and prior to their comparison with the stored images, the obtained images are compared with a series of stored images representing different unoccupied states of the compartment and each of the stored images representing unoccupied states of the compartment is subtracted from the obtained images.

19. The method of claim 17, wherein the step of comparing the obtained images of the interior of the compartment with stored images comprises:

analyzing the total number of pixels in each obtained image reduced below a threshold level; and

analyzing the minimum number of remaining detached pixels.

20. The method of claim 17, further comprising generating a library of images of the compartment with different objects in various positions by

positioning an object on the seat,

transmitting electromagnetic waves into the compartment,

obtaining images of the interior of the compartment,

associating the images with the identification and position of the object, and

repeating the positioning step, transmitting step, image obtaining step and associating step for the same object in different positions and for different objects in different positions.

21. The method of claim 17, further comprising after the identification and position of the objects are obtained, affecting another system in the vehicle based on the obtained identification and position of at least one of the objects.

22. The method of claim 17, further comprising:

measuring the weight applied onto the seat; and

affecting another system in the vehicle based on the measured weight applied onto the seat and the identification and position of the objects in the compartment.

23. A method for determining the identification and position of objects in a compartment of a vehicle, comprising:

obtaining a plurality of images of the interior of the compartment from a single location;

creating a three-dimensional representation of the interior of the compartment from the images; and

applying a pattern recognition technique to the representation in order to determine the identification and position of the objects in the compartment.

24. The method of claim 23, wherein the step of applying a pattern recognition technique to the representation comprises conducting a training stage in which a neural network is trained on representations of the interior of the compartment from images of different objects in various positions and associated data relating at least to the identification and position of the objects in the images.

25. The method of claim 23, wherein the step of obtaining the images comprises operating a laser in a pulse mode, scanning the interior of the compartment by moving the laser and determining the distance from the object being illuminated using range gating.

26. A method for determining the identification and position of objects in a compartment of a vehicle, comprising:

storing images of radiation emanating from different objects in various positions in the compartment taken from a single location in association with data relating to the identification of the objects and the position of the objects relative to the compartment;

obtaining a plurality of images of the interior of the compartment; and

comparing the obtained images with the stored of images to obtain the associated data relating to the identification and position of the object of the stored image which the obtained images of the interior of the compartment matches most closely.

27. The method of claim 26, further comprising generating a library of images of the compartment with different objects in various positions by

positioning an object on the seat,

obtaining images of the object,

associating the images with the identification and position of the object, and

repeating the positioning step, image obtaining step and associating step for the same object in different positions and for different objects in different positions.

28. The method of claim 26, further comprising after the identification and position of the objects are obtained, affecting another system in the vehicle based on the obtained identification and position of at least one of the objects.

29. The method of claim 26, further comprising:

measuring the weight applied onto the seat; and

affecting another system in the vehicle based on the measured weight applied onto the seat and the identification and position of the objects in the compartment.

30. A method of developing a system for determining the occupancy state of a seat in a passenger compartment of a vehicle, comprising:

mounting at least one imaging device in the vehicle;

obtaining a series of images of the compartment from each of the least one imaging device;

forming sets of the images from each of the at least one imaging device with one image of each set being an empty seat and another image of each set having one of a plurality of different occupancy states;

performing image subtraction on the sets of the images to obtain differential data indicative of differences between the images of each set; and

creating a first, trained pattern recognition algorithm from the differential data capable of producing an output indicative of the occupancy state of the seat upon inputting differential data.