Optical monitoring of vehicle interiors

• US 9,290,146 B2
• Filed: 10/18/2007
• Issued: 03/22/2016
• Est. Priority Date: 05/05/1992
• Status: Expired due to Fees

• Current Assignee: Intelligent Technologies International, Inc.
• Original Assignee: Intelligent Technologies International, Inc.
• Inventors: Breed, David S.
• Primary Examiner(s): YANG, JAMES J
• Application Number:
  US11/874,343
• Publication Number:
  US 20080036580A1
• Time in Patent Office: 3,078 Days
• Field of Search: 340/425.5, 340/438, 340/457
• US Class Current: 1/1
• CPC Class Codes: B60J 7/0573 power driven arrangements, e.g. electrical
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/015 Attaching seats directly to vehicle chassis
• CPC Class Codes: B60N 2/02246 Electric motors therefor
• CPC Class Codes: B60N 2/0244 with logic circuits
• CPC Class Codes: B60N 2/0248 with memory of positions
• CPC Class Codes: B60N 2/0252 with relations between different adjustments, e.g. height of headrest following longitudinal position of seat
• CPC Class Codes: B60N 2/0268 using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant
• CPC Class Codes: B60N 2/0272 using sensors or detectors for detecting the position of seat parts
• CPC Class Codes: B60N 2/0276 reaction to emergency situations, e.g. crash
• CPC Class Codes: B60N 2/067 by linear actuators, e.g. linear screw mechanisms
• CPC Class Codes: B60N 2/28 Seats readily mountable on, and dismountable from, existing seats or other parts of the vehicle
• CPC Class Codes: B60N 2/2806 for securing the child seat to the vehicle
• CPC Class Codes: B60N 2/2863 backward facing
• CPC Class Codes: B60N 2/66 Lumbar supports
• CPC Class Codes: B60N 2/829 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/853 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/888 with arrangements for protecting against abnormal g-forces, e.g. by displacement of the head-rest
• CPC Class Codes: B60R 16/037 for occupant comfort , e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
• CPC Class Codes: B60R 2001/1223 with sensors or transducers
• CPC Class Codes: B60R 2001/1253 : with cameras, video cameras or video screens
• CPC Class Codes: B60R 2021/01315 : monitoring occupant displacement
• CPC Class Codes: B60R 21/013 : including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/0132 : responsive to vehicle motion parameters , e.g. to vehicle longitudinal or transversal deceleration or speed value
• CPC Class Codes: B60R 21/0134 : responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: B60R 21/0136 : responsive to actual contact with an obstacle , e.g. to vehicle deformation, bumper displacement or bumper velocity relative to the vehicle monitoring crash strength in combination with passenger seat detection B60R21/01558
• CPC Class Codes: B60R 21/01516 : using force or pressure sensing means
• CPC Class Codes: B60R 21/0152 : using strain gauges
• CPC Class Codes: B60R 21/01526 : using piezoelectric elements
• CPC Class Codes: B60R 21/0153 : using field detection presence sensors
• CPC Class Codes: B60R 21/01532 : using electric or capacitive field sensors
• CPC Class Codes: B60R 21/01534 : using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 : using ultrasonic waves
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 : detecting passenger motion
• CPC Class Codes: B60R 21/01544 : detecting seat belt parameters, e.g. length, tension or height-adjustment
• CPC Class Codes: B60R 21/01546 : using belt buckle sensors
• CPC Class Codes: B60R 21/01548 : sensing the amount of belt winded on retractor
• CPC Class Codes: B60R 21/01552 : detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01554 : Seat position sensors
• CPC Class Codes: B60R 21/203 : in steering wheels or steering columns
• CPC Class Codes: B60R 21/21656 : Steering wheel covers or similar cup-shaped covers
• CPC Class Codes: B60R 21/276 : with means to vent the inflation fluid source, e.g. in case of overpressure B60R21/30 takes precedence; venting means on inflatable member walls B60R21/239
• CPC Class Codes: B60R 25/25 : using biometry
• CPC Class Codes: B60R 25/252 : Fingerprint recognition
• CPC Class Codes: B60R 25/255 : Eye recognition
• CPC Class Codes: B60R 25/257 : Voice recognition
• CPC Class Codes: E05F 15/43 : responsive to disruption of energy beams, e.g. light or sound
• CPC Class Codes: E05F 15/431 : specially adapted for vehicle windows or roofs
• CPC Class Codes: E05F 2015/433 : using reflection from the obstruction
• CPC Class Codes: E05Y 2900/516 : for trucks or trailers
• CPC Class Codes: E05Y 2900/542 : Roof panels
• CPC Class Codes: E05Y 2900/55 : Windows
• CPC Class Codes: G01F 23/20 : by measurement of weight, e.g. to determine the level of stored liquefied gas
• CPC Class Codes: G01F 23/2962 : Measuring transit time of reflected waves
• CPC Class Codes: G01F 23/804 : containing circuits handling parameters other than liquid level
• CPC Class Codes: G01S 13/04 : Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 15/04 : Systems determining presence of a target
• CPC Class Codes: G01S 15/06 : Systems determining the position data of a target
• CPC Class Codes: G01S 15/42 : Simultaneous measurement of distance and other co-ordinates indirect measurement G01S15/46
• CPC Class Codes: G01S 15/87 : Combinations of sonar systems
• CPC Class Codes: G01S 15/88 : Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 17/04 : Systems determining the presence of a target
• CPC Class Codes: G01S 17/88 : Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/89 : for mapping or imaging
• CPC Class Codes: G01S 7/417 : involving the use of neural networks
• CPC Class Codes: G01S 7/4802 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/539 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G06F 3/0219 : Special purpose keyboards
• CPC Class Codes: G06F 3/0233 : Character input methods
• CPC Class Codes: G06F 3/0237 : using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0238 : Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 : Recognising seat occupancy
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G07C 5/008 : communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0808 : Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G08B 13/1427 : with transmitter-receiver for distance detection
• CPC Class Codes: G08B 21/0286 : Tampering or removal detection of the child unit from child or article
• CPC Class Codes: G08B 29/181 : due to failing power supply
• CPC Class Codes: H01Q 1/3291 : mounted in or on other locations inside the vehicle or vehicle body

Abstract

Interior monitoring system of a vehicle wherein at least one transducer is arranged on a first side of the structure of the vehicle and transmits waves toward an opposite, second side of the structure so that any objects contained in the interior space of the structure are situated between the first and second sides. Each transducer receives waves reflected from any objects between the first and second sides. A processor is coupled to each transducer and obtains information about any objects from which waves have been reflected and received thereby. Each transducer may be arranged to transmit infrared waves into the interior space, e.g., infrared waves in an eye-safe range.

Claims

1. A system for remotely monitoring an interior space of a vehicle from a location apart from the vehicle while the vehicle is in different locations, comprising:

a cargo container including a plurality of rectangular sides defining said interior space for containing one or more inanimate objects;

a system that obtains information about inanimate objects in said interior space that constitute cargo of said container including whether inanimate objects are present in said interior space or whether said interior space is empty, said system comprising;

at least one imaging transducer arranged on a first one of said sides of said container and oriented toward an opposite, second side of said container whereby any inanimate objects contained in said interior space of said container are situated between said first and second sides, said at least one imaging transducer being configured to generate images of any inanimate objects in said interior space between said first and second sides;

a processor arranged on said container, coupled to said at least one imaging transducer and configured to control said at least one imaging transducer to receive images of any inanimate objects in said interior space at a plurality of different times and obtain information about any inanimate objects based on analysis of images received by said at least one transducer, said processor being configured to control said at least one imaging transducer to receive images of any inanimate objects in said interior space at least one time while the vehicle is in each of a plurality of different locations,

said processor being configured to obtain images received by said at least one transducer at a first time when said container is empty and at a second time later and different than said first time when said container is known to contain inanimate objects,

said processor being further configured to determine the presence of inanimate objects in said container based on a comparison of said images obtained at said second time relative to said images obtained at said first time;

a communications unit coupled to said processor and arranged on said container; and

a location determining system partly arranged on said container, coupled to said processor and configured to obtain location information that enables the location of said container to be determined;

said communications unit being coupled to said location determining system and configured to wirelessly transmit in the same communication, the information obtained by said processor and the location information obtained by said location determining system to a remote location separate and apart from the vehicle, including the information about the presence of inanimate objects in said container, said communications unit being configured to wirelessly transmit the information via infrastructure separate and apart from the vehicle,

whereby information about the presence of inanimate objects in said container at each time a communication is transmitted by said communications unit is associated with the location information about said container as obtained by said location determining system when the communication is transmitted to thereby provide an inventory of inanimate objects with respect to location of said container.

2. The vehicle of claim 1, wherein said at least one imaging transducer comprises a camera including a strobe flash that illuminates said interior space in conjunction with said camera obtaining images of said interior space.

3. The vehicle of claim 1, wherein said container is defined by two pairs of opposed walls, a ceiling and a floor forming said sides of said container, said walls in each pair of walls being opposite one another and said ceiling being opposite said floor to provide said interior space with an empty state wherein movable, inanimate objects are not present between any of said walls, said at least one transducer being mounted to one of said walls constituting said first side of said container.

4. The vehicle of claim 1, wherein said first one of said sides of said container is a ceiling of said container such that said at least one transducer is arranged on said ceiling of said container.

5. The vehicle of claim 1, wherein said processor comprises a neural network configured to determine whether an object is present after being trained in a training stage in which images received by said at least one transducer in the absence of objects in said interior space and images received by said at least one transducer with objects present in said interior space are collected and used to derive the neural network.

6. The vehicle of claim 1, wherein said at least one transducer is configured to transmit infrared waves into said interior space.

7. The vehicle of claim 1, further comprising a wakeup sensor which initiates said at least one transducer to obtain an image of said interior space.

8. The vehicle of claim 1, wherein said communications unit is configured to transmit at least one of said images obtained by said at least one transducer to the remote location in a communication with location information obtained by said location determining system when the at least one of said images was obtained to thereby provide a visual record of objects in said interior space with respect to location.

9. The vehicle of claim 7, wherein said wakeup sensor is activated upon detection of motion in said interior space.

10. The vehicle of claim 1, further comprising:

a door controlling access to said interior space of said cargo container; and

a door status sensor that detects when said door is opened from a closed position or closed from an open position,

said processor being coupled to said door status sensor and being configured to obtain images from said at least one transducer at the first and second times dependent on opening and closing of said door as detected by said door status sensor,

whereby when access to said interior space of said container is provided by opening of said door, said processor obtains images of said interior space at the second time and determines whether objects have been introduced into said interior space or removed therefrom.

11. The vehicle of claim 1, wherein said at least one transducer is configured to be commanded to obtain images of any objects in said interior space between said first and second sides.

12. The vehicle of claim 1, wherein said processor is configured to generate a signal characteristic of the object based on the received images, and categorize the signal to thereby provide an identification of the object, said communications unit being configured to wirelessly transmit the identification of the object to the remote location.

13. The vehicle of claim 8, wherein said processor comprises a pattern recognition system that recognizes and thus identifies a class of the object by processing the signal into a categorization of the signal based on data relating to images received by said at least one transducer stored within said pattern recognition system and associated with possible classes of objects, said communications unit being configured to wirelessly transmit the identification of the class of the object to the remote location.

14. The vehicle of claim 1, wherein said processor is configured to apply a pattern recognition algorithm generated from images received from said interior space with different objects therein in order to obtain object-specific information about the objects, said communications unit being configured to wirelessly transmit the object-specific information about the objects to the remote location.

15. The vehicle of claim 1, further comprising a reactive system coupled to said processor and being controlled by said processor based on the determination by said processor as to whether an object is present in said interior space and the information about the object when present in said interior space.

16. The vehicle of claim 1, wherein the information about the object determined by said processor when an object is present in said interior space is one or more of a position of the object, a type of the object and a size of the object, said communications unit being configured to wirelessly transmit the one or more of a position of the object, a type of the object and a size of the object to the remote location.

17. A method for remotely monitoring an interior space of a vehicle from a location apart from the vehicle while the vehicle is in different locations, comprising:

mounting at least one transducer on a cargo container of the vehicle including a plurality of rectangular sides defining the interior space of the container in which one or more inanimate objects can be contained, the at least one transducer being mounted on a first one of the sides of the container and oriented toward an opposite, second side of the container;

receiving images at the at least one transducer during use of the vehicle and at a plurality of different times;

controlling the at least one imaging transducer to receive images of any inanimate objects in the interior space at least one time while the vehicle is in each of a plurality of different locations;

when images are received by the at least one transducer, determining using a processor whether any inanimate objects constituting cargo of the container are present in the interior space based on analysis of received images and when one or more inanimate objects are determined to be present, obtaining information using the processor about the inanimate object or objects based on the analysis of the received images,

the processor being arranged to obtain images received by the at least one transducer at a first time when the container is empty and at a second time later and different than the first time when the container is known to contain inanimate objects,

the processor being further arranged to determine the presence of inanimate objects in the interior space of the container based on a comparison of the images obtained at the second time relative to the images obtained at the first time;

monitoring the location of the container using a location determining system partly arranged on the container;

coupling a communications unit to the processor that is configured to wirelessly transmit information via infrastructure separate and apart from the vehicle;

wirelessly transmitting, in a common communication, the information obtained by the processor and location information obtained by the location determining system using the communications unit to a remote location separate and apart from the vehicle, including information about the presence of inanimate objects in the container,

whereby information about the presence of inanimate objects in the container at each time a communication is transmitted by the communications unit is associated with the location information about the container as obtained by the location determining system when the communication is transmitted to thereby provide an inventory of inanimate objects with respect to location of the container.

18. The method of claim 17, further comprising distinguishing between different objects or different arrangements of objects when one or more objects are determined to be present in the interior space of the vehicle.

19. The method of claim 17, wherein the step of determining whether any objects are present and obtaining information about the object or objects when present comprises training a pattern recognition algorithm in a training stage to provide such output, and in the training stage, collecting images received by the at least one transducer in the absence of objects in the interior space and with objects present in the interior space.

20. The method of claim 17, wherein the vehicle includes a door controlling access to the interior space of the cargo container and a door status sensor that detects when the door is opened from a closed position or closed from an open position, further comprising:

coupling the processor to the door status sensor; and

obtaining images from the at least one transducer at the first and second times dependent on opening and closing of the door as detected by the door status sensor,

whereby when access to the interior space of the container is provided by opening of the door, the processor obtains images of the interior space at the second time and determines whether objects have been introduced into the interior space or removed therefrom.

21. The method of claim 17, further comprising controlling a reactive system based on the determination as to whether any objects are present in the interior space and the information about the object or objects when present in the interior space.

22. The method of claim 17, wherein the sides of the container include two pairs of walls that are opposite one another, a ceiling and a floor opposite the ceiling, and the at least one transducer is mounted to one of the walls or the ceiling of the container such that the wall or the ceiling constitutes the first side of the container.

23. The method of claim 17, wherein the at least one transducer is configured to transmit infrared waves into the interior space.