Optical monitoring of vehicle interiors

• US 8,948,442 B2
• Filed: 01/04/2007
• Issued: 02/03/2015
• Est. Priority Date: 06/18/1982
• Status: Expired due to Fees

• Current Assignee: Intelligent Technologies International, Inc.
• Original Assignee: Intelligent Technologies International, Inc.
• Inventors: Breed, David S, DuVall, Wilbur E, Johnson, Wendell C
• Primary Examiner(s): YENTRAPATI, AVINASH
• Application Number:
  US11/619,863
• Publication Number:
  US 20070262574A1
• Time in Patent Office: 2,952 Days
• Field of Search: 382/291, 382/321, 382/156, 382/159, 382/165, 382/100
• US Class Current: 382/100
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/29 for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
• CPC Class Codes: B60R 2021/0027 Post collision measures, e.g. notifying emergency services
• CPC Class Codes: B60R 21/015 including means for detecting the presence or position of passengers, passenger seats or child seats , and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
• CPC Class Codes: B60R 21/0152 using strain gauges
• CPC Class Codes: B60R 21/01526 using piezoelectric elements
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 2300/8073 for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Vehicle includes a structure defining an interior space for containing or designed to contain one or more objects, and an arrangement mounted on, joined or coupled to the structure for determining whether an object is present in the interior space, or the presence of multiple objects. One arrangement includes at least one optical imaging device arranged to receive images of the interior space defined by the structure, and a processor coupled to the optical imaging device(s) and arranged to determine whether one or more objects are present in the interior space and when one or more objects are determined to be present, to obtain information about the object or objects. The processor is preferably arranged to distinguish between different objects and/or different arrangements of objects. A related method for obtaining information about objects in interior spaces of vehicles is also disclosed.

Claims

1. A movable vehicle adapted to retain cargo and thereby move the cargo from one place to another, comprising:

a structure defining an interior space of the movable vehicle for containing one or more inanimate objects constituting the cargo of the movable vehicle, said structure being a cargo container, a railroad car, a truck trailer, a ship having a hold in which the cargo is retained or an airplane having a cargo area in which the cargo is retained;

an arrangement mounted on said structure and within said structure such that said arrangement moves with said structure, said arrangement monitoring said interior space defined by said structure and determining whether said interior space defined by said structure is empty, whether an inanimate object is present in said interior space defined by said structure and when an inanimate object is determined to be present, whether the inanimate object has changed its position while still being present in said interior space,

said arrangement comprising;

at least one optical imaging device that receives images of said interior space defined by said structure; and

a processor coupled to said at least one optical imaging device and configured to determine whether said interior space defined by said structure is empty by comparing an image of said interior space that is obtained by said at least one optical imaging device to a previously obtained image of said interior space when empty and considering said interior space defined by said structure to be empty when the images are the same; and

a communications system mounted on said structure and coupled to said arrangement such that said communications system moves with said structure, said communications system wirelessly transmitting from said structure, the determination that said interior space is empty,

said processor being further configured to determine whether one or more inanimate objects are present in said interior space by comparing an image of said interior space that is obtained by said at least one optical imaging device to the previously obtained image of said interior space when empty,

said processor being further configured to obtain, when one or more inanimate objects are determined to be present, identification information about the inanimate object or objects,

said processor being further configured to analyze images received by said at least one optical imaging device at different times to determine changes in the presence of the inanimate objects indicative of insertion of a new object into said interior space or removal of an object from said interior space,

said processor being further configured to analyze images received by said at least one optical imaging device at different times to determine changes in position of the same inanimate object indicative of movement of that object within said interior space,

whereby said processor is arranged to distinguish between different inanimate objects or different arrangements of inanimate objects; and

said communications system wirelessly transmitting from said structure, the determination of the presence of one or more inanimate object, the information obtained about the inanimate object or objects, the determination of changes in the presence of the inanimate objects and the determination of changes in position of the same inanimate object to a site remote and separate from the vehicle.

2. The vehicle of claim 1, wherein said at least one optical imaging device consists of a single optical imaging device.

3. The vehicle of claim 1, wherein said processor comprises a neural network that determines whether an object is present after being trained in a training stage in which images received by said at least one optical imaging device in the absence of objects in said interior space and images received by said at least one optical imaging device with objects present in said interior space are collected and used to derive the neural network.

4. The vehicle of claim 1, further comprising an illumination device mounted on said structure for illuminating at least a portion of said interior space with electromagnetic radiation.

5. The vehicle of claim 1, further comprising a structured light generator which transmits structured light into said interior space.

6. The vehicle of claim 1, wherein said processor is configured to generate, when one or more inanimate objects are determined to be present in said interior space, a signal characteristic of the inanimate object based on the received images, and categorize the signal to thereby identify the inanimate object and provide the identification information about the inanimate object.

7. The vehicle of claim 6, wherein said processor comprises a pattern recognition technique that recognizes and thus identifies a class of the object by processing the signal into a categorization of the signal based on data relating to images received by said at least one optical imaging device and associated with possible classes of objects.

8. The vehicle of claim 1, wherein said processor is configured to apply a pattern recognition algorithm generated from images of said interior space with different objects therein in order to obtain information about the objects.

9. The vehicle of claim 1, wherein said at least one optical imaging device comprises a plurality of optical imaging devices spaced apart from one another and that obtain images of a common portion of said interior space defined by said structure.

10. The vehicle of claim 1, wherein said structure includes a roof, said at least one optical imaging device being mounted on or proximate to said roof.

11. The vehicle of claim 1, further comprising a reactive system coupled to said processor and being controlled by said processor, when one or more inanimate objects are determined to be present in said interior space, based on the determination by said processor as to whether an inanimate object is present in said interior space and the information about the inanimate object when present in said interior space.

12. The vehicle of claim 1, wherein said processor is further configured to obtain, when one or more inanimate objects are determined to be present, one or more of a position of the inanimate object, a type of the inanimate object and a size of the inanimate object.

18. The vehicle of claim 1, wherein said communications system is arranged to transmit the determination of the presence of one or more inanimate objects and the information obtained about the inanimate object or objects to the site remote and separate from the vehicle only when one of the following occurs:

receipt of a request from the remote site;

said processor determines, based on analysis of the images received from said at least one optical imaging device, that status of the interior space has changed to a new status requiring notification to the remote site;

receipt of an indication that security of the interior space has been violated;

receipt of an indication that an environment within the interior space has changed;

passage of a predetermined amount of time; and

receipt of occurrence of an event relating to a change in location of the vehicle.

20. The vehicle of claim 1, wherein when said processor compares an image of said interior space that is obtained by said at least one optical imaging device to the previously obtained image of said interior space when empty, said processor performs image subtraction.

22. The vehicle of claim 1, wherein said structure includes a door for accessing said interior space, said at least one imaging device being configured to obtain the image of said interior space when empty based on closure of said door.

13. A method for monitoring an interior space of a movable vehicle adapted to retain cargo and thereby move the cargo from one place to another, comprising:

mounting at least one optical imaging device within a structure defining the interior space of the movable vehicle, the structure being a cargo container, a railroad car, a truck trailer, a ship having a hold in which the cargo is retained or an airplane having a cargo area in which the cargo is retained;

receiving images at the at least one optical imaging device during use of the vehicle; and

when images are received, monitoring the interior space defined by the structure, and

determining, using a processor mounted on the structure and that moves with the structure, whether the interior space defined by the structure is empty by comparing an image of the interior space to a previously obtained image of the interior space when empty and considering the interior space defined by the structure to be empty when the images are the same, and wirelessly transmitting from the structure, via a communications system mounted on the structure and that moves with the structure, the determination that the interior space is empty to a site remote and separate from the vehicle, and

determining whether any inanimate objects constituting the cargo are present in the interior space defined by the structure by comparing an image of the interior space to the previously obtained image of the interior space when empty, and wirelessly transmitting from the structure, via the communications system, the determination of the presence of one or more inanimate objects in the interior space to the site remote and separate from the vehicle, and

when one or more inanimate objects are determined to be present,

obtaining identification information about the inanimate object or objects based on analysis of the received images, whereby the processor is arranged to distinguish between different inanimate objects or different arrangements of inanimate objects;

analyzing, using the processor, images received at the at least one optical imaging device at different times to determine changes in the presence of the inanimate objects indicative of insertion of a new object into the interior space or removal of an object from the interior space and wirelessly transmitting from the structure, via the communications system, the determination of changes in the presence of the inanimate objects to the site remote and separate from the vehicle;

analyzing, using the processor, images received by the at least one optical imaging device at different times to determine changes in position of the same inanimate object indicative of movement of that object within the interior space and wirelessly transmitting from the structure, via the communications system the determination of changes in position of the same inanimate object to the site remote and separate from the vehicle.

14. The method of claim 13, further comprising distinguishing, using the processor and only when one or more objects are determined to be present in the interior space of the vehicle, between different objects or different arrangements of objects when one or more objects are determined to be present in the interior space of the vehicle.

15. The method of claim 13, wherein the step of determining whether any objects are present and obtaining information about the object or objects when present comprises training a pattern recognition algorithm in a training stage to provide such output, and in the training stage, collecting images received by the at least one optical imaging device in the absence of objects in the interior space and with objects present in the interior space.

16. The method of claim 13, further comprising illuminating at least a portion of the interior space with electromagnetic radiation.

17. The method of claim 13, further comprising controlling, only when one or more objects are determined to be present in the interior space of the vehicle, a reactive system based on the determination as to whether any objects are present in the interior space and the information about the object or objects when present in the interior space.

19. The method of claim 13, wherein the communications system is arranged to transmit the determination of the presence of one or more inanimate objects and the information obtained about the inanimate object or objects to the site remote and separate from the vehicle only when one of the following occurs:

receipt of a request from the remote site;

the processor determines, based on analysis of the images received from the at least one optical imaging device, that status of the interior space has changed to a new status requiring notification to the remote site;

receipt of an indication that security of the interior space has been violated;

receipt of an indication that an environment within the interior space has changed;

passage of a predetermined amount of time; and

receipt of occurrence of an event relating to a change in location of the vehicle.

21. The method of claim 13, wherein when the processor compares an image of the interior space that is obtained by the at least one optical imaging device to the previously obtained image of the interior space when empty, the processor performs image subtraction.

23. The method of claim 13, further comprising obtaining the image of the interior space when empty based on closure of a door of the structure that enables access to the interior space.