Vehicular restraint system control system and method using multiple optical imagers
First Claim
1. A vehicle comprising:
- a compartment;
a crash sensor system arranged to detect a crash involving the vehicle;
a restraint system having a variable actuation in the event of a crash involving the vehicle;
an occupant monitoring system for obtaining information about occupancy of said compartment and controlling said restraint system based on the obtained information, said occupant monitoring system comprising;
at least first and second optical imagers for obtaining images of a common area of said compartment, said first and second imagers being spaced apart from one another; and
processing circuitry coupled to said crash sensor system, said restraint system and said first and second imagers and arranged to derive information from images obtained by said first and second imagers and control the actuation of said restraint system based on the derived information and on detection of a crash by said crash sensor system,said processing circuitry being arranged to identify the occupant such that the presence of different types of occupants is determinable and said restraint system is controllable to provide different actuations depending on the type of the occupant.
2 Assignments
0 Petitions

Accused Products

Abstract
System and method for obtaining information about occupancy of a compartment in a movable object in which at least first and second optical imagers obtain images of a common area of the compartment and spaced apart from one another. Processing circuitry derives information from the images obtained by the imagers. A light source may illuminate the common area of the compartment and be interposed between the imagers. The processing circuitry can include a microprocessor with at least one pattern recognition algorithm and be arranged to determine the distance between the imagers and an object in the common area by locating a specific feature in the common area by first locating the feature in only the image obtained by one imager, then determining the location of the same feature in the image obtained by another imager, and determining the distance of the feature from the imagers by triangulation.
455 Citations
A METHOD FOR OPERATING A TRANSPORT VEHICLE, A TRANSPORT VEHICLE, A METHOD FOR CONTROLLLING OPERATION OF A WORK SITE AND A WORK SITE SYSTEM | ||
Patent #
US 20110066336A1
Filed 06/17/2008
|
Current Assignee
Volvo Construction Equipment AB
|
Original Assignee
Volvo Construction Equipment AB
|
Image acquisition and processing systems for vehicle equipment control | ||
Patent #
US 7,881,839 B2
Filed 11/14/2005
|
Current Assignee
Gentex Corporation
|
Original Assignee
Gentex Corporation
|
SENSOR ASSEMBLY FOR A MOVABLE SEAT | ||
Patent #
US 20110043010A1
Filed 08/18/2010
|
Current Assignee
Lear Corporation
|
Original Assignee
Lear Corporation
|
IMAGE ACQUISITION AND PROCESSING SYSTEMS FOR VEHICLE EQUIPMENT CONTROL | ||
Patent #
US 20110125374A1
Filed 01/31/2011
|
Current Assignee
Gentex Corporation
|
Original Assignee
Eric J. Walstra, Timothy S. Debruine, Gregory S. Bush, Joseph S. Stam
|
WIRELESS ANTI-THEFT SECURITY COMMUNICATIONS DEVICE AND SERVICE | ||
Patent #
US 20110149078A1
Filed 12/18/2009
|
Current Assignee
ATT Intellectual Property I LP
|
Original Assignee
ATT Intellectual Property I LP
|
AD-HOC MOBILE IP NETWORK FOR INTELLIGENT TRANSPORTATION SYSTEM | ||
Patent #
US 20110106337A1
Filed 10/11/2010
|
Current Assignee
Cisco Technology Incorporated
|
Original Assignee
Cisco Technology Incorporated
|
SYNCHRONIZING A RADIO NETWORK WITH END USER RADIO TERMINALS | ||
Patent #
US 20110183606A1
Filed 03/24/2011
|
Current Assignee
CSR Technology Incorporated
|
Original Assignee
SiRF Technology
|
Judging method of load distribution | ||
Patent #
US 8,019,530 B2
Filed 08/25/2005
|
Current Assignee
Sumitomo Rubber Industries Limited
|
Original Assignee
Sumitomo Rubber Industries Limited
|
Vehicle cabin lighting apparatus | ||
Patent #
US 8,055,023 B2
Filed 07/05/2005
|
Current Assignee
Nippon Soken Inc., DENSO Corporation
|
Original Assignee
Nippon Soken Inc., DENSO Corporation
|
Image acquisition and processing systems for vehicle equipment control | ||
Patent #
US 8,065,053 B2
Filed 01/31/2011
|
Current Assignee
Gentex Corporation
|
Original Assignee
Gentex Corporation
|
Ad-hoc mobile IP network for intelligent transportation system | ||
Patent #
US 8,036,782 B2
Filed 10/11/2010
|
Current Assignee
Cisco Technology Incorporated
|
Original Assignee
Cisco Technology Incorporated
|
Vehicular heads-up display system | ||
Patent #
US 8,032,264 B2
Filed 10/30/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Active sound effect generating apparatus | ||
Patent #
US 8,045,723 B2
Filed 07/17/2008
|
Current Assignee
Honda Motor Company
|
Original Assignee
Honda Motor Company
|
Distributed process control system and method utilizing wireless communication of packet messages | ||
Patent #
US 8,050,624 B2
Filed 06/24/2005
|
Current Assignee
Rosemount Incorporated
|
Original Assignee
Rosemount Incorporated
|
Method and apparatus for assessing contact clusters | ||
Patent #
US 7,750,840 B2
Filed 12/04/2007
|
Current Assignee
Raytheon Company
|
Original Assignee
Raytheon Company
|
Techniques for Controlling Operation of a Device with a Virtual Touchscreen | ||
Patent #
US 20100103141A1
Filed 10/27/2008
|
Current Assignee
Lenovo PC International Limited
|
Original Assignee
Lenovo PC International Limited
|
Device for adjusting seat components | ||
Patent #
US 7,740,096 B2
Filed 01/12/2005
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Techniques for Creating A Virtual Touchscreen | ||
Patent #
US 20100090983A1
Filed 10/15/2008
|
Current Assignee
Lenovo PC International Limited
|
Original Assignee
Lenovo PC International Limited
|
Image display device for rotating an image displayed on a display screen | ||
Patent #
US 7,746,365 B2
Filed 04/21/2005
|
Current Assignee
NEC Display Solutions Limited
|
Original Assignee
NEC Viewtechnology Ltd.
|
Enhanced glare reduction | ||
Patent #
US 20100065721A1
Filed 10/20/2009
|
Current Assignee
Leon D. Rosen, Craig Broude
|
Original Assignee
Leon D. Rosen, Craig Broude
|
Method and device for controlling at least one deceleration device and/or an output-determining actuating element of a vehicle drive device | ||
Patent #
US 7,660,668 B2
Filed 10/21/2003
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Method and device for detecting objects in the surroundings of a vehicle | ||
Patent #
US 7,729,856 B2
Filed 04/14/2004
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Remote vehicle diagnostic management | ||
Patent #
US 7,650,210 B2
Filed 08/14/2006
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Image Recordation Device with a Plurality of Lenses | ||
Patent #
US 20100271533A1
Filed 04/07/2010
|
Current Assignee
Crispin Porter Bogusky LLC
|
Original Assignee
Crispin Porter Bogusky LLC
|
Interactive programmable container security and compliance system | ||
Patent #
US 7,696,869 B2
Filed 04/05/2007
|
Current Assignee
Health Hero Network Incorporated
|
Original Assignee
Health Hero Network Incorporated
|
SYSTEM AND METHOD FOR WELLNESS MONITORING IN A VEHICLE | ||
Patent #
US 20100268051A1
Filed 04/16/2009
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Monitoring system for machine vibration | ||
Patent #
US 7,798,004 B2
Filed 01/28/2008
|
Current Assignee
Caterpillar Incorporated
|
Original Assignee
Caterpillar Incorporated
|
Method for selecting and adjusting scanner illuminant | ||
Patent #
US 7,679,791 B2
Filed 01/06/2003
|
Current Assignee
Intellectual Ventures I LLC
|
Original Assignee
Transpacific Systems LLC
|
SYSTEMS AND METHODS FOR PREVENTING MOTOR VEHICLE SIDE DOORS FROM COMING INTO CONTACT WITH OBSTACLES | ||
Patent #
US 20100082206A1
Filed 09/29/2008
|
Current Assignee
GM Global Technology Operations LLC
|
Original Assignee
GM Global Technology Operations Incorporated
|
Systems, methods and computer products for drowsy driver detection and response | ||
Patent #
US 7,719,431 B2
Filed 10/05/2007
|
Current Assignee
Steering Solutions IP Holding Corporation
|
Original Assignee
GM Global Technology Operations Incorporated
|
Occupant sensor and method for seat belt or other monitoring | ||
Patent #
US 7,791,476 B2
Filed 02/19/2007
|
Current Assignee
Nidec Elesys Americas Corporation
|
Original Assignee
Elesys North America Incorporated
|
Apparatus and method for detecting objects | ||
Patent #
US 7,840,026 B2
Filed 12/02/2005
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan Motor Co. Ltd.
|
VEHICULAR SAFETY SYSTEMS STATUS DISPLAY | ||
Patent #
US 20100332069A1
Filed 06/26/2009
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
ACTIVE SOUND EFFECT GENERATING APPARATUS | ||
Patent #
US 20090028353A1
Filed 07/17/2008
|
Current Assignee
Honda Motor Company
|
Original Assignee
Honda Motor Company
|
Method and device for detecting the occupancy state of a seat | ||
Patent #
US 7,492,923 B2
Filed 02/24/2003
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Artificial intelligence analyzer and generator | ||
Patent #
US 7,512,570 B2
Filed 05/30/2006
|
Current Assignee
Zaracom Technologies Inc.
|
Original Assignee
Zaracom Technologies Inc.
|
Glare reduction | ||
Patent #
US 20090058126A1
Filed 09/05/2007
|
Current Assignee
Leon D. Rosen, Craig Broude
|
Original Assignee
Leon D. Rosen, Craig Broude
|
Optical processor for an artificial neural network | ||
Patent #
US 7,512,573 B2
Filed 10/16/2006
|
Current Assignee
Alcatel-Lucent USA Inc.
|
Original Assignee
Alcatel-Lucent USA Inc.
|
SYSTEMS, METHODS AND COMPUTER PRODUCTS FOR DROWSY DRIVER DETECTION AND RESPONSE | ||
Patent #
US 20090091435A1
Filed 10/05/2007
|
Current Assignee
Steering Solutions IP Holding Corporation
|
Original Assignee
Delphi Technologies Inc.
|
Method and apparatus for locating an object of interest within an image | ||
Patent #
US 7,516,005 B2
Filed 09/04/2007
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Semiconductor device and method for operating the same | ||
Patent #
US 20090129593A1
Filed 05/25/2006
|
Current Assignee
Semiconductor Energy Laboratory Co. Ltd.
|
Original Assignee
Semiconductor Energy Laboratory Co. Ltd.
|
METHOD AND APPARATUS FOR ASSESSING CONTACT CLUSTERS | ||
Patent #
US 20090140909A1
Filed 12/04/2007
|
Current Assignee
Raytheon Company
|
Original Assignee
Raytheon Company
|
Monitoring system for machine vibration | ||
Patent #
US 20090188323A1
Filed 01/28/2008
|
Current Assignee
Caterpillar Incorporated
|
Original Assignee
Caterpillar Incorporated
|
Prognostic method and system for hybrid and electric vehicle components | ||
Patent #
US 7,558,655 B2
Filed 09/10/2004
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Virtual reality scene generator for generating training images for a pattern recognition classifier | ||
Patent #
US 7,574,018 B2
Filed 12/07/2005
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Network system, information processor and electronic apparatus | ||
Patent #
US 7,580,361 B2
Filed 06/20/2003
|
Current Assignee
Brother Kogyo Kabushiki Kaisha
|
Original Assignee
Brother Kogyo Kabushiki Kaisha
|
Headrest control apparatus and method | ||
Patent #
US 7,614,690 B2
Filed 08/23/2007
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha, Aisin Seiki Co. Ltd.
|
Dynamic weight sensing and classification of vehicular occupants | ||
Patent #
US 7,620,521 B2
Filed 10/26/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Tire Module and Method For Sensing Wheel State Variables and/or Tire State Variables | ||
Patent #
US 20090299570A1
Filed 05/10/2007
|
Current Assignee
Continental Teves AG Co. OHG
|
Original Assignee
Continental Teves AG Co. OHG
|
SIGNAL PROCESSING SYSTEM AND COMPONENTS THEREOF | ||
Patent #
US 20090315710A1
Filed 10/01/2007
|
Current Assignee
Microchip Technology Company Limited
|
Original Assignee
Microchip Technology Company Limited
|
E-field imaging and proximity detection using a spatially and temporally modulated source | ||
Patent #
US 20090322328A1
Filed 03/20/2009
|
Current Assignee
Thomas G. Cehelnik
|
Original Assignee
Thomas G. Cehelnik
|
Method and apparatus for locating an object of interest within an image | ||
Patent #
US 20080004776A1
Filed 09/04/2007
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Dynamic Weight Sensing and Classification of Vehicular Occupants | ||
Patent #
US 20080046200A1
Filed 10/26/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Vehicular Heads-Up Display System | ||
Patent #
US 20080048930A1
Filed 10/30/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Headrest control apparatus and method | ||
Patent #
US 20080067841A1
Filed 08/23/2007
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha, Aisin Seiki Co. Ltd.
|
Gesture-Based Control of Vehicular Components | ||
Patent #
US 20080065291A1
Filed 11/08/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Tire pressure monitor system with side entry pressure port | ||
Patent #
US 7,377,156 B1
Filed 02/16/2007
|
Current Assignee
LEAER CORPORATION
|
Original Assignee
Lear Corporation
|
MOBILE PHONE BASED NAVIGATION SYSTEM | ||
Patent #
US 20080114543A1
Filed 11/12/2007
|
Current Assignee
INTERCHAIN SOLUTION PRIVATE LIMITED
|
Original Assignee
INTERCHAIN SOLUTION PRIVATE LIMITED
|
AIR-DISTRIBUTION SYSTEM, WITH CONTACTLESS ACTIVATION OF AIR OUTLETS | ||
Patent #
US 20080108292A1
Filed 04/03/2007
|
Current Assignee
C.R.F. Societa Consortile Per Azioni
|
Original Assignee
C.R.F. Societa Consortile Per Azioni
|
Asset Monitoring System Using Multiple Imagers | ||
Patent #
US 20080100706A1
Filed 01/03/2008
|
Current Assignee
Intelligent Technologies International Inc.
|
Original Assignee
Intelligent Technologies International Inc.
|
VEHICLE CONVERTIBLE TOP WITH AUTOMATIC CLOSING FEATURE | ||
Patent #
US 20080106116A1
Filed 11/02/2007
|
Current Assignee
Charles Boulakia
|
Original Assignee
Charles Boulakia
|
Method for Adjusting Sensitivity of Load Detecting Device of Seat for a Vehicle | ||
Patent #
US 20080114517A1
Filed 10/31/2007
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha, Aisin Seiki Co. Ltd.
|
Optical processor for an artificial neural network | ||
Patent #
US 20080154815A1
Filed 10/16/2006
|
Current Assignee
Alcatel-Lucent USA Inc.
|
Original Assignee
Lucent Technologies Inc.
|
Vehicle and equipment monitoring apparatus | ||
Patent #
US 7,386,377 B2
Filed 04/07/2005
|
Current Assignee
David I. Sorensen
|
Original Assignee
David I. Sorensen
|
Artificial neuron with phase-encoded logic | ||
Patent #
US 7,401,058 B2
Filed 04/29/2005
|
Current Assignee
University Of Massachusetts
|
Original Assignee
University Of Massachusetts
|
Eye-Location Dependent Vehicular Heads-Up Display System | ||
Patent #
US 20080158096A1
Filed 03/13/2008
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Control and/or monitoring apparatus and method | ||
Patent #
US 7,397,363 B2
Filed 09/16/2002
|
Current Assignee
Joao Control and Monitoring Systems LLC
|
Original Assignee
Joao Control and Monitoring Systems LLC
|
Movable partition monitoring systems and methods | ||
Patent #
US 7,402,971 B2
Filed 02/02/2006
|
Current Assignee
Robert Bosch Corporation, Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Frequency shifting isolator system | ||
Patent #
US 7,415,397 B2
Filed 02/04/2005
|
Current Assignee
Lockheed Martin Corporation
|
Original Assignee
Lockheed Martin Corporation
|
UPLINK SCHEDULING FOR FAIRNESS IN CHANNEL ESTIMATION PERFORMANCE | ||
Patent #
US 20080205333A1
Filed 02/28/2007
|
Current Assignee
Qualcomm Inc.
|
Original Assignee
Qualcomm Inc.
|
Vehicle Accessories For Enhancing Smells | ||
Patent #
US 20080188172A1
Filed 02/07/2006
|
Current Assignee
Koninklijke Philips N.V.
|
Original Assignee
Koninklijke Philips N.V.
|
VEHICLE AND EQUIPMENT MONITORING APPARATUS | ||
Patent #
US 20080234889A1
Filed 05/30/2008
|
Current Assignee
David I. Sorensen
|
Original Assignee
David I. Sorensen
|
Image-capturing apparatus | ||
Patent #
US 7,421,091 B2
Filed 04/22/2004
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan Motor Co. Ltd.
|
Location sharing and tracking using mobile phones or other wireless devices | ||
Patent #
US 20080227473A1
Filed 03/11/2008
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Device for Adjusting Seat Components | ||
Patent #
US 20080246318A1
Filed 01/12/2005
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Apparatus and Method For Detecting Objects | ||
Patent #
US 20080247596A1
Filed 12/02/2005
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan Motor Co. Ltd.
|
DIRECTIONAL LOUDSPEAKER TO REDUCE DIRECT SOUND | ||
Patent #
US 20080247575A1
Filed 04/05/2007
|
Current Assignee
Harman International Industries Incorporated
|
Original Assignee
Harman International Industries Incorporated
|
INTERACTIVE PROGRAMMABLE CONTAINER SECURITY AND COMPLIANCE SYSTEM | ||
Patent #
US 20080246598A1
Filed 04/05/2007
|
Current Assignee
Health Hero Network Incorporated
|
Original Assignee
Health Hero Network Incorporated
|
Method and apparatus to predict the remaining service life of an operating system | ||
Patent #
US 7,457,785 B1
Filed 08/25/2000
|
Current Assignee
Battelle Memorial Institute
|
Original Assignee
Battelle Memorial Institute
|
Vehicular Heads-Up Display System | ||
Patent #
US 20080276191A1
Filed 10/26/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Remote Vehicle Diagnostic Management | ||
Patent #
US 20070005202A1
Filed 08/14/2006
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Integrated vehicle system | ||
Patent #
US 20070067082A1
Filed 09/20/2005
|
Current Assignee
Robert Bosch Corporation, Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Virtual reality scene generator for generating training images for a pattern recognition classifier | ||
Patent #
US 20070127811A1
Filed 12/07/2005
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
System for reducing carbon brake wear | ||
Patent #
US 20070179686A1
Filed 01/31/2006
|
Current Assignee
Hydro-Aire Incorporated Subsidiary of Crane Company
|
Original Assignee
Hydro-Aire Incorporated Subsidiary of Crane Company
|
OCCUPANT SENSOR AND METHOD FOR SEAT BELT OR OTHER MONITORING | ||
Patent #
US 20070194900A1
Filed 02/19/2007
|
Current Assignee
Nidec Elesys Americas Corporation
|
Original Assignee
Elesys North America Incorporated
|
Movable partition monitoring systems and methods | ||
Patent #
US 20070176463A1
Filed 02/02/2006
|
Current Assignee
Robert Bosch Corporation, Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Apparatus and method for providing shipment information | ||
Patent #
US 7,253,731 B2
Filed 01/22/2002
|
Current Assignee
Transcend Shipping Systems LLC
|
Original Assignee
Raymond Anthony Joao
|
Method for controlling an electromagnetic valve, in particular for an automatic transmission of a motor vehicle | ||
Patent #
US 7,260,462 B2
Filed 02/06/2004
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Method and device for detecting objects in the surroundings of a vehicle | ||
Patent #
US 20070182587A1
Filed 04/14/2004
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Controller system for a vehicle occupant protection device | ||
Patent #
US 7,283,901 B2
Filed 01/13/2005
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Vehicle occupant discriminating apparatus | ||
Patent #
US 7,295,910 B2
Filed 03/21/2006
|
Current Assignee
Aisin Seiki Co. Ltd.
|
Original Assignee
Aisin Seiki Co. Ltd.
|
Artificial neuron with phase-encoded logic | ||
Patent #
US 20060004681A1
Filed 04/29/2005
|
Current Assignee
University Of Massachusetts
|
Original Assignee
University Of Massachusetts
|
Vehicle cabin lighting apparatus | ||
Patent #
US 20060018641A1
Filed 07/05/2005
|
Current Assignee
Nippon Soken Inc., DENSO Corporation
|
Original Assignee
Nippon Soken Inc., DENSO Corporation
|
Keyless entry using biometric identification | ||
Patent #
US 20060056663A1
Filed 09/13/2004
|
Current Assignee
General Motors Corporation
|
Original Assignee
General Motors Corporation
|
PROGNOSTIC METHOD AND SYSTEM FOR HYBRID AND ELETRIC VEHICLE COMPONENTS | ||
Patent #
US 20060058932A1
Filed 09/10/2004
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Judging method of load distribution | ||
Patent #
US 20060064238A1
Filed 08/25/2005
|
Current Assignee
Sumitomo Rubber Industries Limited
|
Original Assignee
Sumitomo Rubber Industries Limited
|
Image acquisition and processing systems for vehicle equipment control | ||
Patent #
US 20060106518A1
Filed 11/14/2005
|
Current Assignee
Gentex Corporation
|
Original Assignee
Gentex Corporation
|
Method and devcie for detecting the occupancy state of a seat | ||
Patent #
US 20060161321A1
Filed 02/24/2003
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Method and apparatus for locating an object of interest within an image | ||
Patent #
US 20060155442A1
Filed 01/13/2005
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Frequency shifting isolator system | ||
Patent #
US 20060178859A1
Filed 02/04/2005
|
Current Assignee
Lockheed Martin Corporation
|
Original Assignee
Lockheed Martin Corporation
|
Vehicle occupant discriminating apparatus | ||
Patent #
US 20060217862A1
Filed 03/21/2006
|
Current Assignee
Aisin Seiki Co. Ltd.
|
Original Assignee
Aisin Seiki Co. Ltd.
|
Vehicle and equipment monitoring apparatus | ||
Patent #
US 20060229779A1
Filed 04/07/2005
|
Current Assignee
David I. Sorenson
|
Original Assignee
David I. Sorenson
|
Distributed process control system and method utilizing wireless communication of packet messages | ||
Patent #
US 20060291438A1
Filed 06/24/2005
|
Current Assignee
Rosemount Incorporated
|
Original Assignee
Rosemount Incorporated
|
Image display device for rotating an image displayed on a display screen | ||
Patent #
US 20050237346A1
Filed 04/21/2005
|
Current Assignee
NEC Display Solutions Limited
|
Original Assignee
NEC Viewtechnology Ltd.
|
Network system, information processor and electronic apparatus | ||
Patent #
US 20040049578A1
Filed 06/20/2003
|
Current Assignee
Brother Kogyo Kabushiki Kaisha
|
Original Assignee
Brother Kogyo Kabushiki Kaisha
|
Method for selecting and adjusting scanner illuminant | ||
Patent #
US 20040130756A1
Filed 01/06/2003
|
Current Assignee
Intellectual Ventures I LLC
|
Original Assignee
Transpacific Systems LLC
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 20040160319A1
Filed 02/20/2004
|
Current Assignee
Raymond Anthony Joao
|
Original Assignee
Raymond Anthony Joao
|
Method for controlling an electromagnetic valve, in particular for an automatic transmission of a motor vehicle | ||
Patent #
US 20040225429A1
Filed 02/06/2004
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Image-capturing apparatus | ||
Patent #
US 20040233300A1
Filed 04/22/2004
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan Motor Co. Ltd.
|
Eyewear with exchangeable temples housing a radio frequency transceiver | ||
Patent #
US 20040239874A1
Filed 07/02/2004
|
Current Assignee
III Holdings 4 LLC
|
Original Assignee
QR Spex
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 20030016130A1
Filed 09/16/2002
|
Current Assignee
Joao Control and Monitoring Systems LLC
|
Original Assignee
Joao Control and Monitoring Systems LLC
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 20030193404A1
Filed 04/23/2003
|
Current Assignee
GTJ Ventures LLC
|
Original Assignee
Raymond Anthony Joao
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 20030206102A1
Filed 04/23/2003
|
Current Assignee
GTJ Ventures LLC
|
Original Assignee
Raymond Anthony Joao
|
Apparatus and method for providing shipment information | ||
Patent #
US 20020099567A1
Filed 01/22/2002
|
Current Assignee
Transcend Shipping Systems LLC
|
Original Assignee
Raymond Anthony Joao
|
Artificial Intelligence Analyzer and Generator | ||
Patent #
US 20070282768A1
Filed 05/30/2006
|
Current Assignee
Zaracom Technologies Inc.
|
Original Assignee
Zaracom Technologies Inc.
|
Method for adjusting sensitivity of load detecting device of seat for a vehicle | ||
Patent #
US 8,086,375 B2
Filed 10/31/2007
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha, Aisin Seiki Co. Ltd.
|
Directional loudspeaker to reduce direct sound | ||
Patent #
US 8,121,336 B2
Filed 04/05/2007
|
Current Assignee
Harman International Industries Incorporated
|
Original Assignee
Harman International Industries Incorporated
|
Sensor detection controller and occupant detection apparatus having the same | ||
Patent #
US 20120065845A1
Filed 09/08/2011
|
Current Assignee
Nippon Soken Inc., DENSO Corporation
|
Original Assignee
DENSO Corporation
|
COLLISION DETERMINING APPARATUS FOR VEHICLE | ||
Patent #
US 20120072078A1
Filed 08/31/2011
|
Current Assignee
Keihin Corporation
|
Original Assignee
Keihin Corporation
|
Individualizable Post-Crash Assist System | ||
Patent #
US 20120078472A1
Filed 09/27/2010
|
Current Assignee
GM Global Technology Operations LLC
|
Original Assignee
GM Global Technology Operations Incorporated
|
Enhanced glare reduction | ||
Patent #
US 8,143,563 B2
Filed 10/20/2009
|
Current Assignee
Leon D. Rosen, Craig Broude
|
Original Assignee
Leon D. Rosen, Craig Broude
|
Image sharing system and method | ||
Patent #
US 8,144,944 B2
Filed 08/14/2007
|
Current Assignee
Olympus Corporation
|
Original Assignee
Olympus Corporation
|
Multi Sensor Detection, Stall to Stop and Lock Disabling System | ||
Patent #
US 20120086569A1
Filed 11/03/2011
|
Current Assignee
Larry Golden
|
Original Assignee
Larry Golden
|
VEHICLE SYSTEM REACTION TO MEDICAL CONDITIONS | ||
Patent #
US 20120112915A1
Filed 11/08/2010
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
IMAGE PROCESSING APPARATUS | ||
Patent #
US 20120133739A1
Filed 11/30/2011
|
Current Assignee
Subaru Corp.
|
Original Assignee
Fuji Heavy Industries Limited
|
Air-distribution system, with contactless activation of air outlets | ||
Patent #
US 8,210,913 B2
Filed 04/03/2007
|
Current Assignee
C.R.F. Societa Consortile Per Azioni
|
Original Assignee
C.R.F. Societa Consortile Per Azioni
|
Vehicular safety systems status display | ||
Patent #
US 8,239,101 B2
Filed 06/26/2009
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Signal processing system and components thereof | ||
Patent #
US 8,289,169 B2
Filed 10/01/2007
|
Current Assignee
Microchip Technology Company Limited
|
Original Assignee
IDENT Technology AG
|
Ad-hoc mobile IP network for intelligent transportation system | ||
Patent #
US 8,311,726 B2
Filed 09/23/2011
|
Current Assignee
Cisco Technology Incorporated
|
Original Assignee
Cisco Technology Incorporated
|
Individualizable post-crash assist system | ||
Patent #
US 8,335,616 B2
Filed 09/27/2010
|
Current Assignee
GM Global Technology Operations LLC
|
Original Assignee
GM Global Technology Operations LLC
|
Sensor detection controller and occupant detection apparatus having the same | ||
Patent #
US 8,335,617 B2
Filed 09/08/2011
|
Current Assignee
Nippon Soken Inc., DENSO Corporation
|
Original Assignee
Nippon Soken Inc., DENSO Corporation
|
VEHICLE EMERGENCY EVACUATION DEVICE | ||
Patent #
US 20130018549A1
Filed 07/11/2011
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Apparatus And Method For Activating A Localization Process For A Tire Pressure Monitor | ||
Patent #
US 20130038442A1
Filed 08/09/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems US Incorporated
|
SWIR targeted agile raman (STAR) system for on-the-move detection of emplace explosives | ||
Patent #
US 8,379,193 B2
Filed 06/17/2010
|
Current Assignee
Chemimage Corporation
|
Original Assignee
Chemimage Corporation
|
Methods and apparatuses for geospatial-based sharing of information by multiple devices | ||
Patent #
US 8,385,964 B2
Filed 06/07/2011
|
Current Assignee
XONE Incorporated
|
Original Assignee
XONE Incorporated
|
Synchronizing a radio network with end user radio terminals | ||
Patent #
US 8,437,693 B2
Filed 03/24/2011
|
Current Assignee
CSR Technology Incorporated
|
Original Assignee
CSR Technology Incorporated
|
Systems and methods for preventing motor vehicle side doors from coming into contact with obstacles | ||
Patent #
US 8,442,755 B2
Filed 09/29/2008
|
Current Assignee
GM Global Technology Operations LLC
|
Original Assignee
GM Global Technology Operations LLC
|
Techniques for creating a virtual touchscreen | ||
Patent #
US 8,446,389 B2
Filed 10/15/2008
|
Current Assignee
Lenovo PC International Limited
|
Original Assignee
Lenovo Singapore Pte Limited
|
INTRUSION DETECTION APPARATUS AND METHOD | ||
Patent #
US 20130181132A1
Filed 09/12/2012
|
Current Assignee
Electronics and Telecommunications Research Institute
|
Original Assignee
Electronics and Telecommunications Research Institute
|
Protocol misinterpretation avoidance apparatus and method for a tire pressure monitoring system | ||
Patent #
US 8,502,655 B2
Filed 08/09/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Techniques for controlling operation of a device with a virtual touchscreen | ||
Patent #
US 8,525,776 B2
Filed 10/27/2008
|
Current Assignee
Lenovo PC International Limited
|
Original Assignee
Lenovo Singapore Pte Limited
|
Multi sensor detection, stall to stop and lock disabling system | ||
Patent #
US 8,531,280 B2
Filed 11/03/2011
|
Current Assignee
Larry Golden
|
Original Assignee
Larry Golden
|
Location sharing and tracking using mobile phones or other wireless devices | ||
Patent #
US 8,538,458 B2
Filed 03/11/2008
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
APPARATUS FOR MEASURING DISTANCE AND METHOD THEREOF | ||
Patent #
US 20130242089A1
Filed 03/15/2013
|
Current Assignee
LG Innotek Company Limited
|
Original Assignee
LG Innotek Company Limited
|
System for reducing carbon brake wear | ||
Patent #
US 8,548,652 B2
Filed 01/31/2006
|
Current Assignee
Hydro-Aire Incorporated Subsidiary of Crane Company
|
Original Assignee
Hydro-Aire Incorporated Subsidiary of Crane Company
|
Electronic device with a situational awareness function | ||
Patent #
US 8,552,881 B2
Filed 02/09/2011
|
Current Assignee
Harris Corporation
|
Original Assignee
Harris Corporation
|
Protocol arrangement in a tire pressure monitoring system | ||
Patent #
US 8,576,060 B2
Filed 08/09/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Method for operating a transport vehicle, a transport vehicle, a method for controllling operation of a work site and a work site system | ||
Patent #
US 8,589,035 B2
Filed 06/17/2008
|
Current Assignee
Volvo Construction Equipment AB
|
Original Assignee
Volvo Construction Equipment AB
|
Ad-hoc mobile IP network for intelligent transportation system | ||
Patent #
US 8,620,491 B2
Filed 09/14/2012
|
Current Assignee
Cisco Technology Incorporated
|
Original Assignee
Cisco Technology Incorporated
|
Tire module and method for sensing wheel state variables and/or tire state variables | ||
Patent #
US 8,626,413 B2
Filed 05/10/2007
|
Current Assignee
Continental Teves AG Co. OHG
|
Original Assignee
Continental Teves AG Co. OHG
|
Eye-location dependent vehicular heads-up display system | ||
Patent #
US 8,686,922 B2
Filed 03/13/2008
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
American Vehicular Sciences LLC
|
Universal tire pressure monitoring sensor | ||
Patent #
US 8,692,661 B2
Filed 01/26/2012
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Vehicle system reaction to medical conditions | ||
Patent #
US 8,704,669 B2
Filed 11/08/2010
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Methods and systems for temporarily sharing position data between mobile-device users | ||
Patent #
US 8,712,441 B2
Filed 04/11/2013
|
Current Assignee
X One Inc.
|
Original Assignee
XONE Incorporated
|
Increasing vehicle security | ||
Patent #
US 8,725,330 B2
Filed 01/28/2011
|
Current Assignee
Bryan Marc Failing
|
Original Assignee
Bryan Marc Failing
|
CUTTING MACHINE AND EMERGENCY STOP METHOD OF MOTOR | ||
Patent #
US 20140130645A1
Filed 08/08/2012
|
Current Assignee
Hitachi Koki Company Limited
|
Original Assignee
Hitachi Koki Company Limited
|
Tire pressure monitoring apparatus and method | ||
Patent #
US 8,742,914 B2
Filed 08/09/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
System and method for safer detection of unknown materials using dual polarized hyperspectral imaging and Raman spectroscopy | ||
Patent #
US 8,743,358 B2
Filed 03/15/2013
|
Current Assignee
Chemimage Corporation
|
Original Assignee
Chemimage Corporation
|
Method of preparing a universal tire pressure monitoring sensor | ||
Patent #
US 8,742,913 B2
Filed 01/26/2012
|
Current Assignee
Continental Automotive Systems US Incorporated
|
Original Assignee
Continental Automotive Systems Inc.
|
Methods and systems for annotating target locations | ||
Patent #
US 8,750,898 B2
Filed 01/18/2013
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Protocol protection | ||
Patent #
US 8,751,092 B2
Filed 01/13/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Interactivity With A Mixed Reality | ||
Patent #
US 20140173493A1
Filed 02/19/2014
|
Current Assignee
Nant Holdings IP LLC
|
Original Assignee
Nant Holdings IP LLC
|
ULTRASHORT LASER MICRO-TEXTURE PRINTING | ||
Patent #
US 20140185065A1
Filed 03/05/2014
|
Current Assignee
IMRA America Incorporated
|
Original Assignee
IMRA America Incorporated
|
METHOD AND APPARATUS FOR A SEAT SUSPENSION SYSTEM | ||
Patent #
US 20140203602A1
Filed 03/31/2013
|
Current Assignee
Zero Shock Seating LLC
|
Original Assignee
William E. Woodbury II, Jody G. Robbins, Scott A. Boster, Mark J. Miskill
|
FOREIGN OBJECT DETECTION SYSTEM AND METHOD SUITABLE FOR SOURCE RESONATOR OF WIRELESS ENERGY TRANSFER SYSTEM | ||
Patent #
US 20140203629A1
Filed 01/18/2013
|
Current Assignee
Aptiv Technologies Limited
|
Original Assignee
Delphi Technologies Inc.
|
Method and Apparatus for Image Fusion | ||
Patent #
US 20140205204A1
Filed 01/15/2014
|
Current Assignee
Nokia Technologies Oy
|
Original Assignee
Nokia Corporation
|
Methods and systems for sharing position data and tracing paths between mobile-device users | ||
Patent #
US 8,798,645 B2
Filed 01/30/2013
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Tracking proximity of services provider to services consumer | ||
Patent #
US 8,798,647 B1
Filed 10/15/2013
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Location sharing and tracking using mobile phones or other wireless devices | ||
Patent #
US 8,798,593 B2
Filed 05/07/2013
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Vehicular heads-up display system | ||
Patent #
US 8,818,647 B2
Filed 10/26/2007
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
American Vehicular Sciences LLC
|
Arrangement for sensing weight of an occupying item in vehicular seat | ||
Patent #
US 8,820,782 B2
Filed 08/03/2012
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
American Vehicular Sciences LLC
|
Methods and apparatuses for transmission of an alert to multiple devices | ||
Patent #
US 8,831,635 B2
Filed 07/21/2011
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Occupant detection | ||
Patent #
US 8,836,491 B2
Filed 04/29/2010
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Energy transfer with vehicles | ||
Patent #
US 8,841,881 B2
Filed 01/28/2011
|
Current Assignee
Bryan Marc Failing
|
Original Assignee
Bryan Marc Failing
|
HEALTH STATISTICS AND COMMUNICATIONS OF ASSOCIATED VEHICLE USERS | ||
Patent #
US 20140309893A1
Filed 04/15/2014
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
Flextronics America LLC
|
USER PROFILE EXCHANGE VIA VEHICLE SUPPORTED COMMUNICATIONS PROTOCOL | ||
Patent #
US 20140309862A1
Filed 04/15/2014
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
Flextronics America LLC
|
BUILDING PROFILES ASSOCIATED WITH VEHICLE USERS | ||
Patent #
US 20140309866A1
Filed 04/15/2014
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
Flextronics America LLC
|
CELL PHONE HAVING A MONOLITHICALLY INTEGRATED MULTI-SENSOR DEVICE ON A SEMICONDUCTOR SUBSTRATE AND METHOD THEREFOR | ||
Patent #
US 20140264658A1
Filed 03/12/2014
|
Current Assignee
Versana Micro Inc.
|
Original Assignee
Bishnu Prasanna Gogoi
|
Gas Spring Compensation Marine Acoustic Vibrator | ||
Patent #
US 20140340985A1
Filed 12/31/2013
|
Current Assignee
PGS Geophysical As
|
Original Assignee
PGS Geophysical As
|
Omnidirectional Vehicle Camera System | ||
Patent #
US 20140347440A1
Filed 05/23/2014
|
Current Assignee
Sylvester Hatcher
|
Original Assignee
Sylvester Hatcher
|
SYSTEM AND METHOD FOR DETERMINING A POSITION OF A LIVING BEING IN A VEHICLE | ||
Patent #
US 20140347209A1
Filed 05/19/2014
|
Current Assignee
Volvo Car Corporation
|
Original Assignee
Volvo Car Corporation
|
Image acquisition and processing system for vehicle equipment control | ||
Patent #
US 8,924,078 B2
Filed 10/17/2011
|
Current Assignee
Gentex Corporation
|
Original Assignee
Gentex Corporation
|
HEAD-UP DISPLAY AND METHOD WITH SPEED SENSITIVE LIGHT INTENSITY | ||
Patent #
US 20150019072A1
Filed 07/09/2013
|
Current Assignee
Denso International America Incorporated, DENSO Corporation
|
Original Assignee
Denso International America Incorporated, DENSO Corporation
|
Ad-hoc mobile IP network for intelligent transportation system | ||
Patent #
US 8,938,353 B2
Filed 12/03/2013
|
Current Assignee
Cisco Technology Incorporated
|
Original Assignee
Cisco Technology Incorporated
|
Image processing apparatus | ||
Patent #
US 8,947,508 B2
Filed 11/30/2011
|
Current Assignee
Subaru Corp.
|
Original Assignee
Fuji Heavy Industries Limited
|
3D SONAR DISPLAY WITH SEMI-TRANSPARENT SHADING | ||
Patent #
US 20150039221A1
Filed 08/01/2014
|
Current Assignee
Garmin Switzerland GmbH
|
Original Assignee
Garmin Switzerland GmbH
|
Vehicle emergency evacuation device | ||
Patent #
US 8,954,238 B2
Filed 07/11/2011
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
High Occupancy Toll Lane Compliance | ||
Patent #
US 20150062340A1
Filed 09/03/2013
|
Current Assignee
GlobalFoundries Inc.
|
Original Assignee
International Business Machines Corporation
|
WINK DETECTING CAMERA | ||
Patent #
US 20150070482A1
Filed 11/12/2014
|
Current Assignee
Cutting Edge Vision LLC
|
Original Assignee
Cutting Edge Vision LLC
|
SYSTEMS AND METHODS FOR ILLUMINATION CONTROL AND DISTRIBUTION DURING A VEHICLE BANK | ||
Patent #
US 20150081168A1
Filed 05/08/2014
|
Current Assignee
J.W. Speaker Corporation
|
Original Assignee
Bradley William Kay, Dragolsav Popovic, Kevin Jay Mcwithey
|
Sensor assembly for a movable seat | ||
Patent #
US 8,985,542 B2
Filed 08/18/2010
|
Current Assignee
Lear Corporation
|
Original Assignee
Lear Corporation
|
Ultrashort laser micro-texture printing | ||
Patent #
US 8,995,029 B2
Filed 03/05/2014
|
Current Assignee
IMRA America Incorporated
|
Original Assignee
IMRA America Incorporated
|
System and method for eye safe detection of unknown targets | ||
Patent #
US 8,994,934 B1
Filed 11/10/2011
|
Current Assignee
Chemimage Corporation
|
Original Assignee
Chemimage Corporation
|
ROADWAY-INDUCED RIDE QUALITY RECONNAISSANCE AND ROUTE PLANNING | ||
Patent #
US 20150094948A1
Filed 09/30/2013
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Detecting a user-to-wireless device association in a vehicle | ||
Patent #
US 9,008,641 B2
Filed 12/27/2012
|
Current Assignee
Intel Corporation
|
Original Assignee
Intel Corporation
|
Head-up display and method with speed sensitive light intensity | ||
Patent #
US 9,008,908 B2
Filed 07/09/2013
|
Current Assignee
Denso International America Incorporated, DENSO Corporation
|
Original Assignee
Denso International America Incorporated, DENSO Corporation
|
Apparatus and method for activating a localization process for a tire pressure monitor | ||
Patent #
US 9,024,743 B2
Filed 08/09/2011
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems US Incorporated
|
Apparatus and method for obtaining content on a cellular wireless device based on proximity to other wireless devices | ||
Patent #
US 9,031,581 B1
Filed 11/07/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
SWIR targeted agile raman system for detection of unknown materials using dual polarization | ||
Patent #
US 9,052,290 B2
Filed 03/15/2013
|
Current Assignee
Chemimage Corporation
|
Original Assignee
Chemimage Corporation
|
REAR VISION SYSTEM FOR A VEHICLE | ||
Patent #
US 20150179072A1
Filed 03/09/2015
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
SYSTEM AND METHOD FOR LEARNING DRIVING INFORMATION IN VEHICLE | ||
Patent #
US 20150189241A1
Filed 03/13/2014
|
Current Assignee
Electronics and Telecommunications Research Institute
|
Original Assignee
Electronics and Telecommunications Research Institute
|
Vehicle operator and/or occupant information apparatus and method | ||
Patent #
US 9,075,136 B1
Filed 03/01/1999
|
Current Assignee
NavBlazer LLC
|
Original Assignee
GTJ Ventures LLC
|
Vehicle access and security based on biometrics | ||
Patent #
US 9,082,237 B2
Filed 01/24/2014
|
Current Assignee
Intelligent Technologies International Inc.
|
Original Assignee
Intelligent Technologies International Inc.
|
Multi sensor detection, stall to stop and lock disabling system | ||
Patent #
US 9,096,189 B2
Filed 09/09/2013
|
Current Assignee
Larry Golden
|
Original Assignee
Larry Golden
|
REMOTE CONTROL SEATBELT HEIGHT ADJUSTER | ||
Patent #
US 20150232061A1
Filed 02/17/2014
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Roadway-induced ride quality reconnaissance and route planning | ||
Patent #
US 9,109,913 B2
Filed 09/30/2013
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Increasing vehicle security | ||
Patent #
US 9,114,719 B1
Filed 05/12/2014
|
Current Assignee
Bryan Marc Failing
|
Original Assignee
Bryan Marc Failing
|
OBJECT IDENTIFIER | ||
Patent #
US 20150242694A1
Filed 06/20/2013
|
Current Assignee
Honda Motor Company
|
Original Assignee
Honda Motor Company
|
Medical data acquisition and provision | ||
Patent #
US 9,122,775 B2
Filed 01/03/2011
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
METHOD AND APPARATUS FOR CONTROLLING AN ACTUATABLE RESTRAINING DEVICE USING MULTI-REGION ENHANCED DISCRIMINATION | ||
Patent #
US 20150266439A1
Filed 12/03/2013
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Asset monitoring system using multiple imagers | ||
Patent #
US 9,151,692 B2
Filed 01/03/2008
|
Current Assignee
Intelligent Technologies International Inc.
|
Original Assignee
Intelligent Technologies International Inc.
|
Methods and systems for sharing position data between subscribers involving multiple wireless providers | ||
Patent #
US 9,167,558 B2
Filed 06/12/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
VEHICLE NAVIGATION | ||
Patent #
US 20150316391A1
Filed 12/24/2013
|
Current Assignee
Harman International Industries Incorporated
|
Original Assignee
Ping Zhou, Wenping Xiao, Xiaopei Huang
|
Apparatus and method to transmit content to a cellular wireless device based on proximity to other wireless devices | ||
Patent #
US 9,185,522 B1
Filed 11/07/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Rear vision system for a vehicle | ||
Patent #
US 9,183,749 B2
Filed 03/09/2015
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Systems and methods for predicting weather performance for a vehicle | ||
Patent #
US 9,187,099 B2
Filed 10/16/2014
|
Current Assignee
Fathym Inc.
|
Original Assignee
Richard M. Powers, Reeves E. Duer, John C. Mickey
|
Vehicle emergency evacuation device | ||
Patent #
US 9,205,816 B2
Filed 07/11/2011
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Vehicle system reaction to medical conditions | ||
Patent #
US 9,208,289 B2
Filed 02/27/2014
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Hybrid wireless-wired architecture based on power lines for intra-vehicular communication | ||
Patent #
US 9,227,579 B1
Filed 07/02/2014
|
Current Assignee
GM Global Technology Operations LLC
|
Original Assignee
GM Global Technology Operations LLC
|
Eyewear with exchangeable temples housing a radio frequency transceiver | ||
Patent #
US 9,244,292 B2
Filed 07/02/2004
|
Current Assignee
III Holdings 4 LLC
|
Original Assignee
III Holdings 4 LLC
|
DISTANCE MEASUREMENT | ||
Patent #
US 20160025836A1
Filed 06/23/2015
|
Current Assignee
Goodix Technology Co. Ltd.
|
Original Assignee
NXP B.V.
|
Apparatus and method for obtaining content on a cellular wireless device based on proximity | ||
Patent #
US 9,253,616 B1
Filed 03/24/2015
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
SYSTEMS AND METHODS FOR DETECTING OBSTRUCTIONS IN A CAMERA FIELD OF VIEW | ||
Patent #
US 20160031372A1
Filed 10/12/2015
|
Current Assignee
MobilEye Vision Technologies Ltd.
|
Original Assignee
MobilEye Vision Technologies Ltd.
|
Apparatus and method for data transmissions in a tire pressure monitor | ||
Patent #
US 9,259,980 B2
Filed 10/27/2014
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Application of smooth pursuit cognitive testing paradigms to clinical drug development | ||
Patent #
US 9,265,458 B2
Filed 12/04/2012
|
Current Assignee
Sync Think Inc.
|
Original Assignee
Sync Think Inc.
|
MULTI-CAMERA VISION SYSTEM FOR A VEHICLE | ||
Patent #
US 20160059782A1
Filed 11/09/2015
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Object identifier | ||
Patent #
US 9,292,751 B2
Filed 06/20/2013
|
Current Assignee
Honda Motor Company
|
Original Assignee
Honda Motor Company
|
Foreign object detection system and method suitable for source resonator of wireless energy transfer system | ||
Patent #
US 9,304,042 B2
Filed 01/18/2013
|
Current Assignee
Aptiv Technologies Limited
|
Original Assignee
Delphi Technologies Inc.
|
METHOD FOR CONTROLLING A TIME OF ACTIVATION OF A REVERSIBLE RESTRAINT SYSTEM IN A VEHICLE | ||
Patent #
US 20160129881A1
Filed 10/28/2015
|
Current Assignee
Volvo Car Corporation
|
Original Assignee
Volvo Car Corporation
|
Optical spread spectrum detection and ranging | ||
Patent #
US 9,354,318 B1
Filed 03/05/2015
|
Current Assignee
Horizon Hobby Llc
|
Original Assignee
Horizon Hobby Llc
|
Systems and methods of creating and delivering item of manufacture specific information to remote devices | ||
Patent #
US 9,367,973 B2
Filed 07/31/2014
|
Current Assignee
Tweddle Group Incorporated
|
Original Assignee
Tweddle Group Incorporated
|
Optical neuroinformatics | ||
Patent #
US 9,380,976 B2
Filed 03/11/2013
|
Current Assignee
Sync Think Inc.
|
Original Assignee
Sync Think Inc.
|
Energy transfer with vehicles | ||
Patent #
US 9,393,878 B1
Filed 09/23/2014
|
Current Assignee
Bryan Marc Failing
|
Original Assignee
Bryan Marc Failing
|
VEHICLE CONTROL APPARATUS | ||
Patent #
US 20160229394A1
Filed 09/02/2014
|
Current Assignee
Hitachi Automotive Systems Limited
|
Original Assignee
Hitachi Automotive Systems Limited
|
Sonar Data Collection | ||
Patent #
US 20160146936A1
Filed 11/26/2014
|
Current Assignee
Navico Holding A.S.
|
Original Assignee
Navico Holding A.S.
|
Remote control seatbelt height adjuster | ||
Patent #
US 9,421,944 B2
Filed 02/17/2014
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
3D sonar display with semi-transparent shading | ||
Patent #
US 9,423,259 B2
Filed 08/01/2014
|
Current Assignee
Garmin Switzerland GmbH
|
Original Assignee
Garmin Switzerland GmbH
|
Multi-camera vision system for a vehicle | ||
Patent #
US 9,440,586 B2
Filed 11/09/2015
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Automatic passenger airbag switch | ||
Patent #
US 9,446,730 B1
Filed 11/08/2015
|
Current Assignee
Thunder Power Hong Kong Limited
|
Original Assignee
Thunder Power Hong Kong Limited
|
Methods and apparatus for adaptive vehicle response to air quality states | ||
Patent #
US 9,449,514 B2
Filed 05/18/2011
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Pressure check tool and method of operating the same | ||
Patent #
US 9,446,636 B2
Filed 02/26/2014
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Methods and systems for temporarily sharing position data between mobile-device users | ||
Patent #
US 9,467,832 B2
Filed 09/05/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Method and Apparatus for Determining the Operation of a Vehicle Safety System | ||
Patent #
US 20160300106A1
Filed 04/09/2015
|
Current Assignee
Bendix Commercial Vehicle Systems LLC
|
Original Assignee
Bendix Commercial Vehicle Systems LLC
|
Wink detecting camera | ||
Patent #
US 9,485,403 B2
Filed 11/12/2014
|
Current Assignee
Cutting Edge Vision LLC
|
Original Assignee
Cutting Edge Vision LLC
|
System and method for learning driving information in vehicle | ||
Patent #
US 9,485,474 B2
Filed 03/13/2014
|
Current Assignee
Electronics and Telecommunications Research Institute
|
Original Assignee
Electronics and Telecommunications Research Institute
|
BUILDING PROFILES ASSOCIATED WITH VEHICLE USERS | ||
Patent #
US 20160318467A1
Filed 05/02/2016
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
AutoConnect Holdings LLC
|
HEALTH STATISTICS AND COMMUNICATIONS OF ASSOCIATED VEHICLE USERS | ||
Patent #
US 20160318468A1
Filed 05/02/2016
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
AutoConnect Holdings LLC
|
Method and apparatus for image fusion | ||
Patent #
US 9,501,852 B2
Filed 01/15/2014
|
Current Assignee
Nokia Technologies Oy
|
Original Assignee
Nokia Technologies Oy
|
Vehicle control system | ||
Patent #
US 9,506,558 B2
Filed 04/06/2016
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
RF transmission method and apparatus in a tire pressure monitoring system | ||
Patent #
US 9,517,664 B2
Filed 02/20/2015
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
DEVICE, SYSTEM AND METHOD FOR REDUCING PHYSICAL IMPACTS | ||
Patent #
US 20170002593A1
Filed 07/29/2015
|
Current Assignee
Cloud Network Technology Singapore Pte. Ltd.
|
Original Assignee
Hon Hai Precision Industry Co. Ltd.
|
COMMUNICATION SYSTEM FOR GATHERING AND VERIFYING INFORMATION | ||
Patent #
US 20170024621A1
Filed 07/15/2016
|
Current Assignee
Dura Operating LLC
|
Original Assignee
Dura Operating LLC
|
Optical spread spectrum detection and ranging | ||
Patent #
US 9,575,185 B2
Filed 05/27/2016
|
Current Assignee
Horizon Hobby Llc
|
Original Assignee
Horizon Hobby Llc
|
Cell phone having a monolithically integrated multi-sensor device on a semiconductor substrate and method therefor | ||
Patent #
US 9,580,302 B2
Filed 03/12/2014
|
Current Assignee
Versana Micro Inc.
|
Original Assignee
Versana Micro Inc.
|
Rendez vous management using mobile phones or other mobile devices | ||
Patent #
US 9,584,960 B1
Filed 12/23/2013
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
SYSTEM FOR USE IN A VEHICLE | ||
Patent #
US 20170059703A1
Filed 02/04/2015
|
Current Assignee
Jaguar Land Rover Automotive PLC
|
Original Assignee
Jaguar Land Rover Automotive PLC
|
Multi sensor detection, stall to stop and lock disabling system | ||
Patent #
US 9,589,439 B2
Filed 07/23/2015
|
Current Assignee
Larry Golden
|
Original Assignee
Larry Golden
|
Interactivity with a mixed reality | ||
Patent #
US 9,600,935 B2
Filed 02/19/2014
|
Current Assignee
Nant Holdings IP LLC
|
Original Assignee
Nant Holdings IP LLC
|
System and method for determining a position of a living being in a vehicle | ||
Patent #
US 9,612,322 B2
Filed 05/19/2014
|
Current Assignee
Volvo Car Corporation
|
Original Assignee
Volvo Car Corporation
|
Techniques for communication within closed groups of mobile devices | ||
Patent #
US 9,615,204 B1
Filed 07/22/2015
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
METHOD AND APPARATUS FOR INDIVIDUALIZED THREE DIMENSIONAL DISPLAY CALIBRATION | ||
Patent #
US 20170099482A1
Filed 10/02/2015
|
Current Assignee
Atheer Inc.
|
Original Assignee
Atheer Inc.
|
Systems and methods for predicting weather performance for a vehicle | ||
Patent #
US 9,616,897 B2
Filed 10/14/2015
|
Current Assignee
Fathym Inc.
|
Original Assignee
Fathym Inc.
|
SYSTEMS, METHODS, AND APPARATUS FOR DISPENSING AMBIENT, COLD, AND CARBONATED WATER | ||
Patent #
US 20170121165A1
Filed 01/17/2017
|
Current Assignee
Aqueduct Holdings Limited
|
Original Assignee
Aqueduct Holdings Limited
|
METHOD AND DEVICE FOR TESTING SAFETY INSIDE VEHICLE | ||
Patent #
US 20170124831A1
Filed 12/28/2015
|
Current Assignee
Leauto Intelligent Technology Beijing Co. Ltd.
|
Original Assignee
Leauto Intelligent Technology Beijing Co. Ltd.
|
UNIVERSAL BUCK FOR SLED TEST | ||
Patent #
US 20170131182A1
Filed 07/01/2016
|
Current Assignee
Hyundai Motor Company
|
Original Assignee
Hyundai Motor Company
|
Method and apparatus for controlling an actuatable restraining device using multi-region enhanced discrimination | ||
Patent #
US 9,650,006 B2
Filed 12/03/2013
|
Current Assignee
TRW Automotive USLLC
|
Original Assignee
TRW Automotive USLLC
|
Techniques for sharing position data between first and second devices | ||
Patent #
US 9,654,921 B1
Filed 09/20/2016
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
APPARATUS AND METHOD FOR SENDING VEHICLE BROADCAST INFORMATION | ||
Patent #
US 20170141861A1
Filed 01/27/2017
|
Current Assignee
Huawei Technologies Co. Ltd.
|
Original Assignee
Huawei Technologies Co. Ltd.
|
AUTOMATED ENTRY | ||
Patent #
US 20170161979A1
Filed 02/15/2017
|
Current Assignee
Consumer 2.0 Inc.
|
Original Assignee
Merrick Schmidt-Lackner, Clark Zhe Li
|
Tire pressure monitor system apparatus and method | ||
Patent #
US 9,676,238 B2
Filed 11/06/2015
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Systems and methods of creating and delivering item of manufacture specific information to remote devices | ||
Patent #
US 9,679,423 B2
Filed 12/14/2015
|
Current Assignee
Tweddle Group Incorporated
|
Original Assignee
Tweddle Group Incorporated
|
Communication module, vehicle including the same, and method for controlling the vehicle | ||
Patent #
US 9,699,290 B2
Filed 03/18/2016
|
Current Assignee
Hyundai Motor Company
|
Original Assignee
Hyundai Motor Company
|
Automatic passenger airbag switch | ||
Patent #
US 9,725,061 B2
Filed 12/14/2015
|
Current Assignee
Thunder Power Hong Kong Limited
|
Original Assignee
Thunder Power New Energy Vehicle Development Company Limited
|
IMAGING SYSTEM FOR FUEL TANK ANALYSIS | ||
Patent #
US 20170230635A1
Filed 02/04/2016
|
Current Assignee
Simmonds Precision Products Inc.
|
Original Assignee
Simmonds Precision Products Inc.
|
Techniques for sharing relative position between mobile devices | ||
Patent #
US 9,736,618 B1
Filed 07/16/2015
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Rendez vous management using mobile phones or other mobile devices | ||
Patent #
US 9,749,790 B1
Filed 01/12/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Apparatus and method for sending vehicle broadcast information | ||
Patent #
US 9,755,771 B2
Filed 01/27/2017
|
Current Assignee
Huawei Technologies Co. Ltd.
|
Original Assignee
Huawei Technologies Co. Ltd.
|
Method for detecting the detachment of a sensor device mounted in a wheel of a vehicle | ||
Patent #
US 9,773,359 B2
Filed 12/04/2015
|
Current Assignee
Continental Automotive GmbH
|
Original Assignee
Continental Automotive France, Continental Automotive GmbH
|
Vehicle component including a lighting system | ||
Patent #
US 9,776,560 B1
Filed 03/23/2016
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan North America Incorporated
|
Apparatus and method for data transmissions in a tire pressure monitor | ||
Patent #
US 9,776,463 B2
Filed 08/11/2015
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Measuring distance from object by using size of pattern projected onto object | ||
Patent #
US 9,784,577 B2
Filed 03/15/2013
|
Current Assignee
LG Innotek Company Limited
|
Original Assignee
LG Innotek Company Limited
|
Multi-camera vision system for a vehicle | ||
Patent #
US 9,796,331 B2
Filed 09/12/2016
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
EYEBALL MOVEMENT DETECTION DEVICE | ||
Patent #
US 20170312622A1
Filed 08/26/2015
|
Current Assignee
Sharp Electronics Corporation
|
Original Assignee
Sharp Electronics Corporation
|
Device, system and method for reducing physical impacts | ||
Patent #
US 9,840,865 B2
Filed 07/29/2015
|
Current Assignee
Cloud Network Technology Singapore Pte. Ltd.
|
Original Assignee
Hon Hai Precision Industry Co. Ltd.
|
Formation of wireless device location sharing group | ||
Patent #
US 9,854,402 B1
Filed 01/27/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Ad hoc location sharing group between first and second cellular wireless devices | ||
Patent #
US 9,854,394 B1
Filed 02/09/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Gas spring compensation marine acoustic vibrator | ||
Patent #
US 9,864,080 B2
Filed 12/31/2013
|
Current Assignee
PGS Geophysical As
|
Original Assignee
PGS Geophysical As
|
DYNAMIC LAYERS FOR NAVIGATION DATABASE SYSTEMS | ||
Patent #
US 20180023964A1
Filed 07/18/2017
|
Current Assignee
Harman Becker Automotive Systems GmbH
|
Original Assignee
Harman International Industries Incorporated
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 9,880,635 B2
Filed 04/28/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Automated entry | ||
Patent #
US 9,881,347 B2
Filed 07/17/2012
|
Current Assignee
Consumer 2.0 Inc.
|
Original Assignee
Consumer 2.0 Inc.
|
Rendez vous management using mobile phones or other mobile devices | ||
Patent #
US 9,883,360 B1
Filed 01/05/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Systems and methods for predicting weather performance for a vehicle | ||
Patent #
US 9,903,728 B2
Filed 03/07/2017
|
Current Assignee
Fathym Inc.
|
Original Assignee
Fathym Inc.
|
Vehicle-to-pedestrian communication systems | ||
Patent #
US 9,928,734 B2
Filed 08/29/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Pictures using voice commands and automatic upload | ||
Patent #
US 9,936,116 B2
Filed 06/21/2016
|
Current Assignee
Cutting Edge Vision LLC
|
Original Assignee
Cutting Edge Vision LLC
|
Location sharing group for services provision | ||
Patent #
US 9,942,705 B1
Filed 02/09/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Vehicle with a soft-touch antenna for communicating sensitive information | ||
Patent #
US 9,946,906 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Methods, systems and apparatuses for the formation and tracking of location sharing groups | ||
Patent #
US 9,955,298 B1
Filed 08/28/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Method and system for authentication in autonomous vehicles | ||
Patent #
US 9,963,106 B1
Filed 12/28/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Methods and systems for locating health facilities based on cost of healthcare | ||
Patent #
US 9,964,416 B2
Filed 02/04/2011
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Location sharing group map management | ||
Patent #
US 9,967,704 B1
Filed 01/27/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
INTERFERENCE MITIGATION VIA ADAPTIVE DEPTH IMAGING | ||
Patent #
US 20180130209A1
Filed 11/04/2016
|
Current Assignee
Microsoft Technology Licensing LLC
|
Original Assignee
Denis Demandolx, Raymond Kirk Price, Michael Bleyer, Jian Zhao
|
Adaptive tracking system for spatial input devices | ||
Patent #
US 9,984,285 B2
Filed 07/07/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Vehicle identification or authentication | ||
Patent #
US 9,984,522 B2
Filed 10/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Method and system for sharing parking space availability among autonomous vehicles | ||
Patent #
US 9,984,572 B1
Filed 01/16/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Visual collaboration interface | ||
Patent #
US 9,990,046 B2
Filed 03/07/2016
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Facial recognition database created from social networking sites | ||
Patent #
US 9,994,229 B2
Filed 04/25/2016
|
Current Assignee
AutoConnect Holdings LLC
|
Original Assignee
AutoConnect Holdings LLC
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 10,011,247 B2
Filed 04/23/2003
|
Current Assignee
GTJ Ventures LLC
|
Original Assignee
GTJ Ventures LLC
|
Systems and methods for detecting obstructions in a camera field of view | ||
Patent #
US 10,011,226 B2
Filed 10/12/2015
|
Current Assignee
MobilEye Vision Technologies Ltd.
|
Original Assignee
MobilEye Vision Technologies Ltd.
|
Systems and methods for illumination control and distribution during a vehicle bank | ||
Patent #
US 10,023,103 B2
Filed 05/08/2014
|
Current Assignee
J.W. Speaker Corporation
|
Original Assignee
J.W. Speaker Corporation
|
Method and system for using weather information in operation of autonomous vehicles | ||
Patent #
US 10,031,521 B1
Filed 01/16/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Method and system for behavioral sharing in autonomous vehicles | ||
Patent #
US 10,031,523 B2
Filed 12/28/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Bifurcated communications to a third party through a vehicle | ||
Patent #
US 10,032,319 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
DETECTION APPARATUS, IMAGING APPARATUS, VEHICLE, AND DETECTION METHOD | ||
Patent #
US 20180208114A1
Filed 07/29/2016
|
Current Assignee
Kyocera Corporation
|
Original Assignee
Kyocera Corporation
|
Method for controlling a time of activation of a reversible restraint system in a vehicle | ||
Patent #
US 10,035,491 B2
Filed 10/28/2015
|
Current Assignee
Volvo Car Corporation
|
Original Assignee
Volvo Car Corporation
|
Universal buck for sled test | ||
Patent #
US 10,036,687 B2
Filed 07/01/2016
|
Current Assignee
Hyundai Motor Company
|
Original Assignee
Hyundai Motor Company
|
SURROUNDINGS MONITORING APPARATUS | ||
Patent #
US 20180229657A1
Filed 09/14/2016
|
Current Assignee
Aisin Seiki Co. Ltd.
|
Original Assignee
Aisin Seiki Co. Ltd.
|
Distance measurement | ||
Patent #
US 10,061,010 B2
Filed 06/23/2015
|
Current Assignee
Goodix Technology Co. Ltd.
|
Original Assignee
NXP B.V.
|
Control system for navigating a principal dimension of a data space | ||
Patent #
US 10,061,392 B2
Filed 09/19/2016
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Automatic upload of pictures from a camera | ||
Patent #
US 10,063,761 B2
Filed 11/24/2015
|
Current Assignee
Cutting Edge Vision LLC
|
Original Assignee
Cutting Edge Vision LLC
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 10,067,571 B2
Filed 12/15/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Optical spread spectrum detection and ranging | ||
Patent #
US 10,071,733 B2
Filed 01/11/2017
|
Current Assignee
Horizon Hobby Llc
|
Original Assignee
Horizon Hobby Llc
|
Secured vehicle for user use only | ||
Patent #
US 10,074,223 B2
Filed 01/13/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
PROXIMITY SWITCH AND HUMIDITY SENSOR ASSEMBLY | ||
Patent #
US 20180265043A1
Filed 03/14/2017
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Method and system for collective autonomous operation database for autonomous vehicles | ||
Patent #
US 10,083,604 B2
Filed 12/29/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Eyeball movement detection device | ||
Patent #
US 10,086,268 B2
Filed 08/26/2015
|
Current Assignee
Sharp Electronics Corporation
|
Original Assignee
Sharp Electronics Corporation
|
SYSTEMS AND METHODS FOR ILLUMINATION CONTROL AND DISTRIBUTION DURING A VEHICLE BANK | ||
Patent #
US 20180297509A1
Filed 06/08/2018
|
Current Assignee
J.W. Speaker Corporation
|
Original Assignee
J.W. Speaker Corporation
|
Steerable rotating projectile | ||
Patent #
US 10,118,696 B1
Filed 03/31/2017
|
Current Assignee
Steven M Hoffberg
|
Original Assignee
Steven M Hoffberg
|
Configuration of vehicle compartments | ||
Patent #
US 10,118,577 B1
Filed 08/10/2017
|
Current Assignee
Uatc LLC
|
Original Assignee
Uber Technologies Inc.
|
Energy transfer with vehicles | ||
Patent #
US 10,124,691 B1
Filed 09/23/2014
|
Current Assignee
Bryan Marc Failing
|
Original Assignee
Bryan Marc Failing
|
Multi-camera vision system for a vehicle | ||
Patent #
US 10,144,353 B2
Filed 10/23/2017
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Location sharing service between GPS-enabled wireless devices, with shared target location exchange | ||
Patent #
US 10,149,092 B1
Filed 02/09/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
COLLISION DETERMINATION SYSTEM, COLLISION DETERMINATION TERMINAL, AND COMPUTER PROGRAM PRODUCT | ||
Patent #
US 20180350241A1
Filed 10/04/2016
|
Current Assignee
DENSO Corporation
|
Original Assignee
DENSO Corporation
|
Control, monitoring, and/or security apparatus and method | ||
Patent #
US 10,152,876 B2
Filed 07/13/2005
|
Current Assignee
GTJ Ventures LLC
|
Original Assignee
GTJ Ventures LLC
|
Methods, systems and apparatuses for the formation and tracking of location sharing groups | ||
Patent #
US 10,165,059 B2
Filed 10/24/2014
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
SENSOR MOUNTING STATE DETERMINATION DEVICE AND SENSOR MOUNTING STATE DETERMINATION METHOD | ||
Patent #
US 20190018122A1
Filed 09/14/2018
|
Current Assignee
Panasonic Intellectual Property Management Co. Ltd.
|
Original Assignee
Panasonic Intellectual Property Corporation of America
|
Map presentation on cellular device showing positions of multiple other wireless device users | ||
Patent #
US 10,200,811 B1
Filed 02/09/2017
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
NEURAL NETWORK METHOD AND APPARATUS | ||
Patent #
US 20190051291A1
Filed 12/07/2017
|
Current Assignee
Samsung Electronics Co. Ltd.
|
Original Assignee
Samsung Electronics Co. Ltd.
|
Apparatus, system and method for configuring a tire information sensor with a transmission protocol based on vehicle trigger characteristics | ||
Patent #
US 10,220,660 B2
Filed 08/01/2016
|
Current Assignee
Continental Automotive Systems Inc.
|
Original Assignee
Continental Automotive Systems Inc.
|
Vehicle occupant detection system | ||
Patent #
US 10,229,572 B1
Filed 06/14/2017
|
Current Assignee
Elizabeth Casey, Mark Casey
|
Original Assignee
Elizabeth Casey, Mark Casey
|
Adaptive route and motion planning based on learned external and internal vehicle environment | ||
Patent #
US 10,234,302 B2
Filed 06/27/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes | ||
Patent #
US 10,235,412 B2
Filed 08/25/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Lease observation and event recording | ||
Patent #
US 10,249,104 B2
Filed 12/29/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Adaptive tracking system for spatial input devices | ||
Patent #
US 10,255,489 B2
Filed 03/20/2018
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Pictures using voice commands | ||
Patent #
US 10,257,401 B2
Filed 11/24/2015
|
Current Assignee
Cutting Edge Vision LLC
|
Original Assignee
Cutting Edge Vision LLC
|
Conditional or temporary feature availability | ||
Patent #
US 10,262,469 B2
Filed 10/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Machine learning for personalized driving | ||
Patent #
US 10,286,915 B2
Filed 01/17/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 10,296,099 B2
Filed 05/17/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Apparatus and method for adjusting driving position of driver | ||
Patent #
US 10,298,722 B2
Filed 06/08/2016
|
Current Assignee
Hyundai Motor Company
|
Original Assignee
Hyundai Motor Company
|
Server-implemented methods and systems for sharing location amongst web-enabled cell phones | ||
Patent #
US 10,299,071 B2
Filed 11/24/2018
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Duplicated wireless transceivers associated with a vehicle to receive and send sensitive information | ||
Patent #
US 10,304,261 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
METHOD FOR DETECTING THE DETACHMENT OF A SENSOR DEVICE MOUNTED IN A WHEEL OF A VEHICLE | ||
Patent #
US 20160163134A1
Filed 12/04/2015
|
Current Assignee
Continental Automotive GmbH
|
Original Assignee
Continental Automotive France, Continental Automotive GmbH
|
Rear backup system for a vehicle | ||
Patent #
US 10,308,180 B2
Filed 11/29/2018
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Location sharing and map support in connection with services request | ||
Patent #
US 10,313,826 B2
Filed 12/21/2018
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Imaging system for fuel tank analysis | ||
Patent #
US 10,326,980 B2
Filed 02/04/2016
|
Current Assignee
Simmonds Precision Products Inc.
|
Original Assignee
Simmonds Precision Products Inc.
|
Navigation of autonomous vehicles to enhance safety under one or more fault conditions | ||
Patent #
US 10,338,594 B2
Filed 03/13/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Visual collaboration interface | ||
Patent #
US 10,338,693 B2
Filed 04/10/2018
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Location sharing with facilitated meeting point definition | ||
Patent #
US 10,341,809 B2
Filed 12/13/2018
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Location sharing for commercial and proprietary content applications | ||
Patent #
US 10,341,808 B2
Filed 11/27/2018
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 10,353,483 B2
Filed 08/01/2018
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Methods and systems for associating sensitive information of a passenger with a vehicle | ||
Patent #
US 10,354,460 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Mobile terminal for sharing image with nearby blackbox device and method for controlling the same | ||
Patent #
US 10,354,530 B2
Filed 12/22/2015
|
Current Assignee
LG Electronics Inc.
|
Original Assignee
LG Electronics Inc.
|
Automated passenger airbag switch | ||
Patent #
US 10,358,104 B2
Filed 07/26/2016
|
Current Assignee
Thunder Power Hong Kong Limited
|
Original Assignee
Thunder Power New Energy Vehicle Development Company Limited
|
System and method for generating vehicle crash data | ||
Patent #
US 10,360,742 B1
Filed 06/30/2016
|
Current Assignee
State Farm Mutual Automobile Insurance Company
|
Original Assignee
State Farm Mutual Automobile Insurance Company
|
Method and apparatus for determining the operation of a vehicle safety system | ||
Patent #
US 10,366,285 B2
Filed 04/09/2015
|
Current Assignee
Bendix Commercial Vehicle Systems LLC
|
Original Assignee
Bendix Commercial Vehicle Systems LLC
|
Method and apparatus for individualized three dimensional display calibration | ||
Patent #
US 10,368,059 B2
Filed 10/02/2015
|
Current Assignee
Atheer Inc.
|
Original Assignee
Atheer Inc.
|
Controlling access to a vehicle using wireless access devices | ||
Patent #
US 10,369,966 B1
Filed 05/23/2018
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Control and coordination of driverless fuel replenishment for autonomous vehicles | ||
Patent #
US 10,369,974 B2
Filed 07/14/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Secure communications with sensitive user information through a vehicle | ||
Patent #
US 10,388,081 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Image monitoring apparatus, image display system, and vehicle | ||
Patent #
US 10,397,564 B2
Filed 05/16/2017
|
Current Assignee
ROHM Co. Ltd.
|
Original Assignee
ROHM Co. Ltd.
|
Vehicle navigation | ||
Patent #
US 10,408,632 B2
Filed 12/24/2013
|
Current Assignee
Harman International Industries Incorporated
|
Original Assignee
Harman International Industries Incorporated
|
Vehicle autonomy level selection based on user context | ||
Patent #
US 10,410,250 B2
Filed 12/30/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
System for tracking and identifying vehicles and pedestrians | ||
Patent #
US 10,410,064 B2
Filed 12/01/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Autonomous vehicle logic to identify permissioned parking relative to multiple classes of restricted parking | ||
Patent #
US 10,423,162 B2
Filed 05/08/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
System and method for driving hazard estimation using vehicle-to-vehicle communication | ||
Patent #
US 10,424,203 B2
Filed 01/29/2016
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Toyota Motor Engineering Manufacturing North America Incorporated
|
Imaging system for fuel tank analysis | ||
Patent #
US 10,424,076 B2
Filed 02/04/2016
|
Current Assignee
Simmonds Precision Products Inc.
|
Original Assignee
Simmonds Precision Products Inc.
|
Interactivity with a mixed reality | ||
Patent #
US 10,463,961 B2
Filed 11/03/2013
|
Current Assignee
Nant Holdings IP LLC
|
Original Assignee
Nant Holdings IP LLC
|
Voice biometric pre-purchase enrollment for autonomous vehicles | ||
Patent #
US 10,464,530 B2
Filed 01/17/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Self-destruct zone and autonomous vehicle navigation | ||
Patent #
US 10,471,829 B2
Filed 01/16/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Air-spring compensation in a piston-type marine vibrator | ||
Patent #
US 10,488,536 B2
Filed 05/12/2016
|
Current Assignee
PGS Geophysical As
|
Original Assignee
PGS Geophysical As
|
Neural network method and apparatus | ||
Patent #
US 10,504,506 B2
Filed 12/07/2017
|
Current Assignee
Samsung Electronics Co. Ltd.
|
Original Assignee
Samsung Electronics Co. Ltd.
|
Method and system for data optimization | ||
Patent #
US 10,515,390 B2
Filed 12/28/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes | ||
Patent #
US 10,521,021 B2
Filed 02/01/2019
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Spatially mediated augmentations of and interactions among distinct devices and applications via extended pixel manifold | ||
Patent #
US 10,529,302 B2
Filed 07/06/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
System and method for infrasonic cardiac monitoring | ||
Patent #
US 10,542,961 B2
Filed 06/14/2016
|
Current Assignee
The Research Foundation for The State University of New York
|
Original Assignee
The Research Foundation for The State University of New York
|
Control, monitoring, and/or security, apparatus and method for premises, vehicles, and/or articles | ||
Patent #
US 10,546,441 B2
Filed 05/20/2014
|
Current Assignee
Raymond Anthony Joao
|
Original Assignee
Raymond Anthony Joao
|
Rear backup system for a vehicle | ||
Patent #
US 10,556,542 B2
Filed 06/03/2019
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Driver's physical condition detection device and method | ||
Patent #
US 10,561,356 B2
Filed 08/22/2017
|
Current Assignee
Mazda Motor Corporation
|
Original Assignee
Mazda Motor Corporation
|
Control, monitoring and/or security apparatus and method | ||
Patent #
US 10,562,492 B2
Filed 04/23/2003
|
Current Assignee
GTJ Ventures LLC
|
Original Assignee
GTJ Ventures LLC
|
Accelerometer-based external sound monitoring during low speed maneuvers | ||
Patent #
US 10,562,449 B2
Filed 09/25/2017
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Multi-process interactive systems and methods | ||
Patent #
US 10,565,030 B2
Filed 09/30/2016
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Sensor mounting state determination device and sensor mounting state determination method | ||
Patent #
US 10,571,553 B2
Filed 09/14/2018
|
Current Assignee
Panasonic Intellectual Property Management Co. Ltd.
|
Original Assignee
Panasonic Intellectual Property Management Co. Ltd.
|
Collision determination system, collision determination terminal, and computer program product for determining possibility of collision | ||
Patent #
US 10,573,181 B2
Filed 10/04/2016
|
Current Assignee
DENSO Corporation
|
Original Assignee
DENSO Corporation
|
Systems and methods for operator hand position adjustment for airbag deployment | ||
Patent #
US 10,577,008 B2
Filed 10/19/2017
|
Current Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Toyota Research Institute Inc.
|
Access point environment characterization | ||
Patent #
US 10,595,162 B2
Filed 03/18/2016
|
Current Assignee
Qualcomm Inc.
|
Original Assignee
Qualcomm Inc.
|
Visual place recognition based self-localization for autonomous vehicles | ||
Patent #
US 10,606,274 B2
Filed 10/30/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Interactivity with a mixed reality | ||
Patent #
US 10,617,951 B2
Filed 02/06/2017
|
Current Assignee
Nant Holdings IP LLC
|
Original Assignee
Nant Holdings IP LLC
|
Methods and systems for multiple access to a single hardware data stream | ||
Patent #
US 10,623,721 B2
Filed 06/12/2019
|
Current Assignee
Atheer Inc.
|
Original Assignee
Atheer Inc.
|
User authentication based on telematics information | ||
Patent #
US 10,623,401 B1
Filed 01/06/2017
|
Current Assignee
Allstate Insurance Company
|
Original Assignee
Allstate Insurance Company
|
Visual collaboration interface | ||
Patent #
US 10,627,915 B2
Filed 05/21/2019
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Surroundings monitoring apparatus | ||
Patent #
US 10,632,915 B2
Filed 09/14/2016
|
Current Assignee
Aisin Seiki Co. Ltd.
|
Original Assignee
Aisin Seiki Co. Ltd.
|
Systems and methods for detecting obstructions in a camera field of view | ||
Patent #
US 10,632,916 B2
Filed 06/13/2018
|
Current Assignee
MobilEye Vision Technologies Ltd.
|
Original Assignee
MobilEye Vision Technologies Ltd.
|
Vehicle path-planner monitor and controller | ||
Patent #
US 10,635,109 B2
Filed 10/17/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Driving consciousness estimation device | ||
Patent #
US 10,640,122 B2
Filed 04/05/2017
|
Current Assignee
National Institute of Advanced Industrial Science and Technology, Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Original Assignee
Ibaraki Toyota Jidosha Kabushiki Kaisha
|
Obtaining identifying information when intrusion is detected | ||
Patent #
US 10,647,300 B2
Filed 06/29/2018
|
Current Assignee
Toyota Motor Engineering Manufacturing North America Incorporated
|
Original Assignee
Toyota Motor Engineering Manufacturing North America Incorporated
|
Operating environment comprising multiple client devices, multiple displays, multiple users, and gestural control | ||
Patent #
US 10,656,724 B2
Filed 03/12/2018
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Proteins, pools, and slawx in processing environments | ||
Patent #
US 10,664,327 B2
Filed 09/14/2017
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Detection apparatus, imaging apparatus, vehicle, and detection method | ||
Patent #
US 10,668,855 B2
Filed 07/29/2016
|
Current Assignee
Kyocera Corporation
|
Original Assignee
Kyocera Corporation
|
Methods and systems for automatically sending rule-based communications from a vehicle | ||
Patent #
US 10,672,060 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Systems and methods for illumination control and distribution during a vehicle bank | ||
Patent #
US 10,676,018 B2
Filed 06/08/2018
|
Current Assignee
J.W. Speaker Corporation
|
Original Assignee
J.W. Speaker Corporation
|
Methods and systems for communicating estimated time of arrival to a third party | ||
Patent #
US 10,679,276 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
System and method for associating user and vehicle information for communication to a third party | ||
Patent #
US 10,685,503 B2
Filed 12/31/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Network-based system for selling and servicing cars | ||
Patent #
US 10,692,126 B2
Filed 07/05/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Using vehicle sensor data to monitor pedestrian health | ||
Patent #
US 10,694,357 B2
Filed 12/01/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Smart refill assistant for electric vehicles | ||
Patent #
US 10,699,305 B2
Filed 12/28/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
User-adjusted display devices and methods of operating the same | ||
Patent #
US 10,699,326 B2
Filed 12/28/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Vehicular multi-camera vision system including rear backup camera | ||
Patent #
US 10,703,274 B2
Filed 02/10/2020
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Using vehicle sensor data to monitor environmental and geologic conditions | ||
Patent #
US 10,708,547 B2
Filed 12/01/2016
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Control of complex parking maneuvers and autonomous fuel replenishment of driverless vehicles | ||
Patent #
US 10,710,633 B2
Filed 07/14/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Interference mitigation via adaptive depth imaging | ||
Patent #
US 10,712,561 B2
Filed 11/04/2016
|
Current Assignee
Microsoft Technology Licensing LLC
|
Original Assignee
Microsoft Technology Licensing LLC
|
Assessing historical telematic vehicle component maintenance records to identify predictive indicators of maintenance events | ||
Patent #
US 10,713,864 B2
Filed 12/19/2018
|
Current Assignee
GeoTab Incorporated
|
Original Assignee
GeoTab Incorporated
|
System and method for controlling a vehicle using secondary access methods | ||
Patent #
US 10,717,412 B2
Filed 11/13/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 10,739,865 B2
Filed 06/04/2019
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Temporary location sharing group with event based termination | ||
Patent #
US 10,750,310 B2
Filed 11/27/2019
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Application-based tracking and mapping function in connection with vehicle-based services provision | ||
Patent #
US 10,750,311 B2
Filed 02/04/2020
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Ad hoc location sharing group establishment for wireless devices with designated meeting point | ||
Patent #
US 10,750,309 B2
Filed 11/27/2019
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Method for adjusting the seat of a vehicle | ||
Patent #
US 10,773,609 B2
Filed 02/14/2018
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Operating systems for a vehicle storage device | ||
Patent #
US 10,787,129 B2
Filed 07/30/2018
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Location sharing for commercial and proprietary content applications | ||
Patent #
US 10,791,414 B2
Filed 04/23/2019
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Apparatus and method for providing shipment information | ||
Patent #
US 10,796,268 B2
Filed 01/05/2019
|
Current Assignee
Transcend Shipping Systems LLC
|
Original Assignee
GTJ Ventures LLC
|
Vehicular vision system including rear backup camera | ||
Patent #
US 10,800,331 B1
Filed 07/06/2020
|
Current Assignee
Magna Electronics Incorporated
|
Original Assignee
Magna Electronics Incorporated
|
Method and system for vehicle suspension system | ||
Patent #
US 10,801,923 B2
Filed 05/17/2018
|
Current Assignee
Ford Global Technologies LLC
|
Original Assignee
Ford Global Technologies LLC
|
Operating environment with gestural control and multiple client devices, displays, and users | ||
Patent #
US 10,824,238 B2
Filed 03/28/2019
|
Current Assignee
Oblong Industries Inc.
|
Original Assignee
Oblong Industries Inc.
|
Method and device for adapting a driving behavior of a semi, highly or fully automated vehicle | ||
Patent #
US 10,829,110 B2
Filed 10/11/2018
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Productive and accident-free driving modes for a vehicle | ||
Patent #
US 10,837,790 B2
Filed 08/01/2017
|
Current Assignee
Nio Usa Inc.
|
Original Assignee
Nio Usa Inc.
|
Approximate value iteration with complex returns by bounding | ||
Patent #
US 10,839,302 B2
Filed 11/22/2016
|
Current Assignee
The Research Foundation for The State University of New York
|
Original Assignee
The Research Foundation for The State University of New York
|
Vehicle seat belt monitoring | ||
Patent #
US 10,850,702 B2
Filed 03/18/2019
|
Current Assignee
Pony.ai Inc.
|
Original Assignee
Pony.ai Inc.
|
Application-based two-way tracking and mapping function with selected individuals | ||
Patent #
US 10,856,099 B2
Filed 02/04/2020
|
Current Assignee
X One Inc.
|
Original Assignee
X One Inc.
|
Method for classifying an occupant and providing the occupant classification for a safety device in a motor vehicle | ||
Patent #
US 10,882,418 B2
Filed 08/25/2017
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Vehicular occupant detection arrangements | ||
Patent #
US 6,553,296 B2
Filed 01/19/2001
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Method of operating a vehicle occupancy state sensor system | ||
Patent #
US 6,272,411 B1
Filed 09/30/1998
|
Current Assignee
Robert Bosch Corporation
|
Original Assignee
Robert Bosch Corporation
|
Methods for determining the identification and position of and monitoring objects in a vehicle | ||
Patent #
US 6,324,453 B1
Filed 12/30/1999
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Video occupant detection and classification | ||
Patent #
US 5,983,147 A
Filed 02/06/1997
|
Current Assignee
Sandia Corporation
|
Original Assignee
Sandia Corporation
|
Occupant type and position detection system | ||
Patent #
US 6,005,958 A
Filed 04/23/1997
|
Current Assignee
Automotive Systems Laboratory Incorporated
|
Original Assignee
Automotive Systems Laboratory Incorporated
|
Optical identification and monitoring system using pattern recognition for use with vehicles | ||
Patent #
US 5,835,613 A
Filed 06/07/1995
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Vehicle interior identification and monitoring system | ||
Patent #
US 5,829,782 A
Filed 04/30/1996
|
Current Assignee
Automotive Technologies International Incorporated
|
Original Assignee
Automotive Technologies International Incorporated
|
Optical identification and monitoring system using pattern recognition for use with vehicles | ||
Patent #
US 5,845,000 A
Filed 06/07/1995
|
Current Assignee
American Vehicular Sciences LLC
|
Original Assignee
Automotive Technologies International Incorporated
|
Automotive occupant sensing device | ||
Patent #
US 5,528,698 A
Filed 03/27/1995
|
Current Assignee
Bendix Commercial Vehicle Systems LLC
|
Original Assignee
Rockwell International Corporation
|
Apparatus and method for controlling an occupant restraint system | ||
Patent #
US 5,531,472 A
Filed 05/01/1995
|
Current Assignee
TRW Vehicle Safety Systems Incorporated
|
Original Assignee
TRW Vehicle Safety Systems Incorporated
|
Shock absorbing interior system for vehicle passengers | ||
Patent #
US 5,398,185 A
Filed 01/28/1993
|
Current Assignee
Nissan Motor Co. Ltd.
|
Original Assignee
Nissan Motor Co. Ltd.
|
Adjustable crash discrimination system with occupant position detection | ||
Patent #
US 5,446,661 A
Filed 04/15/1993
|
Current Assignee
Automotive Systems Laboratory Incorporated
|
Original Assignee
Automotive Systems Laboratory Incorporated
|
Computational methods and electronic camera apparatus for determining distance of objects, rapid autofocusing, and obtaining improved focus images | ||
Patent #
US 5,193,124 A
Filed 06/15/1990
|
Current Assignee
The Research Foundation for The State University of New York
|
Original Assignee
The Research Foundation for The State University of New York
|
Method and apparatus for protecting motor vehicle occupants | ||
Patent #
US 5,118,134 A
Filed 12/14/1990
|
Current Assignee
Robert Bosch GmbH
|
Original Assignee
Robert Bosch GmbH
|
Passenger out-of-position sensor | ||
Patent #
US 5,071,160 A
Filed 08/27/1990
|
Current Assignee
Automotive Systems Laboratory Incorporated
|
Original Assignee
Automotive Systems Laboratory Incorporated
|
Air bag system for automobile | ||
Patent #
US 5,074,583 A
Filed 07/28/1989
|
Current Assignee
Mazda Motor Corporation
|
Original Assignee
Mazda Motor Corporation
|
Apparatus for commanding energization of electrical device | ||
Patent #
US 4,768,088 A
Filed 12/02/1986
|
Current Assignee
Aisin Seiki Kabushikikaisha
|
Original Assignee
Aisin Seiki Kabushikikaisha
|
23 Claims
-
1. A vehicle comprising:
-
a compartment; a crash sensor system arranged to detect a crash involving the vehicle; a restraint system having a variable actuation in the event of a crash involving the vehicle; an occupant monitoring system for obtaining information about occupancy of said compartment and controlling said restraint system based on the obtained information, said occupant monitoring system comprising; at least first and second optical imagers for obtaining images of a common area of said compartment, said first and second imagers being spaced apart from one another; and processing circuitry coupled to said crash sensor system, said restraint system and said first and second imagers and arranged to derive information from images obtained by said first and second imagers and control the actuation of said restraint system based on the derived information and on detection of a crash by said crash sensor system, said processing circuitry being arranged to identify the occupant such that the presence of different types of occupants is determinable and said restraint system is controllable to provide different actuations depending on the type of the occupant. - View Dependent Claims (2, 3, 4, 5, 6, 7, 18, 19, 20, 21)
-
-
8. A vehicle comprising:
-
a compartment; a crash sensor system arranged to detect a crash involving the vehicle; a restraint system having a variable actuation in the event of a crash involving the vehicle; an arrangement for adjusting or controlling said restraint system based on information about occupancy of said compartment, said arrangement comprising; at least first and second optical imagers for obtaining images of a common area of said compartment, said first and second imagers being spaced apart from one another; processing circuitry coupled to said crash sensor system and said first and second imagers and arranged to derive information from images obtained by said first and second images and determine the manner in which said restraint system is to be actuated based on the derived information and on detection of a crash by said crash sensor system; and an adjustment or control system coupled to said processing circuitry for adjusting or controlling said restraint system to provide for the determined manner of actuation of said restraint system as determined by said processing circuitry, said processing circuitry deriving a model of an occupying item in said compartment based on at least one initial set of images from said first and second imagers and subsequently deriving information about movement of the occupying item based on variations between the model and subsequently obtained images from said first and second imagers. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A method for controlling actuation of a restraint system in a vehicle based on occupancy of a compartment in the vehicle, comprising:
-
arranging a crash sensor system on the vehicle to detect a crash involving the vehicle; arranging a restraint system on the vehicle which has a variable actuation in the event of a crash involving the vehicle; arranging at least first and second optical imagers on or in connection with a wall defining the compartment, the first and second imagers being spaced apart from one another; obtaining images of a common area of the compartment via the first and second imagers; deriving information from the images obtained by the first and second imagers, said step of deriving information comprising identifying an occupant in the compartment such that the presence of different types of occupants is determinable; and controlling actuation of the restraint system based on the derived information and on detection of a crash by the crash sensor system, the restraint system being controlled to provide different actuations depending on the type of the identified occupant. - View Dependent Claims (16, 17)
-
-
22. A method for controlling actuation of a restraint system in a vehicle based on occupancy of a compartment in the vehicle, comprising:
-
arranging a crash sensor system on the vehicle to detect a crash involving the vehicle; arranging a restraint system on the vehicle which has a variable actuation in the event of a crash involving the vehicle; arranging at least first and second optical imagers on or in connection with a wall defining the compartment, the first and second imagers being spaced apart from one another; adjusting or controlling the restraint system based on information about occupancy of the compartment, said adjusting or controlling step comprising arranging at least first and second optical imagers spaced apart from one another on the vehicle; obtaining images of a common area of the compartment via the first and second imagers; deriving information from images obtained by the first and second imagers; determining the manner in which the restraint system is to be actuated based on the derived information and on detection of a crash by the crash sensor system, the restraint system being adjusted or controlled to provide for the determined manner of actuation of the restraint system; deriving a model of an occupying item in the compartment based on at least one initial set of images from the first and second imagers; and subsequently deriving information about movement of the occupying item based on variations between the model and subsequently obtained images from the first and second imagers.
-
-
23. A vehicle comprising:
-
a compartment; a crash sensor system arranged to detect a crash involving the vehicle; a restraint system having a variable actuation in the event of a crash involving the vehicle; an occupant monitoring system for obtaining information about occupancy of said compartment and controlling said restraint system based on the obtained information, said occupant monitoring system comprising; at least first and second optical imagers for obtaining images of a common area of said compartment, said first and second imagers being spaced apart from one another; and processing circuitry coupled to said crash sensor system, said restraint system and said first and second imagers and arranged to derive information from images obtained by said first and second imagers and control the actuation of said restraint system based on the derived information and on detection of a crash by said crash sensor system, said processing circuitry comprising a microprocessor with at least one pattern recognition algorithm, said at least one pattern recognition algorithm comprising a trained pattern recognition algorithm trained in a training stage to identify different occupants by obtaining images from said first and second imagers when each of a plurality of different occupants is present in the common area of said compartment and associating a derivation of the images with an identification of the occupant.
-
1 Specification
This application claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. Nos. 60/534,926 filed Jan. 8, 2004 and 60/502,565 filed Sep. 12, 2003, and is:
1. a continuation-in-part of U.S. patent application Ser. No. 10/191,692 filed Jul. 9, 2002 which is a continuation-in-part of U.S. patent application Ser. No. 10/152,160 filed May 21, 2002 which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/292,386 filed May 21, 2001;
2. a continuation-in-part of U.S. patent application Ser. No. 10/303,364 filed Nov. 25, 2002, now U.S. Pat. No. 6,784,379;
3. a continuation-in-part of U.S. patent application Ser. No. 10/174,803 filed Jun. 19, 2002 which is a continuation-in-part of:
a) U.S. patent application Ser. No. 09/500,346 filed Feb. 8, 2000, now U.S. Pat. No. 6,442,504, which is a continuation-in-part of U.S. patent application Ser. No. 09/128,490, now U.S. Pat. No. 6,078,854, which is a continuation-in-part of:
- 1) U.S. patent application Ser. No. 08/474,783 filed Jun. 7, 1995, now U.S. Pat. No. 5,822,707, and
- 2) U.S. patent application Ser. No. 08/970,822 filed Nov. 14, 1997, now U.S. Pat. No. 6,081,757;
b) U.S. patent application Ser. No. 09/849,558 filed May 4, 2001, now U.S. Pat. No. 6,653,577, which is a continuation-in-part of U.S. patent application Ser. No. 09/193,209 filed Nov. 17, 1998, now U.S. Pat. No. 6,242,701, which is a continuation-in-part of U.S. patent application Ser. No. 09/128,490 filed Aug. 4, 1998, now U.S. Pat. No. 6,078,854, which is a continuation-in-part of:
- 1) U.S. patent application Ser. No. 08/474,783 filed Jun. 7, 1995, now U.S. Pat. No. 5,822,707, and
- 2) U.S. patent application Ser. No. 08/970,822 filed Nov. 14, 1997, now U.S. Pat. No. 6,081,757;
c) U.S. patent application Ser. No. 09/849,559 filed May 4, 2001, now U.S. Pat. No. 6,689,962, which is a continuation-in-part of U.S. patent application Ser. No. 09/193,209 filed Nov. 17, 1998, now U.S. Pat. No. 6,242,701, which is a continuation-in-part of U.S. patent application Ser. No. 09/128,490 filed Aug. 4, 1998, now U.S. Pat. No. 6,078,854, which is a continuation-in-part of:
- 1) U.S. patent application Ser. No. 08/474,783 filed Jun. 7, 1995, now U.S. Pat. No. 5,822,707, and
- 2) U.S. patent application Ser. No. 08/970,822 filed Nov. 14, 1997, now U.S. Pat. No. 6,081,757;
d) U.S. patent application Ser. No. 09/901,879 filed Jul. 9, 2001, now U.S. Pat. No. 6,555,766, which is a continuation of U.S. patent application Ser. No. 09/849,559 filed May 4, 2001 which is a continuation-in-part of U.S. patent application Ser. No. 09/193,209 filed Nov. 17, 1998, now U.S. Pat. No. 6,242,701, which is a continuation-in-part of U.S. patent application Ser. No. 09/128,490 filed Aug. 4, 1998, now U.S. Pat. No. 6,078,854, which is a continuation-in-part of:
- 1) U.S. patent application Ser. No. 08/474,783 filed Jun. 7, 1995, now U.S. Pat. No. 5,822,707, and
- 2) U.S. patent application Ser. No. 08/970,822 filed Nov. 14, 1997, now U.S. Pat. No. 6,081,757;
e) U.S. patent application Ser. No. 09/753,186 filed Jan. 2, 2001, now U.S. Pat. No. 6,484,080;
f) U.S. patent application Ser. No. 09/767,020 filed Jan. 23, 2001, now U.S. Pat. No. 6,533,316; and
g) U.S. patent application Ser. No. 09/770,974 filed Jan. 26, 2001, now U.S. Pat. No. 6,648,367;
4. a continuation-in-part of U.S. patent application Ser. No. 10/341,554 filed Jan. 13, 2003 which is a continuation-in-part of U.S. patent application Ser. No. 09/827,961 filed Apr. 6, 2001, now U.S. Pat. No. 6,517,107, which is a continuation of U.S. patent application Ser. No. 09/328,566 filed Jun. 9, 1999, now U.S. Pat. No. 6,279,946, which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/088,386 filed Jun. 9, 1998;
5. a continuation-in-part of U.S. patent application Ser. No. 10/234,067 filed Sep. 3, 2002 which is a continuation-in-part of U.S. patent application Ser. No. 09/778,137, now U.S. Pat. No. 6,513,830, which is a continuation of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537, which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462, which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned;
6. a continuation-in-part of U.S. patent application Ser. No. 09/639,303 filed Aug. 16, 2000, which is:
a) a continuation of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537, which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462, which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned;
b) a continuation-in-part of U.S. patent application Ser. No. 09/409,625 filed Oct. 1, 1999, now U.S. Pat. No. 6,270,116;
c) a continuation-in-part of U.S. patent application Ser. No. 09/448,337 filed Nov. 23, 1999, now U.S. Pat. No. 6,283,503; and
d) a continuation-in-part of U.S. patent application Ser. No. 09/448,338 filed Nov. 23, 1999, now U.S. Pat. No. 6,168,198;
7. a continuation-in-part of U.S. patent application Ser. No. 10/356,202 filed Jan. 31, 2003, now U.S. Pat. No. 6,793,242;
8. a continuation-in-part of U.S. patent application Ser. No. 10/227,780 filed Aug. 26, 2002, which is a continuation-in-part of U.S. patent application Ser. No. 09/838,920 filed Apr. 20, 2001, now U.S. Pat. No. 6,778,672, which is a continuation-in-part of U.S. patent application Ser. No. 09/563,556 filed May 3, 2000, now U.S. Pat. No. 6,474,683, which is a continuation-in-part of U.S. patent application Ser. No. 09/437,535 filed Nov. 10, 1999, now U.S. Pat. No. 6,712,387, which is a continuation-in-part of U.S. patent application Ser. No. 09/047,703 filed Mar. 25, 1998, now U.S. Pat. No. 6,039,139, which is:
a) a continuation-in-part of U.S. patent application Ser. No. 08/640,068 filed Apr. 30, 1996, now U.S. Pat. No. 5,829,782, which is a continuation application of U.S. patent application Ser. No. 08/239,978 filed May 9, 1994, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
b) a continuation-in-part of U.S. patent application Ser. No. 08/905,876 filed Aug. 4, 1997, now U.S. Pat. No. 5,848,802, which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462, which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
9. a continuation-in-part of U.S. patent application Ser. No. 10/613,453 filed Jul. 3, 2003 which is a continuation of U.S. patent application Ser. No. 10/188,673 filed Jul. 3, 2002, now U.S. Pat. No. 6,738,697, which is:
a) a continuation-in-part of U.S. patent application Ser. No. 10/174,709 filed Jun. 19, 2002, now U.S. Pat. No. 6,735,506;
b) a continuation-in-part of U.S. patent application Ser. No. 09/753,186 filed Jan. 2, 2001, now U.S. Pat. No. 6,484,080, which is a continuation-in-part of U.S. patent application Ser. No. 09/137,918 filed Aug. 20, 1998, now U.S. Pat. No. 6,175,787, which is a continuation-in-part of U.S. patent application Ser. No. 08/476,077 filed Jun. 7, 1995, now U.S. Pat. No. 5,809,437; and
c) a continuation-in-part of U.S. patent application Ser. No. 10/079,065 filed Feb. 19, 2002, now U.S. Pat. No. 6,662,642, which:
- 1) is a continuation-in-part of U.S. patent application Ser. No. 09/765,558 filed Jan. 19, 2001, now U.S. Pat. No. 6,748,797, which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/231,378 filed Sep. 8, 2000; and
- 2) claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/269,415 filed Feb. 16, 2001, U.S. provisional patent application Ser. No. 60/291,511 filed May 16, 2001 and U.S. provisional patent application Ser. No. 60/304,013 filed Jul. 9, 2001;
10. a continuation-in-part of U.S. patent application Ser. No. 10/058,706 filed Jan. 28, 2002 which is:
a. a continuation-in-part of U.S. patent application Ser. No. 09/891,432 filed Jun. 26, 2001, now U.S. Pat. No. 6,513,833, which is a continuation-in-part of U.S. patent application Ser. No. 09/838,920 filed Apr. 20, 2001, now U.S. Pat. No. 6,778,672, which is a continuation-in-part of U.S. patent application Ser. No. 09/563,556 filed May 3, 2000, now U.S. Pat. No. 6,474,683, which is a continuation-in-part of U.S. patent application Ser. No. 09/437,535 filed Nov. 10, 1999 which is a continuation-in-part of U.S. patent application Ser. No. 09/047,703 filed Mar. 25, 1998, now U.S. Pat. No. 6,039,139, which is:
- 1) a continuation-in-part of U.S. patent application Ser. No. 08/640,068 filed Apr. 30, 1996, now U.S. Pat. No. 5,829,782, which is a continuation of U.S. patent application Ser. No. 08/239,978 filed May 9, 1994, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
- 2) a continuation-in-part of U.S. patent application Ser. No. 08/905,876 filed Aug. 4, 1997, now U.S. Pat. No. 5,848,802, which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462, which is a continuation of the Ser. No. 08/040,978 application which is a continuation-in-part of the Ser. No. 07/878,571 application;
b. a continuation-in-part of U.S. patent application Ser. No. 09/639,299 filed Aug. 15, 2000, now U.S. Pat. No. 6,422,595, which is:
- 1) a continuation-in-part of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537; which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462; which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned;
- 2) a continuation-in-part of U.S. patent application Ser. No. 09/409,625 filed Oct. 1, 1999, now U.S. Pat. No. 6,270,116, which is a continuation-in-part of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537; which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462; which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned; which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned;
- 3) a continuation-in-part of U.S. patent application Ser. No. 09/448,337 filed Nov. 23, 1999, now U.S. Pat. No. 6,283,503, which is a continuation-in-part of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537; which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462; which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned; which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
- 4) a continuation-in-part of U.S. patent application Ser. No. 09/448,338 filed Nov. 23, 1999, now U.S. Pat. No. 6,168,198, which is a continuation-in-part of U.S. patent application Ser. No. 08/905,877 filed Aug. 4, 1997, now U.S. Pat. No. 6,186,537; which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462; which is a continuation of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned; which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
c. a continuation-in-part of U.S. patent application Ser. No. 09/543,678 filed Apr. 7, 2000, now U.S. Pat. No. 6,412,813, which is a continuation-in-part of U.S. patent application Ser. No. 09/047,704 filed Mar. 25, 1998, now U.S. Pat. No. 6,116,638, which is:
- 1) a continuation-in-part of U.S. patent application Ser. No. 08/640,068 filed Apr. 30, 1996, now U.S. Pat. No. 5,829,782, which is a continuation of U.S. patent application Ser. No. 08/239,978 filed May 9, 1994, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is a continuation-in-part of U.S. patent application Ser. No. 07/878,571 filed May 5, 1992, now abandoned; and
- 2) a continuation-in-part of U.S. patent application Ser. No. 08/905,876 filed Aug. 4, 1997, now U.S. Pat. No. 5,848,802, which is a continuation of U.S. patent application Ser. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462, which is a continuation of the Ser. No. 08/040,978 application which is a continuation-in-part of the Ser. No. 07/878,571 application; and
11. a continuation-in-part of U.S. patent application Ser. No. 10/114,533 filed Apr. 2, 2002 which is a continuation-in-part of U.S. patent application Ser. No. 10/058,706 filed Jan. 28, 2002, the history of which is set forth above;
12. a continuation-in-part of U.S. patent application Ser. No. 10/805,903 filed Mar. 22, 2004 which is a continuation-in-part of:
A. U.S. patent application Ser. No. 10/174,709, filed Jun. 19, 2002, now U.S. Pat. No. 6,735,506, which is:
- 1. a continuation-in-part of U.S. patent application Ser. No. 09/753,186 filed Jan. 2, 2001, now U.S. Pat. No. 6,484,080, which is a continuation-in-part of U.S. patent application Ser. No. 09/137,918 filed Aug. 20, 1998, now U.S. Pat. No. 6,175,787, which is a continuation-in-part of U.S. patent application Ser. No. 08/476,077 filed Jun. 7, 1995, now U.S. Pat. No. 5,809,437;
- 2. a continuation-in-part of U.S. patent application Ser. No. 10/079,065 filed Feb. 19, 2002, now U.S. Pat. No. 6,662,642, which:
- a. claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/269,415 filed Feb. 16, 2001, U.S. provisional patent application Ser. No. 60/291,511 filed May 16, 2001 and U.S. provisional patent application Ser. No. 60/304,013 filed Jul. 9, 2001; and
- b. is a continuation-in-part of U.S. patent application Ser. No. 09/765,558 filed Jan. 19, 2001, now U.S. Pat. No. 6,748,797, which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/231,378 filed Sep. 8, 2000;
- 3. a continuation-in-part of U.S. patent application Ser. No. 10/114,533 filed Apr. 2, 2002, the history of which is set forth above;
B. a continuation-in-part of U.S. patent application Ser. No. 10/188,673, filed Jul. 3, 2002, now U.S. Pat. No. 6,738,697, which is:
- 1. a continuation-in-part of U.S. patent application Ser. No. 09/753,186 filed Jan. 2, 2001, now U.S. Pat. No. 6,484,080, which is a continuation-in-part of U.S. patent application Ser. No. 09/137,918 filed Aug. 20, 1998, now U.S. Pat. No. 6,175,787, which is a continuation-in-part of U.S. patent application Ser. No. 08/476,077 filed Jun. 7, 1995, now U.S. Pat. No. 5,809,437;
- 2. a continuation-in-part of U.S. patent application Ser. No. 10/079,065 filed Feb. 19, 2002, now U.S. Pat. No. 6,662,642, which:
- a. claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/269,415 filed Feb. 16, 2001, U.S. provisional patent application Ser. No. 60/291,511 filed May 16, 2001 and U.S. provisional patent application Ser. No. 60/304,013 filed Jul. 9, 2001; and
- b. is a continuation-in-part of U.S. patent application Ser. No. 09/765,558 filed Jan. 19, 2001, now U.S. Pat. No. 6,748,797, which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/231,378 filed Sep. 8, 2000; and
C. a continuation-in-part of U.S. patent application Ser. No. 10/174,709 filed Jun. 19, 2002, now U.S. Pat. No. 6,735,506.
13. a continuation-in-part of U.S. patent application Ser. No. 10/457,238 filed Jun. 9, 2003 which claims priority under 35 U.S.C. § 119(e) of U.S. provisional patent application Ser. No. 60/387,792 filed Jun. 11, 2002;
14. a continuation-in-part of U.S. patent application Ser. No. 10/116,808 filed Apr. 5, 2002 which is:
a. a continuation-in-part of U.S. patent application Ser. No. 09/838,919 filed Apr. 20, 2001, now U.S. Pat. No. 6,442,465, which is:
- 1) a continuation-in-part of U.S. patent application Ser. No. 09/765,559 filed Jan. 19, 2001, now U.S. Pat. No. 6,553,296, which is a continuation-in-part of U.S. patent application Ser. No. 09/476,255 filed Dec. 30, 1999, now U.S. Pat. No. 6,324,453, which claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/114,507 filed Dec. 31, 1998; and
- 2) a continuation-in-part of U.S. patent application Ser. No. 09/389,947 filed Sep. 3, 1999, now U.S. Pat. No. 6,393,133, which is a continuation-in-part of U.S. patent application Ser. No. 09/200,614, filed Nov. 30, 1998, now U.S. Pat. No. 6,141,432, which is a continuation of U.S. patent application Ser. No. 08/474,786 filed Jun. 7, 1995, now U.S. Pat. No. 5,845,000;
b. a continuation-in-part of U.S. patent application Ser. No. 09/925,043 filed Aug. 8, 2001, now U.S. Pat. No. 6,507,779, which is a continuation-in-part of U.S. patent application Ser. No. 09/765,559 filed Jan. 19, 2001, now U.S. Pat. No. 6,553,296, and a continuation-in-part of U.S. patent application Ser. No. 09/389,947 filed Sep. 3, 1999, now U.S. Pat. No. 6,393,133;
15. a continuation-in-part of U.S. patent application Ser. No. 10/061,016 filed Jan. 30, 2002 which is a continuation-in-part of U.S. patent application Ser. No. 09/901,879 filed Jul. 9, 2001, now U.S. Pat. No. 6,555,766, which is a continuation of U.S. patent application Ser. No. 09/849,559 filed May 4, 2001, now U.S. Pat. No. 6,689,962, which is a continuation-in-part of U.S. patent application Ser. No. 09/193,209 filed Nov. 17, 1998, now U.S. Pat. No. 6,242,701, which is a continuation-in-part of U.S. patent application Ser. No. 09/128,490 filed Aug. 4, 1998, now U.S. Pat. No. 6,078,854, which is a continuation-in-part of: 1) U.S. patent application Ser. No. 08/474,783 filed Jun. 7, 1995, now U.S. Pat. No. 5,822,707; and 2) U.S. patent application Ser. No. 08/970,822 filed Nov. 14, 1997, now U.S. Pat. No. 6,081,757;
16. a continuation-in-part of U.S. patent application Ser. No. 10/227,781 filed Aug. 26, 2002, now U.S. Pat. No. 6,792,342, which is:
a. a continuation-in-part of U.S. patent application Ser. No. 10/061,016 filed Jan. 30, 2002, the history of which is set forth above; and
b. a continuation-in-part of U.S. patent application Ser. No. 09/500,346 filed Feb. 8, 2000, now U.S. Pat. No. 6,442,504; and
17. a continuation-in-part of U.S. patent application Ser. No. 10/151,615 filed May 20, 2002, now U.S. Pat. No. 6,820,897, which is:
a. a continuation-in-part of U.S. patent application Ser. No. 09/891,432, now U.S. Pat. No. 6,513,833, the history of which is set forth above;
b. a continuation-in-part of U.S. patent application Ser. No. 09/639,299 filed Aug. 15, 2000, now U.S. Pat. No. 6,422,595, the history of which is set forth above; and
c. a continuation-in-part of U.S. patent application Ser. No. 09/543,678 filed Apr. 7, 2000, now U.S. Pat. No. 6,412,813, the history of which is set forth above;
18. a continuation-in-part of U.S. patent application Ser. No. 10/365,129 filed Feb. 12, 2003, which is a continuation-in-part of U.S. patent application Ser. No. 10/114,533 filed Apr. 2, 2002, the history of which is set forth above; and
19. a continuation-in-part of U.S. patent application Ser. No. 10/413,426 filed Apr. 14, 2003 which is:
a. a continuation-in-part of U.S. patent application Ser. No. 09/437,535 filed Nov. 10, 1999 now U.S. Pat. No. 6,712,387, the history of which is set forth above;
b. a continuation-in-part of U.S. patent application Ser. No. 09/765,559 filed Jan. 19, 2001, now U.S. Pat. No. 6,553,296, the history of which is set forth above;
c. a continuation-in-part of U.S. patent application Ser. No. 09/838,920 filed Apr. 20, 2001, now U.S. Pat. No. 6,778,672, the history of which is set forth above;
d. a continuation-in-part of U.S. patent application Ser. No. 09/849,559 filed May 4, 2001, now U.S. Pat. No. 6,689,962, the history of which is set forth above;
e. a continuation-in-part of U.S. patent application Ser. No. 09/901,879 filed Jul. 9, 2001, now U.S. Pat. No. 6,555,766, the history of which is set forth above;
f. a continuation-in-part of U.S. patent application Ser. No. 10/058,706 filed Jan. 28, 2002, the history of which is set forth above;
g. a continuation-in-part of U.S. patent application Ser. No. 10/061,016 filed Jan. 30, 2002, the history of which is set forth above; and
h. a continuation-in-part of U.S. patent application Ser. No. 10/114,533 filed Apr. 2, 2002, the history of which is set forth above;
i. a continuation-in-part of U.S. patent application Ser. No. 10/116,808 filed Apr. 5, 2002, the history of which is set forth above;
j. a continuation-in-part of U.S. patent application Ser. No. 10/151,615 filed May 20, 2002, now U.S. Pat. No. 6,820,897, the history of which is set forth above;
k. a continuation-in-part of U.S. patent application Ser. No. 10/227,781 filed Aug. 26, 2002, now U.S. Pat. No. 6,792,342, the history of which is set forth above;
l. a continuation-in-part of U.S. patent application Ser. No. 10/234,436 filed Sep. 3, 2002, now U.S. Pat. No. 6,757,602, which is:
- 1. a continuation-in-part of U.S. patent application Ser. No. 09/853,118 filed May 10, 2001, now U.S. Pat. No. 6,445,988, which is a continuation-in-part of U.S. patent application Ser. No. 09/474,147 filed Dec. 29, 1999, now U.S. Pat. No. 6,397,136, which is a continuation-in-part of U.S. patent application Ser. No. 09/382,406 filed Aug. 24, 1999, now U.S. Pat. No. 6,529,809, which:
- a. is a continuation-in-part of U.S. patent application Ser. No. 08/919,823, now U.S. Pat. No. 5,943,295, which is a continuation-in-part of U.S. patent application Ser. No. 08/798,029 filed Feb. 6, 1997, now abandoned; and
- b. claims priority under 35 U.S.C. §119(e) of U.S. provisional patent application Ser. No. 60/136,613 filed May 27, 1999;
- 1. a continuation-in-part of U.S. patent application Ser. No. 09/853,118 filed May 10, 2001, now U.S. Pat. No. 6,445,988, which is a continuation-in-part of U.S. patent application Ser. No. 09/474,147 filed Dec. 29, 1999, now U.S. Pat. No. 6,397,136, which is a continuation-in-part of U.S. patent application Ser. No. 09/382,406 filed Aug. 24, 1999, now U.S. Pat. No. 6,529,809, which:
m. a continuation-in-part of U.S. patent application Ser. No. 10/302,105 filed Nov. 22, 2002, now U.S. Pat. No. 6,772,057, which is a continuation-in-part of U.S. patent application Ser. No. 10/116,808 filed Apr. 5, 2002, the history of which is set forth above; and
n. a continuation-in-part of U.S. patent application Ser. No. 10/365,129 filed Feb. 12, 2003, the history of which is set forth above.
All of the above-referenced applications are incorporated by reference herein.
The present invention relates to the use of two or more imagers for monitoring the interior of a vehicle to obtain three-dimensional information relating to the contents or occupying objects of the vehicle.
The present invention also relates to occupant sensing in general and more particular to sensing characteristics or the classification of an occupant of a vehicle for the purpose of controlling a vehicular system, subsystem or component based on the sensed characteristics or classification.
The present invention also relates to an apparatus and method for measuring the seat weight including the weight of an occupying item of the vehicle seat and, more specifically, to a seat weight measuring apparatus having advantages including that the production cost and the assembling cost of such apparatus is lower than existing apparatus.
The present invention also relates to systems for remotely monitoring transportation assets and other movable and/or stationary items which have very low power requirements. In particular, the present invention relates to a system for attachment to shipping containers and other transportation assets which enables remote monitoring of the location, contents, properties and/or interior or exterior environment of shipping containers or other assets and transportation assets and, since it has a low power requirement, lasts for years without needing maintenance.
The present invention also relates to a tracking method and system for tracking shipping containers and other transportation assets and enabling recording of the travels of the shipping container or transportation asset.
The present invention also relates to methods and apparatus for diagnosing components in a vehicle and transmitting data relating to the diagnosis of the components in the vehicle and other information relating to the operating conditions of the vehicle to one or more remote locations distant from the vehicle, e.g., via a telematics link.
The present invention also relates to systems and method for diagnosing the state or condition of a vehicle, e.g., whether the vehicle is about to rollover or is experiencing a crash, and whether the vehicle has a component which is operating abnormally and could possibly fail resulting in a crash or severe handicap for the operator, and transmitting data relating to the diagnosis of the components in the vehicle and optionally other information relating to the operating conditions of the vehicle to one or more remote locations, e.g., via a telematics link.
The present invention further relates to methods and apparatus for diagnosing components in a vehicle and determining the status of occupants in a vehicle and transmitting data relating to the diagnosis of the components in the vehicle, and optionally other information relating to the operating conditions of the vehicle, and data relating to the occupants to one or more remote facilities such as a repair facility and an emergency response station.
The present invention relates to apparatus for obtaining information about an occupying item of a seat, in particular, a seat in an automotive vehicle.
The present invention also relates to apparatus and methods for adjusting a vehicle component, system or subsystem in which the occupancy of a seat, also referred to as the “seated state” herein, is evaluated using at least a weight measuring apparatus and the component, system or subsystem may then be adjusted based on the evaluated occupancy thereof. The vehicle component, system or subsystem, hereinafter referred to simply as a component, may be any adjustable component of the vehicle including, but not limited to, the bottom portion and backrest of the seat, the rear view and side mirrors, the brake, clutch and accelerator pedals, the steering wheel, the steering column, a seat armrest, a cup holder, the mounting unit for a cellular telephone or another communications or computing device and the visors. Further, the component may be a system such an as airbag system, the deployment or suppression of which is controlled based on the seated-state of the seat. The component may also be an adjustable portion of a system the operation of which might be advantageously adjusted based on the seated-state of the seat, such as a device for regulating the inflation or deflation of an airbag that is associated with an airbag system.
The present invention also relates to apparatus and method for automatically adjusting a vehicle component to a selected or optimum position for an occupant of a seat based on at least two measured morphological characteristics of the occupant, one of which is the weight of the occupant. Other morphological characteristics include the height of the occupant, the length of the occupant'"'"'s arms, the length of the occupant'"'"'s legs, the occupant'"'"'s head diameter, facial features and the inclination of the occupant'"'"'s back relative to the seat bottom. Other morphological characteristics are also envisioned for use in the invention including iris pattern properties from an iris scan, voice print and finger and hand prints.
The present invention relates to apparatus and methods for adjusting a steering wheel in a vehicle and more particularly, to apparatus and methods for adjusting a steering wheel based on the morphology of the driver, i.e., the driver'"'"'s physical characteristics or dimensions.
The present invention also relates to apparatus and methods for adjusting a steering wheel in which the occupancy of a seat, also referred to as the “seated state” herein, is evaluated using at least a weight measuring apparatus and the steering wheel may then be adjusted based on the evaluated occupancy thereof.
The present invention also relates to apparatus and method for automatically adjusting a steering wheel to a selected or optimum position for a driver based on one or more measured morphological characteristics of the driver. Possible morphological characteristics include the height of the driver, the length of the driver'"'"'s arms, the length of the driver'"'"'s legs and the inclination of the driver'"'"'s back relative to the seat bottom.
At least one of the inventions disclosed herein also relates a system and method for monitoring the presence of an obstacle in an aperture, specifically, an aperture in a vehicle, for the purpose of halting closure of the aperture when an obstacle is detected in the path of the closing member.
The present invention also relates to the field of sensing, detecting, monitoring and identifying various objects, and parts thereof, which are located within the passenger compartment of a motor vehicle. In particular, the present invention provides improvements to ultrasonic transducers, and electromagnetic transducers and systems of such transducers, which improve the speed and/or accuracy and tend to reduce the cost and complexity of systems and which are efficient and highly reliable for detecting a particular object such as a rear facing child seat (RFCS) situated in the passenger compartment in a location where it may interact with a deploying airbag, or for detecting an out-of-position occupant. This permits the selective suppression of airbag deployment when the deployment may result in greater injury to the occupant than the crash forces. In the alternative, it permits the tailoring of the airbag deployment to the particular occupant and in consideration of the position of the occupant. This is accomplished in part through (i) the use of a tubular mounting structure for the transducers; (ii) the use of electronic reduction or suppression of transducer ringing; (iii) the use of mechanical damping of the transducer cone, all three of which permits the use of a single transducer for both sending and receiving; (iv) the use of multiple frequencies thereby permitting the simultaneous transmission of all transducers thereby reducing the time and increasing the accuracy of dynamic occupant position measurements; (v) the use of shaped horns, grills and reflectors for the output of the transducers to precisely control the beam pattern and thereby minimizing false echoes; (vi) the use of a logarithmic compression amplifier to minimize the effects of thermal gradients in the vehicle; (vii) the use of a method of temperature compensation based on the change in transducer properties with temperature; and/or (viii) the use of a dual level network, one level for categorization and the second for occupant position sensing, to improve the accuracy of categorization and the speed of position measurement for dynamic out-of-position. The foregoing can be used individually or in combination with one another.
The present invention additionally relates generally to methods and arrangements for determining that there is a life form, i.e., a human being, in a vehicle and the location of the life form, i.e., in which seat the life form is situated.
More specifically, the present invention relates to methods and arrangement for obtaining information about occupancy of a vehicle and utilizing this information for some other purpose, e.g., to control various vehicular systems to benefit the occupants.
Even more specifically, the present invention relates to methods and arrangements for obtaining information about occupancy of a vehicle, in particular after a crash involving the vehicle, and conveying this information to response personnel to optimize their response to the crash and/or enable proper assistance to be rendered to the occupants after the crash.
The present invention also relates to methods and apparatus for controlling an occupant restraint system in a vehicle based in part on the diagnosed state of the vehicle in an attempt to minimize injury to an occupant.
The present invention also relates to methods and apparatus for disabling an airbag system in a motor vehicle if the seating position is unoccupied or an occupant is out-of-position, i.e., closer to the airbag door than a predetermined distance.
All of the patents, patent applications, technical papers and other references referenced below are incorporated herein by reference in their entirety unless stated otherwise.
Crash sensors for determining that a vehicle is in a crash of sufficient magnitude as to require the deployment of an inflatable restraint system, or airbag, are either mounted in a portion of the front of the vehicle which has crushed by the time that sensor triggering is required, the crush zone, or elsewhere such as the passenger compartment, the non-crush zone. Regardless of where sensors are mounted, there will always be crashes where the sensor triggers late and the occupant has moved to a position near to the airbag deployment cover. In such cases, the occupant may be seriously injured or even killed by the deployment of the airbag. At least one of the inventions disclosed herein is largely concerned with preventing such injuries and deaths by preventing late airbag deployments.
In a Society of Automotive Engineers (SAE) paper by Mertz, Driscoll, Lenox, Nyquist and Weber titled “Response of Animals Exposed to Deployment of Various Passenger Inflatable Restraint System Concepts for a Variety of Collision Severities and Animal Positions” SAE 826074, 1982, the authors show that an occupant can be killed or seriously injured by the airbag deployment if he or she is located out of position near or against the airbag when deployment is initiated. These conclusions were again reached in a more recent paper by Lau, Horsch, Viano and Andrzejak titled “Mechanism of Injury From Air Bag Deployment Loads”, published in Accident Analysis & Prevention, Vol. 25, No. 1, 1993, Pergamon Press, New York, where the authors conclude that “Even an inflator with inadequate gas output to protect a properly seated occupant had sufficient energy to induce severe injuries in a surrogate in contact with the inflating module.” These papers highlight the importance of preventing deployment of an airbag when an occupant is out of position and in close proximity to the airbag module.
The Ball-in-Tube crush zone sensor, such as disclosed in U.S. Pat. No. 4,974,350; U.S. Pat. No. 4,198,864; U.S. Pat. No. 4,284,863; U.S. Pat. No. 4,329,549; U.S. Pat. No. 4,573,706 and U.S. Pat. No. 4,900,880 to D. S. Breed, has achieved the widest use while other technologies, including magnetically damped sensors as disclosed in U.S. Pat. No. 4,933,515 to Behr et al and crush switch sensors such as disclosed in U.S. Pat. No. 4,995,639 to D. S. Breed, are now becoming available. Other sensors based on spring-mass technologies are also being used in the crush zone. Crush zone mounted sensors, in order to function properly, must be located in the crush zone at the required trigger time during a crash or they can trigger late. One example of this was disclosed in a Society of Automotive Engineers (SAE) paper by D. S. Breed and V. Castelli titled “Trends in Sensing Frontal Impacts”, SAE 890750, 1989, and further in U.S. Pat. No. 4,900,880. In impacts with soft objects, the crush of a vehicle can be significantly less than for impacts with barriers, for example. In such cases, even at moderate velocity changes where an airbag might be of help in mitigating injuries, the crush zone mounted sensor might not actually be in the crush zone at the time that sensor triggering is required for timely airbag deployment, and as a result can trigger late when the occupant is already resting against the airbag module.
There is a trend underway toward the implementation of Single Point Sensors (SPS) which are typically located in the passenger compartment. In theory, these sensors use sophisticated computer algorithms to determine that a particular crash is sufficiently severe as to require the deployment of an airbag. In another SAE paper by Breed, Sanders and Castelli titled “A Critique of Single Point Sensing”, SAE 920124, 1992, the authors demonstrate that there is insufficient information in the non-crush zone of the vehicle to permit a decision to be made to deploy an airbag in time for many crashes. Thus, sensors mounted in the passenger compartment or other non-crush zone locations, will also trigger the deployment of the airbag late on many crashes.
A crash sensor is necessarily a predictive device. In order to inflate the airbag in time, the inflation must be started before the full severity of the crash has developed. All predictive devices are subject to error, so that sometimes the airbag will be inflated when it is not needed and at other times it will not be inflated when it could have prevented injury. The accuracy of any predictive device can improve significantly when a longer time is available to gather and process the data. One purpose of the occupant position sensor is to make possible this additional time in those cases where the occupant is farther from the airbag module when the crash begins and/or where, due to seat belt use or otherwise, the occupant is moving toward the airbag module more slowly. In these cases the decision on whether to deploy the airbag can be deferred and a more precise determination made of whether the airbag is needed and the characteristics of such deployment
The discussions of timely airbag deployment above are all based on the seating position of the average male (the so called 50% male) relative to the airbag or steering wheel. For the 50% male, the sensor triggering requirement is typically calculated based on an allowable motion of the occupant of 5 inches before the airbag is fully inflated. Airbags typically require about 30 milliseconds of time to achieve full inflation and, therefore, the sensor must trigger inflation of the airbag 30 milliseconds before the occupant has moved forward 5 inches. The 50% male, however, is actually the 70% person and therefore about 70% of the population sit on average closer to the airbag than the 50% male and thus are exposed to a greater risk of interacting with the deploying airbag. A recent informal survey, for example, found that although the average male driver sits about 12 inches from the steering wheel, about 2% of the population of drivers sit closer than 6 inches from the steering wheel and 10% sit closer than 9 inches. Also, about 1% of drivers sit at about 24 inches and about 16% at least 18 inches from the steering wheel. None of the sensor systems now on the market take account of this variation in occupant seating position and yet this can have a critical effect on the sensor required maximum triggering time.
For example, if a fully inflated airbag is about 7 inches thick, measured from front to back, then any driver who is seated closer than 7 inches will necessarily interact with the deploying airbag and the airbag probably should not be deployed at all. For a recently analyzed 30 mph barrier crash of a mid-sized car, the sensor required triggering time, in order to allow the airbag to inflate fully before the driver becomes closer than 7 inches from the steering wheel, results in a maximum sensing time of 8 milliseconds for an occupant initially positioned 9 inches from the airbag, 25 milliseconds at 12 inches, 45 milliseconds at 18 inches and 57 milliseconds for the occupant who is initially positioned at 24 inches from the airbag. Thus for the same crash, the sensor required triggering time varies from a no trigger to 57 milliseconds, depending on the initial position of the occupant. A single sensor triggering time criterion that fails to take this into account, therefore, will cause injuries to small people or deny the protection of the airbag to larger people. A very significant improvement to the performance of an airbag system will necessarily result from taking the occupant position into account as described herein.
A further complication results from the fact that a greater number of occupants are now wearing seatbelts which tends to prevent many of these occupants from getting too close to the airbag. Thus, just knowing the initial position of the occupant is insufficient and either the position must be continuously monitored or the seatbelt use must be known. Also, the occupant may have fallen asleep or be unconscious prior to the crash and be resting against the steering wheel. Some sensor systems have been proposed that double integrate the acceleration pulse in the passenger compartment and determine the displacement of the occupant based on the calculated displacement of an unrestrained occupant seated at the mid seating position. This sensor system then prevents the deployment of the airbag if, by this calculation, the occupant is too close to the airbag. This calculation can be greatly in error for the different seating positions discussed above and also for the seat-belted occupant, and thus an occupant who wears a seatbelt could be denied the added protection of the airbag in a severe crash.
As the number of vehicles which are equipped with airbags is now rapidly increasing, the incidence of late deployments is also increasing. It has been estimated that out of approximately 400 airbag related complaints to the National Highway Traffic Safety Administration (NHTSA) through 1991, for example, about 5% to 10% involved burns and injuries which were due to late airbag deployments. There are also at least three known fatalities where a late airbag deployment is suspected as the cause.
Automobiles equipped with airbags are well known in the prior art. In such airbag systems, the car crash is sensed and the airbags rapidly inflated thereby insuring the safety of an occupation in a car crash. Many lives have now been saved by such airbag systems. However, depending on the seated state of an occupant, there are cases where his or her life cannot be saved even by present airbag systems. For example, when a passenger is seated on the front passenger seat in a position other than a forward facing, normal state, e.g., when the passenger is out of position and near the deployment door of the airbag, there will be cases when the occupant will be seriously injured or even killed by the deployment of the airbag.
Also, sometimes a child seat is placed on the passenger seat in a rear facing position and there are cases where a child sitting in such a seat has been seriously injured or killed by the deployment of the airbag.
Furthermore, in the case of a vacant seat, there is no need to deploy an airbag and indeed deploying the airbag is undesirable due to a high replacement cost and possible release of toxic gases into the passenger compartment. Nevertheless, most airbag systems will deploy the airbag in a vehicle crash even if the seat is unoccupied.
Thus, whereas thousands of lives have been saved by airbags, a large number of people have also been injured, some seriously, by the deploying airbag, and over 100 people have now been killed. Thus, significant improvements need to be made to airbag systems. As discussed in detail in U.S. Pat. No. 5,653,462, for a variety of reasons vehicle occupants may be too close to the airbag before it deploys and can be seriously injured or killed as a result of the deployment thereof. Also, a child in a rear facing child seat that is placed on the right front passenger seat is in danger of being seriously injured if the passenger airbag deploys. For these reasons and, as first publicly disclosed in Breed, D. S. “How Airbags Work” presented at the International Conference on Seatbelts and Airbags in 1993 in Canada, occupant position sensing and rear facing child seat detection systems are required in order to minimize the damages caused by deploying front and side airbags. It also may be required in order to minimize the damage caused by the deployment of other types of occupant protection and/or restraint devices that might be installed in the vehicle.
For these reasons, there has been proposed an occupant sensor system also known as a seated-state detecting unit such as disclosed in the following U.S. patents assigned to the current assignee of the present application: Breed et al. U.S. Pat. No. 5,563,462, U.S. Pat. No. 5,829,782, U.S. Pat. No. 5,822,707, U.S. Pat. No. 5,694,320, U.S. Pat. No. 5,748,473, U.S. Pat. No. 6,078,854, U.S. Pat. No. 6,081,757 and U.S. Pat. No. 6,242,701 and Varga et al. U.S. Pat. No. 5,943,295. Typically, in some of these designs three or four sensors or sets of sensors are installed at three or four points in a vehicle for transmitting ultrasonic or electromagnetic waves toward the passenger or driver'"'"'s seat and receiving the reflected waves. Using appropriate hardware and software, the approximate configuration of the occupancy of either the passenger or driver seat can be determined thereby identifying and categorizing the occupancy of the relevant seat. Of particular interest, the Breed et al. patents mention that the presence of a child in a rear facing child seat placed on the right front passenger seat may be detected as this has become an industry-wide concern to prevent deployment of an occupant restraint device in these situations. The U.S. automobile industry is continually searching for an easy, economical solution, which will prevent the deployment of the passenger side airbag if a rear facing child seat is present.
These systems will solve the out-of-position occupant and the rear facing child seat problems related to current airbag systems and prevent unneeded and unwanted airbag deployments when a front seat is unoccupied. Some of the airbag systems will also protect rear seat occupants in vehicle crashes and all occupants in side impacts.
However, there is a continual need to improve the systems which detect the presence of occupants, determine if they are out-of-position and to identify the presence of a rear facing child seat in the rear seat as well as the front seat. Future automobiles are expected to have eight or more airbags as protection is sought for rear seat occupants and from side impacts. In addition to eliminating the disturbance and possible harm of unnecessary airbag deployments, the cost of replacing these airbags will be excessive if they all deploy in an accident needlessly. The improvements described below minimize this cost by not deploying an airbag for a seat, which is not occupied by a human being. An occupying item of a seat may be a living occupant such as a human being or dog, another living organism such as a plant, or an inanimate object such as a box or bag of groceries.
The need for an occupant out-of-position sensor has also been observed by others and several methods have been described in certain U.S. patents for determining the position of an occupant of a motor vehicle. However, none of these prior art systems are believed to be capable of solving the many problems associated with occupant sensors and no prior art has been found that describe the methods of adapting such sensors to a particular vehicle model to obtain high system accuracy prior to the disclosure thereof by the current assignee. Also, none of these prior art systems employ operative and effective pattern recognition technologies that are believed to be essential to accurate occupant sensing. Each of these prior are systems will be discussed below.
In 1984, the National Highway Traffic Safety Administration (NHTSA) of the U.S. Department of Transportation issued a requirement for frontal crash protection of automobile occupants known as FMVSS-208. This regulation mandated “passive occupant restraints” for all passenger cars by 1992. A further modification to FMVSS-208 required both driver and passenger side airbags on all passenger cars and light trucks by 1998. FMVSS-208 was later modified to require all vehicles to have occupant sensors. The demand for airbags is constantly accelerating in both Europe and Japan and all vehicles produced in these areas and eventually worldwide will likely be, if not already, equipped with airbags as standard equipment and eventually with occupant sensors.
A device to monitor the vehicle interior and identify its contents is needed to solve these and many other problems. For example, once a Vehicle Interior Identification and Monitoring System (VIMS) for identifying and monitoring the contents of a vehicle is in place, many other products become possible as discussed below.
Inflators now exist which will adjust the amount of gas flowing to the airbag to account for the size and position of the occupant and for the severity of the accident. The VIMS discussed in U.S. Pat. No. 5,829,782 can control such inflators based on the presence and position of vehicle occupants or of a rear facing child seat. The inventions here are improvements on that VIMS system and some use an advanced optical system comprising one or more CCD or CMOS arrays plus a source of illumination preferably combined with a trained neural network pattern recognition system.
In the early 1990'"'"'s, the current assignee (ATI) developed a scanning laser radar optical occupant sensor that had the capability of creating a three-dimensional image of the contents of the passenger compartment. After proving feasibility, this effort was temporarily put aside due to the high cost of the system components and the current assignee then developed an ultrasonic-based occupant sensor that was commercialized and is now in production on some Jaguar models. The current assignee has long believed that optical systems would eventually become the technology of choice when the cost of optical components came down. This has now occurred and for the past several years, ATI has been developing a variety of optical occupant sensors.
The current assignee'"'"'s first camera optical occupant sensing system was an adult zone-classification system that detected the position of the adult passenger. Based on the distance from the airbag, the passenger compartment was divided into three zones, namely safe-seating zone, at-risk zone, and keep-out zone. This system was implemented in a vehicle under a cooperative development program with NHTSA. This proof-of-concept was developed to handle low-light conditions only. It used three analog CMOS cameras and three near-infrared LED clusters. It also required a desktop computer with three image acquisition boards. The locations of the camera/LED modules were: the A-pillar, the instrument panel (IP), and near the overhead console. The system was trained to handle camera blockage situations, so that the system still functioned well even when two cameras were blocked. The processing speed of the system was close to 50 fps giving it the capability of tracking an occupant during pre-crash braking situations—that is a dynamic system.
The second camera optical system was an occupant classification system that separated adult occupants from all other situations (i.e., child, child restraint and empty seat). This system was implemented using the same hardware as the first camera optical system. It was also developed to handle low-light conditions only. The results of this proof-of-concept were also very promising.
Since the above systems functioned well even when two cameras were blocked, it was decided to develop a stand alone system that is FMVSS208-compliant, and price competitive with weight-based systems but with superior performance. Thus, a third camera optical system (for occupant classification) was developed. Unlike the earlier systems, this system used one digital CMOS camera and two high-power near-infrared LEDs. The camera/LED module was installed near the overhead console and the image data was processed using a laptop computer. This system was developed to divide the occupancy state into four classes: 1) adult; 2) child, booster seat and forward facing child seat; 3) infant carrier and rearward facing child seat; and 4) empty seat. This system included two subsystems: a nighttime subsystem for handling low-light conditions, and a daytime subsystem for handling ambient-light conditions. Although the performance of this system proved to be superior to the earlier systems, it exhibited some weakness mainly due to a non-ideal aiming direction of the camera.
Finally, a fourth camera optical system was implemented using near production intent hardware using, for example, an ECU (Electronic Control Unit) to replace the laptop computer. In this system, the remaining problems of earlier systems were overcome. The hardware in this system is not unique so the focus below will be on algorithms and software which represent the innovative heart of the system.
1. Prior Art Occupant Sensors
The need for an occupant position sensor has been observed by others and several methods have been disclosed in U.S. patents for determining the position and velocity of an occupant of a motor vehicle. Each of these systems, however, has significant limitations. In White et al. (U.S. Pat. No. 5,071,160), a single acoustic sensor is described and, as illustrated, is disadvantageously mounted lower than the steering wheel. White et al. correctly perceive that such a sensor could be defeated, and the airbag falsely deployed (indicating that the system of White et al. deploys the airbag on occupant motion rather then suppressing it), by an occupant adjusting the control knobs on the radio and thus they suggest the use of a plurality of such sensors. White et al. does not disclose where such sensors would be mounted, other than on the instrument panel below the steering wheel, or how they would be combined to uniquely monitor particular locations in the passenger compartment and to identify the object(s) occupying those locations. The adaptation process to vehicles is not described nor is a combination of pattern recognition algorithms, nor any pattern recognition algorithm.
White et al. also describe the use of error correction circuitry, without defining or illustrating the circuitry, to differentiate between the velocity of one of the occupant'"'"'s hands, as in the case where he/she is adjusting a knob on the radio, and the remainder of the occupant. Three ultrasonic sensors of the type disclosed by White et al. might, in some cases, accomplish this differentiation if two of them indicate that the occupant was not moving while the third indicates that he or she is moving. Such a combination, however, would not differentiate between an occupant with both hands and arms in the path of the ultrasonic transmitter at such a location that they are blocking a substantial view of the occupant'"'"'s head or chest. Since the sizes and driving positions of occupants are extremely varied, trained pattern recognition systems, such as neural networks and combinations thereof, are required when a clear view of the occupant, unimpeded by his/her extremities, cannot be guaranteed. White et al. do not suggest the use of such neural networks.
Mattes et al. (U.S. Pat. No. 5,118,134) describe a variety of methods of measuring the change in position of an occupant including ultrasonic, active or passive infrared and microwave radar sensors, and an electric eye. The sensors measure the change in position of an occupant during a crash and use that information to access the severity of the crash and thereby decide whether or not to deploy the airbag. They are thus using the occupant motion as a crash sensor. No mention is made of determining the out-of-position status of the occupant or of any of the other features of occupant monitoring as disclosed in one or more of the current assignee'"'"'s above-referenced patents and patent applications. Nowhere does Mattes et al. discuss how to use active or passive infrared to determine the position of the occupant. As pointed out in one or more of the current assignee'"'"'s above-referenced patents and patent applications, direct occupant position measurement based on passive infrared is probably not possible with a single detector and, until very recently, was very difficult and expensive with active infrared requiring the modulation of an expensive GaAs infrared laser. Since there is no mention of these problems, the method of use contemplated by Mattes et al. must be similar to the electric eye concept where position is measured indirectly as the occupant passes by a plurality of longitudinally spaced-apart sensors.
The object of an occupant out-of-position sensor is to determine the location of the head and/or chest of the vehicle occupant in the passenger compartment relative to the occupant protection apparatus, such as an airbag, since it is the impact of either the head or chest with the deploying airbag that can result in serious injuries. Both White et al. and Mattes et al. disclose only lower mounting locations of their sensors that are mounted in front of the occupant such as on the dashboard/instrument panel or below the steering wheel. Both such mounting locations are particularly prone to detection errors due to positioning of the occupant'"'"'s hands, arms and legs. This would require at least three, and preferably more, such sensors and detectors and an appropriate logic circuitry, or pattern recognition system, which ignores readings from some sensors if such readings are inconsistent with others for the case, for example, where the driver'"'"'s arms are the closest objects to two of the sensors. The determination of the proper transducer mounting locations, aiming and field angles and pattern recognition system architectures for a particular vehicle model are not disclosed in either White et al. or Mattes et al. and are part of the vehicle model adaptation process described herein.
Fujita et al., in U.S. Pat. No. 5,074,583, describe another method of determining the position of the occupant but do not use this information to control and suppress deployment of an airbag if the occupant is out-of-position, or if a rear facing child seat is present. In fact, the closer that the occupant gets to the airbag, the faster the inflation rate of the airbag is according to the Fujita et al. patent, which thereby increases the possibility of injuring the occupant. Fujita et al. do not measure the occupant directly but instead determine his or her position indirectly from measurements of the seat position and the vertical size of the occupant relative to the seat. This occupant height is determined using an ultrasonic displacement sensor mounted directly above the occupant'"'"'s head.
It is important to note that in all cases in the above-cited prior art, except those assigned to the current assignee of the instant invention, no mention is made of the method of determining transducer location, deriving the algorithms or other system parameters that allow the system to accurately identify and locate an object in the vehicle. In contrast, in one implementation of the instant invention, the return wave echo pattern corresponding to the entire portion of the passenger compartment volume of interest is analyzed from one or more transducers and sometimes combined with the output from other transducers, providing distance information to many points on the items occupying the passenger compartment.
Other patents describing occupant sensor systems include U.S. Pat. No. 5,482,314 (Corrado et al.) and U.S. Pat. No. 5,890,085 (Corrado et al.). These patents, which were filed after the initial filings of the inventions herein and thus not necessarily prior art, describe a system for sensing the presence, position and type of an occupant in a seat of a vehicle for use in enabling or disabling a related airbag activator. A preferred implementation of the system includes two or more different but located together sensors which provide information about the occupant and this information is fused or combined in a microprocessor circuit to produce an output signal to the airbag controller. According to Corrado et al., the fusion process produces a decision as to whether to enable or disable the airbag with a higher reliability than a single phenomena sensor or non-fused multiple sensors. By fusing the information from the sensors to make a determination as to the deployment of the airbag, each sensor has only a partial effect on the ultimate deployment determination. The sensor fusion process is a crude pattern recognition process based on deriving the fusion “rules” by a trial and error process rather than by training.
The sensor fusion method of Corrado et al. requires that information from the sensors be combined prior to processing by an algorithm in the microprocessor. This combination can unnecessarily complicate the processing of the data from the sensors and other data processing methods can provide better results. For example, as discussed more fully below, it has been found to be advantageous to use a more efficient pattern recognition algorithm such as a combination of neural networks or fuzzy logic algorithms that are arranged to receive a separate stream of data from each sensor, without that data being combined with data from the other sensors (as in done in Corrado et al.) prior to analysis by the pattern recognition algorithms. In this regard, it is important to appreciate that sensor fusion is a form of pattern recognition but is not a neural network and that significant and fundamental differences exist between sensor fusion and neural networks. Thus, some embodiments of the invention described below differ from that of Corrado et al. because they include a microprocessor which is arranged to accept only a separate stream of data from each sensor such that the stream of data from the sensors are not combined with one another. Further, the microprocessor processes each separate stream of data independent of the processing of the other streams of data, that is, without the use of any fusion matrix as in Corrado et al.
1.1 Ultrasonics
The use of ultrasound for occupant sensing has many advantages and some drawbacks. It is economical in that ultrasonic transducers cost less than $1 in large quantities and the electronic circuits are relatively simple and inexpensive to manufacture. However, the speed of sound limits the rate at which the position of the occupant can be updated to approximately 7 milliseconds, which though sufficient for most cases, is marginal if the position of the occupant is to be tracked during a vehicle crash. Secondly, ultrasound waves are diffracted by changes in air density that can occur when the heater or air conditioner is operated or when there is a high-speed flow of air past the transducer. Thirdly, the resolution of ultrasound is limited by its wavelength and by the transducers, which are high Q tuned devices. Typically, this resolution is on the order of about 2 to 3 inches. Finally, the fields from ultrasonic transducers are difficult to control so that reflections from unwanted objects or surfaces add noise to the data.
Ultrasonics can be used in several configurations for monitoring the interior of a passenger compartment of an automobile as described in the current assignee'"'"'s above-referenced patents and patent applications and in particular in U.S. RE37260 (a reissue of U.S. Pat. No. 5,943,295). Using the teachings here, the optimum number and location of the ultrasonic and/or optical transducers can be determined as part of the adaptation process for a particular vehicle model.
In the cases of inventions disclosed here, as discussed in more detail below, regardless of the number of transducers used, a trained pattern recognition system is preferably used to identify and classify, and in some cases to locate, the illuminated object and its constituent parts.
The ultrasonic system is the least expensive and potentially provides less information than the optical or radar systems due to the delays resulting from the speed of sound and due to the wave length which is considerably longer than the optical (including infrared) systems. The wavelength limits the detail that can be seen by the system. Additionally, ultrasonic waves are sometimes strongly affected by thermal gradients within the vehicle such as caused by flowing air from the heater or air conditioner or as caused by the sun heating the top of the vehicle resulting in the upper part of the passenger compartment having a higher temperature than the lower part. Thermal gradients cause density changes in the air, which diffract the ultrasonic signal sending in a direction away from an object or the transducer. Although this effect has been reported in the literature, no solution has been proposed prior to the present invention.
In spite of these limitations, ultrasonics can provide sufficient timely information to permit the position and velocity of an occupant to be accurately known and, when used with an appropriate pattern recognition system, it is capable of positively determining the presence of a rear facing child seat. One pattern recognition system that has been successfully used to identify a rear facing child seat employs neural networks and is similar to that described in papers by Gorman et al.
However, in the aforementioned literature using ultrasonics, the pattern of reflected ultrasonic waves from an adult occupant who may be out of position is sometimes similar to the pattern of reflected waves from a rear facing child seat. Also, it is sometimes difficult to discriminate the wave pattern of a normally seated child with the seat in a rear facing position from an empty seat with the seat in a more forward position. In other cases, the reflected wave pattern from a thin slouching adult with raised knees can be similar to that from a rear facing child seat. In still other cases, the reflected pattern from a passenger seat that is in a forward position can be similar to the reflected wave pattern from a seat containing a forward facing child seat or a child sitting on the passenger seat. In each of these cases, the prior art ultrasonic systems can suppress the deployment of an airbag when deployment is desired or, alternately, can enable deployment when deployment is not desired.
If the discrimination between these cases can be improved, then the reliability of the seated-state detecting unit can be improved and more people saved from death or serious injury. In addition, the unnecessary deployment of an airbag can be prevented.
Recently issued U.S. Pat. No. 6,411,202 (Gal et al.) describes a safety system for a vehicle including at least one sensor that receives waves from a region in an interior portion of the vehicle, which thereby defines a protected volume at least partially in front of the vehicle airbag. A processor is responsive to signals from the sensor for determining geometric data of objects in the protected volume. The teachings of this patent, which is based on ultrasonics, are arguably fully disclosed in the prior patents of the current assignee referenced above.
Significant improvements were made to the art in the current assignee'"'"'s U.S. RE37260 which describes the method of placement of the transducers to increase the reliability of detecting and discriminating out-of-position occupants, empty seats, and rear facing child-seats. In order to detect occupants that are very close to the transducer in that invention, separate transducers are used for sending and receiving the ultrasonic waves. Also, although that system is capable of detecting out-of-position occupants for most real world cases, in situations where the crash sensor fails to trigger or triggers very late in a high speed crash, the system based on alternately transmitting and receiving from each location can require as much as 50 milliseconds to determine the location of an occupant which can be too slow. The use of one or two transducers for ranging during the crash, giving 10 or 20 millisecond response time, works in most cases but can be defeated if the selected transducer is blocked by a newspaper, for example. Finally, the wide beam patterns of the transducers used in that system sometimes results in false decisions when an occupant of the rear seat is leaning forward, for example, and the system interprets that as an in-position, forward facing person even though in fact, it may be a rear facing child seat.
Regardless of the number of transducers used, a trained pattern recognition system, as defined herein, can be used to identify and classify, and in some cases to locate, the illuminated object and its constituent parts. The invention herein is partially directed toward improving the invention of U.S. RE37260 by decreasing the sensing time, reducing the cost, improving the system response to objects which are close to the transducer mounting, and improving the ability of the system to compensate for thermal gradients and variations in the speed of sound.
1.2 Optics
Optics can be used in several configurations for monitoring the interior of a passenger compartment or exterior environment of an automobile. In one known method, a laser optical system uses a GaAs infrared laser beam to momentarily illuminate an object, occupant or child seat, in the manner as described and illustrated in FIG. 8 of U.S. Pat. No. 5,829,782. The receiver can be a charge-coupled device or CCD or a CMOS imager to receive the reflected light. The laser can either be used in a scanning mode, or, through the use of a lens, a cone of light can be created which covers a large portion of the object. In these configurations, the light can be accurately controlled to only illuminate particular positions of interest within or around the vehicle. In the scanning mode, the receiver need only comprise a single or a few active elements while in the case of the cone of light, an array of active elements is needed. The laser system has one additional significant advantage in that the distance to the illuminated object can be determined as disclosed in the commonly owned '"'"'462 patent as also described below. When a single receiving element is used, a PIN or avalanche diode is preferred.
In a simpler case, light generated by a non-coherent light emitting diode (LED) device is used to illuminate the desired area. In this case, the area covered is not as accurately controlled and a larger CCD or CMOS array is required. Recently, the cost of CCD and CMOS arrays has dropped substantially with the result that this configuration may now be the most cost-effective system for monitoring the passenger compartment as long as the distance from the transmitter to the objects is not needed. If this distance is required, then the laser system, a stereographic system, a focusing system, a combined ultrasonic and optic system, or a multiple CCD or CMOS array system as described herein is required. Alternately, a modulation system such as used with the laser distance system can be used with a CCD or CMOS camera and distance determined on a pixel by pixel basis.
The optical systems described herein are also applicable for many other sensing applications both inside and outside of the vehicle compartment such as for sensing crashes before they occur as described in U.S. Pat. No. 5,829,782, for a smart headlight adjustment system and for a blind spot monitor (also disclosed in U.S. patent application Ser. No. 09/851,362).
1.3 Ultrasonics and Optics
The laser systems described above are expensive due to the requirement that they be modulated at a high frequency if the distance from the airbag to the occupant, for example, is to be measured. Alternately, modulation of another light source, such as an LED, can be done and the distance measurement accomplished using a CCD or CMOS array on a pixel by pixel basis, as discussed below.
Both laser and non-laser optical systems in general are good at determining the location of objects within the two-dimensional plane of the image and a pulsed laser radar system in the scanning mode can determine the distance of each part of the image from the receiver by measuring the time of flight such as through range gating techniques. Distance can also be determined by using modulated electromagnetic radiation and measuring the phase difference between the transmitted and received waves. It is also possible to determine distance with a non-laser system by focusing, or stereographically if two spaced-apart receivers are used and, in some cases, the mere location in the field of view can be used to estimate the position relative to the airbag, for example. Finally, a recently developed pulsed quantum well diode laser also provides inexpensive distance measurements as discussed in U.S. Pat. No. 6,324,453.
Acoustic systems are additionally quite effective at distance measurements since the relatively low speed of sound permits simple electronic circuits to be designed and minimal microprocessor capability is required. If a coordinate system is used where the z-axis is from the transducer to the occupant, acoustics are good at measuring z dimensions while simple optical systems using a single CCD or CMOS arrays are good at measuring x and y dimensions. The combination of acoustics and optics, therefore, permits all three measurements to be made from one location with low cost components as discussed in commonly assigned U.S. Pat. No. 5,845,000 and U.S. Pat. No. 5,835,613,
One example of a system using these ideas is an optical system which floods the passenger seat with infrared light coupled with a lens and a receiver array, e.g., CCD or CMOS array, which receives and displays the reflected light and an analog to digital converter (ADC) which digitizes the output of the CCD or CMOS and feeds it to an Artificial Neural Network (ANN) or other pattern recognition system for analysis. This system uses an ultrasonic transmitter and receiver for measuring the distances to the objects located in the passenger seat. The receiving transducer feeds its data into an ADC and from there, the converted data is directed into the ANN. The same ANN can be used for both systems thereby providing full three-dimensional data for the ANN to analyze. This system, using low cost components, will permit accurate identification and distance measurements not possible by either system acting alone. If a phased array system is added to the acoustic part of the system, the optical part can determine the location of the driver'"'"'s ears, for example, and the phased array can direct a narrow beam to the location and determine the distance to the occupant'"'"'s ears.
2. Adaptation
The adaptation of an occupant sensor system to a vehicle is the subject of a great deal of research and its own extensive body of knowledge as will be disclosed below. There is not believed to be any significant prior art in the field with the possible exception of the descriptions of sensor fusion methods in the Corrado et al. patents discussed above.
3. Mounting Locations for and Quantity of Transducers
There is little in the literature discussed herein concerning the mounting of cameras or other imagers or transducers in the vehicle other than in the current assignee'"'"'s patents referenced above. Where camera mounting is mentioned, the general locations chosen are the instrument panel, roof or headliner, A-Pillar or rear view mirror assembly. Virtually no discussion is provided as to the methodology for choosing a particular location except in the current assignee'"'"'s patents.
3.1 Single Camera, Dual Camera with Single Light Source
Farmer et al. (U.S. Pat. No. 6,005,958) describes a method and system for detecting the type and position of a vehicle occupant utilizing a single camera unit. The single camera unit is positioned at the driver or passenger side A-pillar in order to generate data of the front seating area of the vehicle. The type and position of the occupant is used to optimize the efficiency and safety in controlling deployment of an occupant protection device such as an air bag.
A single camera is, naturally, the least expensive solution but suffers from the problem that there is no easy method of obtaining three-dimensional information about people or objects in the passenger compartment. A second camera can be added, but to locate the same objects or features in the two images by conventional methods is computationally intensive unless the two cameras are close together. If they are close together, however, then the accuracy of the three dimensional information is compromised. Also, if they are not close together, then the tendency is to add separate illumination for each camera. An alternate solution is to use two cameras located at different positions in the passenger compartment and a single lighting source. This source can be located adjacent to one camera to minimize the installation sites. Since the LED illumination is now more expensive than the imager, the cost of the second camera does not add significantly to the system cost. The correlation of features can then be done using pattern recognition systems such as neural networks.
Two cameras also provide a significant protection from blockage and one or more additional cameras, with additional illumination, can be added to provide almost complete blockage protection.
3.2 Location of the Transducers
The only prior art for occupant sensor location for airbag control is White et al. and Mattes et al. discussed above. Both place their sensors below or on the instrument panel. The first disclosure of the use of cameras for occupant sensing is believed to appear in the current assignee'"'"'s above-referenced patents. The first disclosure of the location of a camera anywhere and especially above the instrument panel such as on the A-pillar, roof or rear view mirror assembly also is believed to appear in the current assignee'"'"'s above-referenced patents.
Corrado U.S. Pat. No. 6,318,697 discloses the placement of a camera onto a special type of rear view mirror. DeLine U.S. Pat. No. 6,124,886 also discloses the placement of a video camera on a rear view mirror for sending pictures using visible light over a cell phone. The general concept of placement of such a transducer on a mirror, among other places, is believed to have been first disclosed in commonly assigned U.S. RE037736 which also first discloses the use of an IR camera and IR illumination that is either co-located or located separately from the camera.
3.3 Color Cameras—Multispectral Imaging
The accurate detection, categorization and eventually recognition of an object in the passenger compartment are aided by using all available information. Initial camera-based systems are monochromic and use active and, in some cases, passive infrared. As microprocessors become more powerful and sensor systems improve, there will be a movement to broaden the observed spectrum to the visual spectrum and then further into the mid and far infrared parts of the spectrum. There is no known literature on this at this time except that provided by the current assignee below and in prior patents.
3.4 High Dynamic Range Cameras
The prior art of high dynamic range cameras centers around the work of the Fraunhofer-Inst. of Microelectronic Circuits & Systems in Duisburg, Germany and the Jet Propulsion Laboratory, licensed to Photobit, and is reflected in several patents including U.S. Pat. No. 5,471,515, U.S. Pat. No. 5,608,204, U.S. Pat. No. 5,635,753, U.S. Pat. No. 5,892,541, U.S. Pat. No. 6,175,383, U.S. Pat. No. 6,215,428, U.S. Pat. No. 6,388,242, and U.S. Pat. No. 6,388,243. The current assignee is believed to be the first to recognize and apply this technology for occupant sensing as well as monitoring the environment surrounding the vehicle and thus there is not believed to be any prior art for this application of the technology.
Related to this is the work done at Columbia University by Professor Nayar as disclosed in PCT patent application WO0079784 assigned to Columbia University, which is also applicable to monitoring the interior and exterior of the vehicle. An excellent technical paper also describes this technique: Nayar, S. K. and Mitsunaga, T. “High Dynamic Range Imaging: Spatially Varying Pixel Exposures” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, June 2000. Again, there does not appear to be any prior art that predates the disclosure of this application of the technology by the current assignee.
A paper entitled “A 256×256 CMOS Brightness Adaptive Imaging Array with Column-Parallel Digital Output” by C. Sodini et al., 1988 IEEE International Conference on Intelligent Vehicles, describes a CMOS image sensor for intelligent transportation system applications such as adaptive cruise control and traffic monitoring. Among the purported novelties is the use of a technique for increasing the dynamic range in a CMOS imager by a factor of approximately 20, which technique is based on a previously described technique for CCD imagers.
Waxman et al. U.S. Pat. No. 5,909,244 discloses a novel high dynamic range camera that can be used in low light situations with a frame rate >25 frames per second for monitoring either the interior or exterior of a vehicle. It is suggested that this camera can be used for automotive navigation but no mention is made of its use for safety monitoring. Similarly, Savoye et al. U.S. Pat. No. 5,880,777 disclose a high dynamic range imaging system similar to that described in the '"'"'244 patent that could be employed in the inventions disclosed herein.
There are numerous technical papers of high dynamic range cameras and some recent ones discuss automotive applications, after the concept was first discussed in the current assignee'"'"'s patents and patent applications. One recent example is T. Lulé, H. Keller, M. Wagner, M. Böhm, C. D. Hamann, L. Humm, U. Efron, “100.000 Pixel 120 dB Imager for Automotive Vision”, presented in the Proceedings of the Conference on Advanced Microsystems for Automotive Applications (AMAA), Berlin, 18./19. March 1999. This paper discusses the desirability of a high dynamic range camera and points out that an integration-based method is preferable to a logarithmic system in that greater contrast is potentially obtained. This brings up the question as to what dynamic range is really needed. The current assignee has considered desiring a high dynamic range camera but after more careful consideration, it is really the dynamic range within a given image that is important and that is usually substantially below 120 db, and in fact, a standard 70+ db camera is fine for most purposes.
As long as the shutter or an iris can be controlled to chose where the dynamic range starts, then, for night imaging a source of illumination is generally used and for imaging in daylight, the shutter time or iris can be substantially controlled to provide an adequate image. For those few cases where there is a very bright sunlight entering the vehicle'"'"'s window but the interior is otherwise in shade, multiple exposures can provide the desired contrast as taught by Nayar and discussed above. This is not to say that a high dynamic range camera is inherently bad, just to illustrate that there are many technologies that can be used to accomplish the same goal.
3.5 Fisheye Lens, Pan and Zoom
There is significant prior art on the use of a fisheye or similar high viewing angle lens and a non-moving pan, tilt, rotation and zoom cameras; however, there appears to be no prior art on the application of these technologies to sensing inside or outside of the vehicle prior to the disclosure by the current assignee. One significant patent is U.S. Pat. No. 5,185,667 to Zimmermann. For some applications, the use of a fisheye type lens can significantly reduce the number of imaging devices that are required to monitor the interior or exterior of a vehicle. An important point is that whereas for human viewing, the images are usually mathematically corrected to provide a recognizable view, when a pattern recognition system such as a neural network is used, it is frequently not necessary to perform this correction, thus simplifying the analysis.
Recently, a paper has been published that describes the fisheye camera system disclosed years ago by the current assignee: V. Ramesh, M. Greiffenhagen, S. Boverie, A. Giratt, “Real-Time Surveillance and Monitoring for Automotive Applications”, SAE 2000-01-0347.
4. 3D Cameras
4.1 Stereo
European Patent Application No. EP0885782A1 describes a purportedly novel motor vehicle control system including a pair of cameras which operatively produce first and second images of a passenger area. A distance processor determines the distances that a plurality of features in the first and second images are from the cameras based on the amount that each feature is shifted between the first and second images. An analyzer processes the determined distances and determines the size of an object on the seat. Additional analysis of the distance also may determine movement of the object and the rate of movement. The distance information also can be used to recognize predefined patterns in the images and thus identify objects. An air bag controller utilizes the determined object characteristics in controlling deployment of the air bag.
Simoncelli in U.S. Pat. No. 5,703,677 discloses an apparatus and method using a single lens and single camera with a pair of masks to obtain three-dimensional information about a scene.
A paper entitled “Sensing Automobile Occupant Position with Optical Triangulation” by W. Chappelle, Sensors, December 1995, describes the use of optical triangulation techniques for determining the presence and position of people or rear-facing infant seats in the passenger compartment of a vehicle in order to guarantee the safe deployment of an air bag. The paper describes a system called the “Takata Safety Shield” which purportedly makes high-speed distance measurements from the point of air bag deployment using a modulated infrared beam projected from an LED source. Two detectors are provided, each consisting of an imaging lens and a position-sensing detector.
A paper entitled “An Interior Compartment Protection System based on Motion Detection Using CMOS Imagers” by S. B. Park et al., 1998 IEEE International Conference on Intelligent Vehicles, describes a purportedly novel image processing system based on a CMOS image sensor installed at the car roof for interior compartment monitoring including theft prevention and object recognition. One disclosed camera system is based on a CMOS image sensor and a near infrared (NIR) light emitting diode (LED) array.
Krumm (U.S. Pat. No. 5,983,147) describes a system for determining the occupancy of a passenger compartment including a pair of cameras mounted so as to obtain binocular stereo images of the same location in the passenger compartment. A representation of the output from the cameras is compared to stored representations of known occupants and occupancy situations to determine which stored representation the output from the cameras most closely approximates. The stored representations include that of the presence or absence of a person or an infant seat in the front passenger seat.
The use of stereo systems for occupant sensing was first described by the current assignee in RE37736, U.S. Pat. No. 5,845,000, U.S. Pat. No. 5,835,613, U.S. Pat. No. 6,186,537, and U.S. Pat. No. 5,848,802 among others.
4.2 Distance by Focusing
A mechanical focusing system, such as used on some camera systems, can determine the initial position of an occupant but is currently too slow to monitor his/her position during a crash or even during pre-crash braking. Although the example of an occupant is used here as an example, the same or similar principles apply to objects exterior to the vehicle. This is a result of the mechanical motions required to operate the lens focusing system, however, methods do exist that do not require mechanical motions. By itself, it cannot determine the presence of a rear facing child seat or of an occupant but when used with a charge-coupled or CMOS device plus some infrared illumination for vision at night, and an appropriate pattern recognition system, this becomes possible. Similarly, the use of three-dimensional cameras based on modulated waves or range-gated pulsed light methods combined with pattern recognition systems are now possible based on the teachings of the inventions disclosed herein and the commonly assigned patents and patent applications referenced above.
U.S. Pat. No. 6,198,998 to Farmer discloses a single IR camera mounted on the A-Pillar where a side view of the contents of the passenger compartment can be obtained. A sort of three-dimensional view is obtained by using a narrow depth of focus lens and a de-blurring filter. IR is used to illuminate the volume and the use of a pattern on the LED to create a sort of structured light is also disclosed. Pattern recognition by correlation is also discussed.
U.S. Pat. No. 6,229,134 to Nayar et al. is an excellent example of the determination of the three-dimensional shape of an object using active blurring and focusing methods. The use of structured light is also disclosed in this patent. The method uses illumination of the scene with a pattern and two images of the scene are sensed with different imaging parameters.
A distance measuring system based on focusing is described in U.S. Pat. No. 5,193,124 and U.S. Pat. No. 5,231,443 (Subbarao) that can either be used with a mechanical focusing system or with two cameras, the latter of which would be fast enough to allow tracking of an occupant during pre-crash braking and perhaps even during a crash depending on the field of view that is analyzed. Although the Subbarao patents provide a good discussion of the camera focusing art, it is a more complicated system than is needed for practicing the instant inventions. In fact, a neural network can also be trained to perform the distance determination based on the two images taken with different camera settings or from two adjacent CCD'"'"'s and lens having different properties as the cameras disclosed in Subbarao making this technique practical for the purposes herein. Distance can also be determined by the system disclosed in U.S. Pat. No. 5,003,166 (Girod) by spreading or defocusing a pattern of structured light projected onto the object of interest. Distance can also be measured by using time of flight measurements of the electromagnetic waves or by multiple CCD or CMOS arrays as is a principle teaching of at least one of the inventions disclosed herein.
Dowski, Jr. in U.S. Pat. No. 5,227,890 provides an automatic focusing system for video cameras which can be used to determine distance and thus enable the creation of a three-dimensional image.
A good description of a camera focusing system is found in G. Zorpette, “Focusing in a flash”, Scientific American, August 2000.
In each of these cases, regardless of the distance measurement system used, a trained pattern recognition system, as defined above, can be used to identify and classify, and in some cases to locate, the illuminated object and its constituent parts.
4.3 Ranging
Cameras can be used for obtaining three dimensional images by modulation of the illumination as described in U.S. Pat. No. 5,162,861. The use of a ranging device for occupant sensing is believed to have been first disclosed by the current assignee in the patents mentioned herein. More recent attempts include the PMD camera as disclosed in PCT application WO09810255 and similar concepts disclosed in U.S. Pat. No. 6,057,909 and U.S. Pat. No. 6,100,517.
A paper by Rudolf Schwarte, et al. entitled “New Powerful Sensory Tool in Automotive Safety Systems Based on PMD-Technology”, Eds. S. Krueger, W. Gessner, Proceedings of the AMAA 2000 Advanced Microsystems for Automotive Applications 2000, Springer Verlag; Berlin, Heidelberg, N.Y., ISBN 3-540-67087-4, describes an implementation of the teachings of the instant invention wherein a modulated light source is used in conjunction with phase determination circuitry to locate the distance to objects in the image on a pixel by pixel basis. This camera is an active pixel camera the use of which for internal and external vehicle monitoring is also a teaching of at least one of the inventions disclosed herein. The novel feature of the PMD camera is that the pixels are designed to provide a distance measuring capability within each pixel itself. This then is a novel application of the active pixel and distance measuring teachings of the instant invention.
The paper “Camera Records Color and Depth”, Laser Focus World, Vol. 36, No. 7, July 2000, describes another method of using modulated light to measure distance.
“Seeing distances-a fast time-of-flight 3D camera”, Sensor Review, Vol. 20, No. 3, 2000, presents a time-of-flight camera that also can be used for internal and external monitoring. Similarly, see “Electro-optical correlation arrangement for fast 3D cameras: properties and facilities of the electro-optical mixer device”, SPIE Vol. 3100, 1997 pp. 254–60. A significant improvement to the PMD technology and to all distance by modulation technologies is to modulate with a code, which can be random or pseudo random, that permits accurate distance measurements over a long range using correlation or other technology. There is a question as to whether there is a need to individually modulate each pixel with the sent signal since the same effect can be achieved using a known Pockel or Kerr cell that covers the entire imager, which should be simpler.
The instant invention as described in the above-referenced commonly assigned patents and patent applications, teaches the use of modulating the light used to illuminate an object and to determine the distance to that object based on the phase difference between the reflected radiation and the transmitted radiation. The illumination can be modulated at a single frequency when short distances such as within the passenger compartment are to be measured. Typically, the modulation wavelength would be selected such that one wave would have a length of approximately one meter or less. This would provide resolution of 1 cm or less.
For larger vehicles, a longer wavelength would be desirable. For measuring longer distances, the illumination can be modulated at more than one frequency to eliminate cycle ambiguity if there is more than one cycle between the source of illumination and the illuminated object. This technique is particularly desirable when monitoring objects exterior to the vehicle to permit accurate measurements of devices that are hundreds of meters from the vehicle as well as those that are a few meters away. Naturally, there are other modulation methods that eliminate the cycle ambiguity such as modulation with a code that is used with a correlation function to determine the phase shift or time delay. This code can be a pseudo random number in order to permit the unambiguous monitoring of the vehicle exterior in the presence of other vehicles with the same system. This is sometimes known as noise radar, noise modulation (either of optical or radar signals), ultra wideband (UWB) or the techniques used in Micropower impulse radar (MIR). Another key advantage is to permit the separation of signals from multiple vehicles.
Although a simple frequency modulation scheme has been disclosed so far, it is also possible to use other coding techniques including the coding of the illumination with one of a variety of correlation patterns including a pseudo-random code. Similarly, although frequency and code domain systems have been described, time domain systems are also applicable wherein a pulse of light is emitted and the time of flight measured. Additionally, in the frequency domain case, a chirp can be emitted and the reflected light compared in frequency with the chirp to determine the distance to the object by frequency difference. Although each of these techniques is known to those skilled in the art, they have previously heretofore not been applied for monitoring objects within or outside of a vehicle.
4.4 Pockel or Kerr Cells for Determining Range
The technology for modulating a light valve or electronic shutter has been known for many years and is sometimes referred to as a Kerr cell or a Pockel cell. These devices are capable of being modulated at up to 10 billion cycles per second. For determining the distance to an occupant or his or her features, modulations between 100 and 500 MHz are needed. The higher the modulation frequency, the more accurate the distance to the object can be determined. However, if more than one wavelength, or better one-quarter wavelength, exists between the camera and the object, then ambiguities result. On the other hand, once a longer wavelength has ascertained the approximate location of the feature, then more accurate determinations can be made by increasing the modulation frequency since the ambiguity will now have been removed. In practice, only a single frequency is used of about 300 MHz. This gives a wavelength of 1 meter, which can allow cm level distance determinations.
In one preferred embodiment of at least one of the inventions disclosed herein, an infrared LED is modulated at a frequency between 100 and 500 MHz and the returning light passes through a light valve such that amount of light that impinges on the CMOS array pixels is determined by a phase difference between the light valve and the reflected light. By modulating a light valve for one frame and leaving the light valve transparent for a subsequent frame, the range to every point in the camera field of view can be determined based on the relative brightness of the corresponding pixels.
Once the range to all of the pixels in the camera view has been determined, range-gating becomes a simple mathematical exercise and permits objects in the image to be easily separated for feature extraction processing. In this manner, many objects in the passenger compartment can be separated and identified independently.
Noise, pseudo noise or code modulation techniques can be used in place of the frequency modulation discussed above. This can be in the form of frequency, amplitude or pulse modulation.
No prior art is believed to exist on this concept.
4.5 Thin Film on ASIC (TFA)
Thin film on ASIC technology, as described in Lake, D. W. “TFA Technology: The Coming Revolution in Photography”, Advanced Imaging Magazine, April, 2002 (WWW.ADVANCEDIMAGINGMAG.COM) shows promise of being the next generation of imager for automotive applications. The anticipated specifications for this technology, as reported in the Lake article, are:
All of these specifications, except for the frame rate, are attractive for occupant sensing. It is believed that the frame rate can be improved with subsequent generations of the technology or more than one imager can be used. Some advantages of this technology for occupant sensing include the possibility of obtaining a three-dimensional image by varying the pixel in time in relation to a modulated illumination in a simpler manner than proposed with the PMD imager or with a Pockel or Kerr cell. The ability to build the entire package on one chip will reduce the cost of this imager compared with two or more chips required by current technology.
Other technical papers on TFA include: (1) M. Böhm “Imagers Using Amorphous Silicon Thin Film on ASIC (TFA) Technology”, Journal of Non-Crystalline Solids, 266–269, pp. 1145–1151, 2000; (2) A. Eckhardt, F. Blecher, B. Schneider, J. Sterzel, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, K. Seibel, F. Mütze, M. Böhm, “Image Sensors in TFA (Thin Film on ASIC) Technology with Analog Image Pre-Processing”, H. Reichl, E. Obermeier (eds.), Proc. Micro System Technologies 98, Potsdam, Germany, pp. 165–170, 1998; (3) T. Lulé, B. Schneider, M. Böhm, “Design and Fabrication of a High Dynamic Range Image Sensor in TFA Technology”, invited paper for IEEE Journal of Solid-State Circuits, Special Issue on 1998 Symposium on VLSI Circuits, 1999. (4) M. Böhm, F. Blecher, A. Eckhardt, B. Schneider, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, R. C. Lind, L. Humm, M. Daniels, N. Wu, H. Yen, “High Dynamic Range Image Sensors in Thin Film on ASIC—Technology for Automotive Applications”, D. E. Ricken, W. Gessner (eds.), Advanced Microsystems for Automotive Applications, Springer-Verlag, Berlin, pp. 157–172, 1998. (5) M. Böhm, F. Blecher, A. Eckhardt, K. Seibel, B. Schneider, J. Sterzel, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, B. Van Uffel, F Librecht, R. C. Lind, L. Humm, U. Efron, E. Rtoh, “Image Sensors in TFA Technology—Status and Future Trends”, Mat. Res. Soc. Symp. Proc., vol. 507, pp. 327–338, 1998.
5. Glare Control
U.S. Pat. No. 5,298,732 and U.S. Pat. No. 5,714,751 to Chen concentrate on locating the eyes of the driver so as to position a light filter between a light source such as the sun or the lights of an oncoming vehicle, and the driver'"'"'s eyes. This patent will be discussed in more detail below. U.S. Pat. No. 5,305,012 to Faris also describes a system for reducing the glare from the headlights of an oncoming vehicle and it is discussed in more detail below.
5.1 Windshield
Using an advanced occupant sensor, as explained below, the position of the driver'"'"'s eyes can be accurately determined and portions of the windshield, or of a special visor, can be selectively darkened to eliminate the glare from the sun or oncoming vehicle headlights. This system can use electro-chromic glass, a liquid crystal device, Xerox Gyricon, Research Frontiers SPD, semiconducting and metallic (organic) polymer displays, spatial light monitors, electronic “Venetian blinds”, electronic polarizers or other appropriate technology, and, in some cases, detectors to detect the direction of the offending light source. In addition to eliminating the glare, the standard sun visor can now also be eliminated. Alternately, the glare filter can be placed in another device such as a transparent sun visor that is placed between the driver'"'"'s eyes and the windshield.
There is no known prior art that places a filter in the windshield. All known designs use an auxiliary system such as a liquid crystal panel that acts like a light valve on a pixel by pixel basis.
A description of SPD can be found at SmartGlass.com and in “New ‘Smart’ glass darkens, lightens in a flash”, Automotive News, Aug. 21, 1998.
5.2 Glare in Rear View Mirrors
There is no known prior art that places a pixel-addressable filter in a rear view mirror to selectively block glare or for any other purpose.
5.3 Visor for Glare Control and HUD
The prior art related to visors for glare control and heads-up displays includes U.S. Pat. No. 4,874,938, U.S. Pat. No. 5,298,732, U.S. Pat. No. 5,305,012 and U.S. Pat. No. 5,714,751 which are discussed elsewhere herein.
6. Weight Measurement and Biometrics
Prior art systems are now being used to identify the vehicle occupant based on a coded key or other object carried by the occupant. This requires special sensors within the vehicle to recognize the coded object. Also, the system only works if the particular person for whom the vehicle was programmed uses the coded object. If a son or daughter, for example, who is using their mother'"'"'s key, uses the vehicle, then the wrong seat, mirror, radio station etc. adjustments are made. Also, these systems preserve the choice of seat position without any regard for the correctness of the seat position. With the problems associated with the 4-way seats, it is unlikely that the occupant ever properly adjusts the seat. Therefore, the error in seat position will be repeated every time the occupant uses the vehicle.
These coded systems are a crude attempt to identify the occupant. An improvement can be made if morphological (or biological) characteristics of the occupant can be measured as described herein. Such measurements can be made of the height and weight, for example, and used not only to adjust a vehicular component to a proper position but also to remember that position, as fine tuned by the occupant, for re-positioning the component the next time the occupant occupies the seat. No prior art is believed to exist on this aspect of the invention. Additional biometrics includes physical and behavioral responses of the eyes, hands, face and voice. Iris and retinal scans are discussed in the literature but the shape of the eyes or hands, structure of the face or hands, how a person blinks or squints, the shape of the hands, how he or she grasps the steering wheel, the electrical conductivity or dielectric constant, blood vessel pattern in the hands, fingers, face or elsewhere, the temperature and temperature differences of different areas of the body, the natural effluent or odor of the person are among the many biometric variables that can be measures to identify an authorized user of a vehicle, for example.
As discussed more fully below, in a preferred implementation, once at least one and preferably two of the morphological characteristics of a driver are determined, for example by measuring his or her height and weight, the component such as the seat can be adjusted and other features or components can be incorporated into the system including, for example, the automatic adjustment of the rear view and/or side mirrors based on seat position and occupant height.
In addition, a determination of an out-of-position occupant can be made and based thereon, airbag deployment suppressed if the occupant is more likely to be injured by the airbag than by the accident without the protection of the airbag. Furthermore, the characteristics of the airbag, including the amount of gas produced by the inflator and the size of the airbag exit orifices, can be adjusted to provide better protection for small lightweight occupants as well as large, heavy people. Even the direction of the airbag deployment can, in some cases, be controlled. The prior art is limited to airbag suppression as disclosed in Mattes (U.S. Pat. No. 5,118,134) and White (U.S. Pat. No. 5,071,160) discussed above.
Still other features or components can now be adjusted based on the measured occupant morphology as well as the fact that the occupant can now be identified. Some of these features or components include the adjustment of seat armrest, cup holder, steering wheel (angle and telescoping), pedals, phone location and for that matter, the adjustment of all things in the vehicle which a person must reach or interact with. Some items that depend on personal preferences can also be automatically adjusted including the radio station, temperature, ride and others.
6.1 Strain Gage Weight Sensors
Previously, various methods have been proposed for measuring the weight of an occupying item of a vehicular seat. The methods include pads, sheets or films that have placed in the seat cushion which attempt to measure the pressure distribution of the occupying item. Prior to its first disclosure in Breed et al. (U.S. Pat. No. 5,822,707), systems for measuring occupant weight based on the strain in the seat structure had not been considered. Prior art weight measurement systems have been notoriously inaccurate. Thus, a more accurate weight measuring system is desirable. The strain measurement systems described herein, substantially eliminate the inaccuracy problems of prior art systems and permit an accurate determination of the weight of the occupying item of the vehicle seat. Additionally, as disclosed herein, in many cases, sufficient information can be obtained for the control of a vehicle component without the necessity of determining the entire weight of the occupant. For example, the force that the occupant exerts on one of the three support members may be sufficient.
A recent U.S. patent application, Publication No. 2003/0168895, is interesting in that it is the first example of the use of time and the opening and closing of a vehicle door to help in the post-processing decision making for distinguishing a child restraint system (CRS) from an adult. This system is based on a load cell (strain gage) weight measuring system.
Automotive vehicles are equipped with seat belts and air bags as equipment for ensuring the safety of the passenger. In recent years, an effort has been underway to enhance the performance of the seat belt and/or the air bag by controlling these devices in accordance with the weight or the posture of the passenger. For example, the quantity of gas used to deploy the air bag or the speed of deployment could be controlled. Further, the amount of pretension of the seat belt could be adjusted in accordance with the weight and posture of the passenger. To this end, it is necessary to know the weight of the passenger sitting on the seat by some technique. The position of the center of gravity of the passenger sitting on the seat could also be referenced in order to estimate the posture of the passenger.
As an example of a technique to determine the weight or the center of gravity of the passenger of this type, a method of measuring the seat weight including the passenger'"'"'s weight by disposing the load sensors (load cells) at the front, rear, left and right corners under the seat and summing vertical loads applied to the load cells has been disclosed in the assignee'"'"'s numerous patents and patent applications on occupant sensing.
Since a seat weight measuring apparatus of this type is intended for use in general automotive vehicles, the cost of the apparatus must be as low as possible. In addition, the wiring and assembly also must be easy. Keeping such considerations in mind, the object of the present invention is to provide a seat weight measuring apparatus having such advantages that the production cost and the assembling cost may be reduced. To provide new and improved vehicular seats in which the weight applied by an occupying item to the seat is measured based on capacitance between conductive and/or metallic members underlying the seat cushion.
A further object of an invention herein is to provide new and improved adjustment apparatus and methods that evaluate the occupancy of the seat and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat and on a measurement of the occupant'"'"'s weight or a measurement of a force exerted by the occupant on the seat.
6.2 Bladder Weight Sensors
Similarly to strain gage weight sensors, the first disclosure of weight sensors based of the pressure in a bladder in or under the seat cushion is believed to have been made in Breed et al. (U.S. Pat. No. 5,822,707) filed Jun. 7, 1995 by the current assignee.
A bladder is disclosed in WO09830411, which claims the benefit of a U.S. provisional application filed on Jan. 7, 1998 showing two bladders. This patent application is assigned to Automotive Systems Laboratory and is part of a series of bladder based weight sensor patents and applications all of which were filed significantly after the current assignee'"'"'s bladder weight sensor patent applications, the earliest filing date being in 1997.
Also U.S. Pat. No. 4,957,286 illustrates a single chamber bladder sensor for an exercise bicycle which measures the weight of a person as he or she in exercising but is not used in a vehicle nor is it used for controlling a safety device or any other component. EP0345806 illustrates a bladder in an automobile seat for the purpose of adjusting the shape of the seat. Although a pressure switch is provided, no attempt is made to measure the weight of the occupant and there is no mention of using the weight to control a vehicle component. IEE of Luxemburg and others have marketed seat sensors that measure the pattern on the object contacting the seat surface but none of these sensors purport to measure the weight of an occupying item of the seat.
6.3 Dynamic Weight Sensing
There does not appear to be any prior art regarding the use of the motion of the vehicle and its contents to dynamically measure the weight of an occupying item.
6.4 Combined Spatial and Weight Sensors
The combination of a weight sensor with a spatial sensor, such as the wave or electric field sensors discussed herein, permits the most accurate determination of the airbag requirements when the crash sensor output is also considered. There is not believed to be any prior art of such a combination. A recent patent, which is not considered prior art, that discloses a similar concept is U.S. Pat. No. 6,609,055.
6.5 Face Recognition (Face and Iris IR Scans)
Ishikawa et al. (U.S. Pat. No. 4,625,329) describes an image analyzer (M5 in FIG. 1) for analyzing the position of driver based on the position of the driver'"'"'s face, including an infrared light source which illuminates the driver'"'"'s face and an image detector which receives light from the driver'"'"'s face, determines the position of facial feature, e.g., the eyes in three dimensions, and thus determines the position of the driver'"'"'s face in three dimensions. A pattern recognition process is used to determine the position of the facial features and entails converting the pixels forming the image to either black or white based on intensity and conducting an analysis based on the white area in order to find the largest contiguous white area and the center point thereof. Based on the location of the center point of the largest contiguous white area, the driver'"'"'s height is derived and a heads-up display is adjusted so information is within driver'"'"'s field of view. The pattern recognition process can be applied to detect the eyes, mouth, or nose of the driver based on the differentiation between the white and black areas. Ishikawa does not attempt to recognize the driver or to determine the location of the driver relative to an airbag or any other vehicle component.
Ando (U.S. Pat. No. 5,008,946) describes a system which recognizes an image and specifically ascertains the position of the pupils and mouth of the occupant to enable movement of the pupils and mouth to control electrical devices installed in the automobile. The system includes a camera which takes a picture of the occupant and applies algorithms based on pattern recognition techniques to analyze the picture, converted into an electrical signal, to determine the position of certain portions of the image, namely the pupils and mouth. Ando also does not attempt to recognize the driver.
Puma (U.S. Pat. No. 5,729,619) describes apparatus and methods for determining the identity of a vehicle operator and whether he or she is intoxicated or falling asleep. Puma uses an iris scan as the identification method and thus requires the driver to place his eyes in a particular position relative to the camera. Intoxication is determined by monitoring the spectral emission from the driver'"'"'s eyes and drowsiness is determined by monitoring a variety of behaviors of the driver. The identification of the driver by any means is believed to have been first disclosed in the current assignee'"'"'s patents referenced above as was identifying the impairment of the driver whether by alcohol, drugs or drowsiness through monitoring driver behavior and using pattern recognition. Puma uses pattern recognition but not neural networks although correlation analysis is implied as also taught in the current assignee'"'"'s prior patents.
Other patents on eye tracking include Moran et al. (U.S. Pat. No. 4,847,486) and Hutchinson (U.S. Pat. No. 4,950,069). In Moran et al., a scanner is used to project a beam onto the eyes of the person and the reflection from the retina through the cornea is monitored to measure the time that the person'"'"'s eyes are closed. In Hutchinson, the eye of a computer operator is illuminated with light from an infrared LED and the reflected light causes bright eye effect which outlines the pupil brighter than the rest of the eye and also causes an even brighter reflection from the cornea. By observing this reflection in the camera'"'"'s field of view, the direction in which the eye is pointing can be determined. In this manner, the motion of the eye can control operation of the computer. Similarly, such apparatus can be used to control various functions within the vehicle such as the telephone, radio, and heating and air conditioning.
U.S. Pat. No. 5,867,587 to Aboutalib et al. also describes a drowsy driver detection unit based on the frequency of eye blinks where an eye blink is determined by correlation analysis with averaged previous states of the eye. U.S. Pat. No. 6,082,858 to Grace describes the use of two frequencies of light to monitor the eyes, one that is totally absorbed by the eye (950 nm) and another that is not and where both are equally reflected by the rest of the face. Thus, subtraction leaves only the eyes. An alternative, not disclosed by Aboutalib et al. or Grace, is to use natural light or a broad frequency spectrum and a filter to filter out all frequencies except 950 nm and then to proportion the intensities. U.S. Pat. No. 6,097,295 to Griesinger also attempts to determine the alertness of the driver by monitoring the pupil size and the eye shutting frequency. U.S. Pat. No. 6,091,334 uses measurements of saccade frequency, saccade speed, and blinking measurements to determine drowsiness. No attempt is made in any of these patents to locate the driver in the vehicle.
There are numerous technical papers on eye location and tracking developed for uses other than automotive including: (1) “Eye Tracking in Advanced Interface Design”, Robert J. K. Jacob, Human-Computer Interaction Lab, Naval Research Laboratory, Washington, D.C.; (2) F. Smeraldi, O. Carmona, J. Bigün, “Saccadic search with Gabor features applied to eye detection and real-time head tracking”, Image and Vision Computing 18 (2000) 323–329, Elsevier; (3) Y. Wang, B. Yuan, “Human Eyes Location Using Wavelet and Neural Networks”, Proceedings of ICSP2000, IEEE; and (4) S. A. Sirohey, A. Rosenfeld, “Eye detection in a face image using linear and nonlinear filters”, Pattern Recognition 34 (2001) 1367–1391, Pergamon.
There are also numerous technical papers on human face recognition including: (1) “Pattern Recognition with Fast Feature Extractions”, M. G. Nakhodkin, Y. S. Musatenko, and V. N. Kurashov, Optical Memory and Neural Networks, Vol. 6, No. 3, 1997; and (2) C. Beumier, M. Acheroy “Automatic 3D Face Recognition”, Image and Vision Computing, 18 (2000) 315–321, Elsevier.
Since the direction of gaze of the eyes is quite precise and relatively easily measured, it can be used to control many functions in the vehicle such as the telephone, lights, windows, HVAC, navigation and route guidance system, and telematics among others. Many of these functions can be combined with a heads-up display and the eye gaze can replace the mouse in selecting many functions and among many choices. It can also be combined with an accurate mapping system to display on a convenient display the writing on a sign that might be hard to read such as a street sign. It can even display the street name when a sign is not present. A gaze at a building can elicit a response providing the address of the building or some information about the building which can be provided either orally or visually. Looking at the speedometer can elicit a response as the local speed limit and looking at the fuel gage can elicit the location of the nearest gas station. None of these functions appear in the prior art discussed above.
Other papers on finding the eyes of a subject are: Wang, Y., Yuan, B., “Human Eye Location Using Wavelet and Neural Network”, Proceedings of the IEEE Internal Conference on Signal Processing 2000, p 1233–1236, and Sirohey, S. A., Rosenfeld, A., “Eye detection in a face using linear and nonlinear filters”, Pattern Recognition 34 (2001) p 1367–1391, Elsevier Science Ltd. The Sirohey et al. article in particular, in addition to a review of the prior art, provides an excellent methodology for eye location determination. The technique makes use of face color to aid in face and eye location.
In all of the above references on eye tracking, natural or visible illumination is used. In a vehicle infrared illumination will be used so as to not distract the occupant. The eyes of a person are particularly noticeable under infrared illumination as discussed in Richards, A., Alien Vision, p. 6–9, 2001, SPIE Press, Bellingham, Wash. The use of infrared radiation to aid in location of the occupant'"'"'s eyes either by itself of along with natural or artificial radiation is a preferred implementation of the teachings of at least one of the inventions disclosed herein. This is illustrated in
6.6 Heartbeat and Health State
Although the concept of measuring the heartbeat of a vehicle occupant is believed to have originated with the current assignee, Bader in U.S. Pat. No. 6,195,008 uses a comparison of the heartbeat with stored data to determine the age of the occupant. Other uses of heartbeat measurement include determining the presence of an occupant on a particular seat, the determination of the total number of vehicle occupants, the presence of an occupant in a vehicle for security purposes, for example, and the presence of an occupant in the trunk etc.
6.7 Other Inputs
Many other inputs can be applied to the interior or exterior monitoring systems of the inventions disclosed herein. For interior monitoring, these can include, among others, the position of the seat and seatback, vehicle velocity, brake pressure, steering wheel position and motion, exterior temperature and humidity, seat weight sensors, accelerometers and gyroscopes, engine behavior sensors, tire monitors and chemical (oxygen, carbon dioxide, alcohol, etc.) sensors. For external monitoring, these can include, among others, temperature and humidity, weather forecasting information, traffic information, hazard warnings, speed limit information, time of day, lighting and visibility conditions and road condition information.
7. Illumination
7.1 Infrared Light
In a passive infrared system, as described in Corrado referenced above, for example, a detector receives infrared radiation from an object in its field of view, in this case the vehicle occupant, and determines the presence and temperature of the occupant based on the infrared radiation. The occupant sensor system can then respond to the temperature of the occupant, which can either be a child in a rear facing child seat or a normally seated occupant, to control some other system. This technology could provide input data to a pattern recognition system but it has limitations related to temperature.
The sensing of the child could pose a problem if the child is covered with blankets, depending on the IR frequency used. It also might not be possible to differentiate between a rear facing child seat and a forward facing child seat. In all cases, the technology can fail to detect the occupant if the ambient temperature reaches body temperature as it does in hot climates. Nevertheless, for use in the control of the vehicle climate, for example, a passive infrared system that permits an accurate measurement of each occupant'"'"'s temperature is useful. Prior art systems are mostly limited to single pixel devices. Use of an IR imager removes many of the problems listed above and is believed to be novel to the inventions disclosed herein.
In a laser optical system, an infrared laser beam is used to momentarily illuminate an object, occupant or child seat in the manner as described, and illustrated in FIG. 8, of Breed et al. (U.S. Pat. No. 5,653,462). In some cases, a CCD or a CMOS device is used to receive the reflected light. In other cases, when a scanning laser is used, a pin or avalanche diode or other photo detector can be used. The laser can either be used in a scanning mode, or, through the use of a lens, a cone of light, swept line of light, or a pattern or structured light can be created which covers a large portion of the object. Additionally, one or more LEDs can be used as a light source. Also, triangulation can be used in conjunction with an offset scanning laser to determine the range of the illuminated spot from the light detector. Various focusing systems also can have applicability in some implementations to measure the distance to an occupant. In most cases, a pattern recognition system, as defined herein, is used to identify, ascertain the identity of and classify, and can be used to locate, and determine the position of, the illuminated object and/or its constituent parts.
Optical systems generally provide the most information about the object and at a rapid data rate. Their main drawback is cost which is usually above that of ultrasonic or passive infrared systems. As the cost of lasers and imagers has now come down, this system is now competitive. Depending on the implementation of the system, there may be some concern for the safety of the occupant if a laser light can enter the occupant'"'"'s eyes. This is minimized if the laser operates in the infrared spectrum particularly at the “eye-safe” frequencies.
Another important feature is that the brightness of the point of light from the laser, if it is in the infrared part of the spectrum and if a filter is used on the receiving detector, can overpower the reflected sun'"'"'s rays with the result that the same classification algorithms can be made to work both at night and under bright sunlight in a convertible. An alternative approach is to use different algorithms for different lighting conditions.
Although active and passive infrared light has been disclosed in the prior art, the use of a scanning laser, modulated light, filters, trainable pattern recognition etc. is believed to have been first disclosed by the current assignee in the above-referenced patents.
7.2 Structured Light
U.S. Pat. No. 5,003,166 provides an excellent treatise on the use of structured light for range mapping of objects in general. It does not apply this technique for automotive applications and in particular for occupant sensing or monitoring inside or outside of a vehicle. The use of structured light in the automotive environment and particularly for sensing occupants is believed to have been first disclosed by the current assignee in the above-referenced patents.
U.S. Pat. No. 6,049,757 to Nakajima et al. describes structured light in the form of bright spots that illuminate the face of the driver to determine the inclination of the face and to issue a warning if the inclination is indicative of a dangerous situation. In the current assignee'"'"'s patents, structured light is disclosed to obtain a determination of the location of an occupant and/or his or her parts. This includes the position of any part of the occupant including the occupant'"'"'s face and thus the invention of this patent is believed to be anticipated by the current assignee'"'"'s patents referenced above.
U.S. Pat. No. 6,298,311 to Griffin et al. repeats much of the teachings of the early patents of the current assignee. A plurality of IR beams are modulated and directed in the vicinity of the passenger seat and used through a photosensitive receiver to detect the presence and location of an object in the passenger seat, although the particular pattern recognition system is not disclosed. The pattern of IR beams used in this patent is a form of structured light.
Structured light is also discussed in numerous technical papers for other purposes than vehicle interior or exterior monitoring including: (1) “3D Shape Recovery and Registration Based on the Projection of Non-Coherent Structured Light” by Roberto Rodella and Giovanna Sansoni, INFM and Dept. of Electronics for the Automation, University of Brescia, Via Branze 38, 1-25123 Brescia—Italy; (2) “A Low-Cost Range Finder using a Visually Located, Structured Light Source”, R. B. Fisher, A. P. Ashbrook, C. Robertson, N. Werghi, Division of Informatics, Edinburgh University, 5 Forrest Hill, Edinburgh EHI 2QL; (3) F. Lerasle, J. Lequellec, M Devy, “Relaxation vs. Maximal Cliques Search for Projected Beams Labeling in a Structured Light Sensor”, Proceedings of the International Conference on Pattern Recognition, 2000 IEEE; and (4) D. Caspi, N. Kiryati, and J. Shamir, “Range Imaging With Adaptive Color Structured Light”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 5, May 1998.
Recently, a paper has been published that describes a structured light camera system disclosed years ago by the current assignee: V. Ramesh, M. Greiffenhagen, S. Boverie, A. Giratt, “Real-Time Surveillance and Monitoring for Automotive Applications”, SAE 2000-01-0347.
7.3 Color and Natural Light
A number of systems have been disclosed that use illumination as the basis for occupant detection. The problem with artificial illumination is that it will not always overpower the sun and thus in a convertible on a bright sunny day, for example, the artificial light can be undetectable unless it is a point. If one or more points of light are not the illumination of choice, then the system must also be able to operate under natural light. The inventions herein accomplish the feat of accurate identification and tracking of an occupant under all lighting conditions by using artificial illumination at night and natural light when it is available. This requires that the pattern recognition system be modular with different modules used for different situations as discussed in more detail below. There is no known prior art for using natural radiation for occupant sensing systems.
When natural illumination is used, a great deal of useful information can be obtained if various parts of the electromagnetic spectrum are used. The ability to locate the face and facial features is enhanced if color is used, for example. Once again, there is no known prior art for the use of color, for example. All known systems that use electromagnetic radiation are monochromatic.
7.4 Radar
The radar portion of the electromagnetic spectrum can also be used for occupant detection as first disclosed by the current assignee in the above-referenced patents. Radar systems have similar properties to the laser system discussed above except the ability to focus the beam, which is limited in radar by the frequency chosen and the antenna size. It is also much more difficult to achieve a scanning system for the same reasons. The wavelength of a particular radar system can limit the ability of the pattern recognition system to detect object features smaller than a certain size. Once again, however, there is some concern about the health effects of radar on children and other occupants. This concern is expressed in various reports available from the United States Food and Drug Administration, Division of Devices.
When the occupying item is human, in some instances the information about the occupying item can be the occupant'"'"'s position, size and/or weight. Each of these properties can have an effect on the control criteria of the component. One system for determining a deployment force of an air bag system in described in U.S. Pat. No. 6,199,904 (Dosdall). This system provides a reflective surface in the vehicle seat that reflects microwaves transmitted from a microwave emitter. The position, size and weight of a human occupant are said to be determined by calibrating the microwaves detected by a detector after the microwaves have been reflected from the reflective surface and pass through the occupant. Although some features disclosed in the '"'"'904 patent are not disclosed in the current assignee'"'"'s above-referenced patents, the use of radar in general for occupant sensing is disclosed in those patents.
7.5 Frequency or Spectrum Considerations
As discussed above, it is desirable to obtain information about an occupying item in a vehicle in order to control a component in the vehicle based on the characteristics of the occupying item. For example, if it were known that the occupying item is inanimate, an airbag deployment system would generally be controlled to suppress deployment of any airbags designed to protect passengers seated at the location of the inanimate object.
Particular parts of the electromagnetic spectrum interact with animal bodies in a manner differently from inanimate objects and allow the positive identification that there is an animal in the passenger compartment, or in the vicinity of the vehicle. The choice of frequencies for both active and passive observation of people is discussed in detail in Richards, A. Alien Vision, Exploring the Electromagnetic Spectrum with Imaging Technology, 2001, SPIE Press Bellingham, Wash. In particular, in the near IR range (˜850 nm), the eyes of a person at night are easily seen when illuminated. In the near UV range (˜360 nm), distinctive skin patterns are observable that can be used for identification. In the SWIR range (1100–2500 nm), the person can be easily separated from the background.
The MWIR range (2.5–7 Microns) in the passive case clearly shows people against a cooler background except when the ambient temperature is high and then everything radiates or reflects energy in that range. However, windows are not transparent to MWIR and thus energy emitted from outside the vehicle does not interfere with the energy emitted from the occupants as long as the windows are closed. This range is particularly useful at night when it is unlikely that the vehicle interior will be emitting significant amounts of energy in this range.
In the LWIR range (7–15 Microns), people are even more clearly seen against a dark background that is cooler than the person. Finally, millimeter wave radar can be used for occupant sensing as discussed elsewhere. It is important to note that an occupant sensing system can use radiation in more than one of these ranges depending on what is appropriate for the situation. For example, when the sun is bright, then visual imaging can be very effective and when the sun has set, various ranges of infrared become useful. Thus, an occupant sensing system can be a combination of these subsystems. Once again, there is not believed to be any prior art on the use of these imaging techniques for occupant sensing other than that of the current assignee.
Finally, terahertz-based devices are now being developed which show promise for vehicle interrogation and monitoring systems. Terahertz is a higher frequency than mm wave but longer than LWIR. Typically, terahertz waves are in the 1 mm to 100 Microns or less. Devices under development will permit a laser like device for generation and an array device for sensing. Life forms will respond in a particular fashion to terahertz radiation as discussed in the book Alien Vision referenced above.
8. Field Sensors
Capacitive reflective occupant sensing computes distance by detecting dielectric constant of water within the operating range of the sensor, and can distinguish a human from an inanimate object in the seat. Another capacitive sensor uses a comparison to the dielectric constant of air. A human who is 80 times more conductive than air will register as being in a seat and the distance recognized. Objects not so conductive will not register. A non-registering object is interpreted as an unoccupied seat. This unoccupied seat message could be used to prevent the airbag from deploying. Force sensing resistors located in the seats can also be used to detect the presence of an occupant. Occupant sensors deactivate airbags if a seat registers as unoccupied or if the occupant is detected too close to the airbag.
The use of a capacitive sensor in a vehicle to generate an output signal indicative of the presence of an object is described in U.S. Pat. No. 6,020,812 to Thompson et al. The presence of the object affects the reflected electric field causing a change in an output signal. The sensor is mounted on the steering wheel assembly for driver position detection or on the instrument panel near the passenger air bag module for passenger position detection. Thompson et al. also describes the use of a second capacitive sensor which generates an electric field which may or may not overlap the electric field generated by the first capacitive sensor. The positioning of the second capacitive sensor determines whether its electric field overlaps. The second capacitive sensor is used to determine whether the occupant is in a normal seating position and based on this determination, affects the decision to activate a safety restraint.
The distance measuring device such as disclosed herein can also be a capacitive proximity sensor or a capacitance sensor. One possible capacitance sensor called a capaciflector is described in U.S. Pat. No. 5,166,679. The capaciflector senses closeness or distance between the sensor and an object based on the capacitive coupling between the sensor and the object. One problem of the system using such a sensor mounted on the steering wheel, for example, is that a driver may have inadvertently placed his hand over the sensor, thus defeating the operation of the device. A second confirming transmitter/receiver is therefore desirable to be placed at some other convenient position such as on the roof or headliner of the passenger compartment as shown in several implementations described below.
Electric and magnetic phenomena can be employed in other ways to sense the presence of an occupant and in particular the fields themselves can be used to determine the dielectric properties, such as the loss tangent or dielectric constant, of occupying items in the passenger compartment. However, it is difficult if not impossible to measure these properties using static fields and thus a varying field is used which once again causes electromagnetic waves. Thus, the use of quasi-static low-frequency fields is really a limiting case of the use of waves as described in detail above. Electromagnetic waves are significantly affected at low frequencies, for example, by the dielectric properties of the material. Such capacitive or electric field sensors, for example are described in U.S. patents by Kithil et al. U.S. Pat. No. 5,366,241, U.S. Pat. No. 5,602,734, U.S. Pat. No. 5,691,693, U.S. Pat. No. 5,802,479, U.S. Pat. No. 5,844,486 and U.S. Pat. No. 6,014,602; by Jinno et al. U.S. Pat. No. 5,948,031; by Saito U.S. Pat. No. 6,325,413; by Kleinberg et al. U.S. Pat. No. 9,770,997; and SAE technical papers 982292 and 971051.
Additionally, as discussed in more detail below, the sensing of the change in the characteristics of the near field that surrounds an antenna is an effective and economical method of determining the presence of water or a water-containing life form in the vicinity of the antenna and thus a measure of occupant presence. Measurement of the near field parameters can also yield a specific pattern of an occupant and thus provide a possibility to discriminate a human being from other objects. The use of electric field and capacitance sensors and their equivalence to the occupant sensors described herein requires a special discussion.
Electric and magnetic field sensors and wave sensors are essentially the same from the point of view of sensing the presence of an occupant in a vehicle. In both cases, a time varying electric and/or magnetic field is disturbed or modified by the presence of the occupant. At high frequencies in the visual, infrared and high frequency radio wave region, the sensor is usually based on the reflection of electromagnetic energy. As the frequency drops and more of the energy passes through the occupant, the absorption of the wave energy is measured and at still lower frequencies, the occupant'"'"'s dielectric properties modify the time varying field produced in the occupied space by the plates of a capacitor. In this latter case, the sensor senses the change in charge distribution on the capacitor plates by measuring, for example, the current wave magnitude or phase in the electric circuit that drives the capacitor.
In all cases, the presence of the occupant reflects, absorbs or modifies the waves or variations in the electric or magnetic fields in the space occupied by the occupant. Thus, for the purposes of at least one of the inventions disclosed herein, capacitance and inductance, electric field and magnetic field sensors are equivalent and will be considered as wave sensors. What follows is a discussion comparing the similarities and differences between two types of wave sensors, electromagnetic beam sensors and capacitive sensors as exemplified by Kithil in U.S. Pat. No. 5,602,734.
An electromagnetic field disturbed or emitted by a passenger in the case of an electromagnetic beam sensor, for example, and the electric field sensor of Kithil, for example, are in many ways similar and equivalent for the purposes of at least one of the inventions disclosed herein. The electromagnetic beam sensor is an actual electromagnetic wave sensor by definition, which exploits for sensing a coupled pair of continuously changing electric and magnetic fields, an electromagnetic wave affected or generated by a passenger. The electric field here is not a static, potential one. It is essentially a dynamic, vortex electric field coupled with a changing magnetic field, that is, an electromagnetic wave. It cannot be produced by a steady distribution of electric charges. It is initially produced by moving electric charges in a transmitter, even if this transmitter is a passenger body for the case of a passive infrared sensor.
In the Kithil sensor, a static electric field is declared as an initial material agent coupling a passenger and a sensor (see column 5, lines 5–7): “The proximity sensors 12 each function by creating an electrostatic field between oscillator input loop 54 and detector output loop 56, which is affected by presence of a person near by, as a result of capacitive coupling, . . . ”. It is a potential, non-vortex electric field. It is not necessarily coupled with any magnetic field. It is the electric field of a capacitor. It can be produced with a steady distribution of electric charges. Thus, it is not an electromagnetic wave by definition but if the sensor is driven by a varying current, then it produces a varying electric field in the space between the plates of the capacitor which necessarily and simultaneously originates an electromagnetic wave. In the strict sense, a varying electric field between the plates of a capacitor is different from an electromagnetic wave that is detached from the device that produces it. For the purposes herein, however, both are varying electric fields and both interact with matter where the interaction is a function of the dielectric constant of the matter and therefore they can be considered in some cases as equivalents.
Kithil declares that he uses a static electric field in his capacitance sensor. Thus, from the consideration above, one can conclude that Kithil'"'"'s sensor cannot be treated as a wave sensor because there are no actual electromagnetic waves but only a static electric field of the capacitor in the sensor system. However, this is not the case. The Kithil system could not operate with a true static electric field because a steady system does not carry any information. Therefore, Kithil is forced to use an oscillator, causing an alternating current in the capacitor and a time varying electric field, or equivalent wave, in the space between the capacitor plates, and a detector to reveal an informative change of the sensor capacitance caused by the presence of an occupant (see
As described in the Kithil patents, the capacitor sensor is a parametric system where the capacitance of the sensor is controlled by influence of the passenger body. This influence is transferred by means of the varying electromagnetic field (i.e., the material agent necessarily originating the wave process) coupling the capacitor electrodes and the body. It is important to note that the same influence takes also place with a true static electric field caused by an unmovable charge distribution, that is in the absence of any wave phenomenon. This would be a situation if there were no oscillator in Kithil'"'"'s system. However, such a system is not workable and thus Kithil reverts to a dynamic system using electromagnetic waves.
Thus, although Kithil declares the coupling is due to a static electric field, such a situation is not realized in his system because an alternating electromagnetic field (“wave”) exists in the system due to the oscillator. Thus, his sensor is actually a wave sensor, that is, it is sensitive to a change of a wave field in the vehicle compartment. This change is measured by measuring the change of its capacitance. The capacitance of the sensor system is determined by the configuration of its electrodes, one of which is a human body, that is, the passenger, and the part which controls the electrode configuration and hence a sensor parameter, the capacitance.
The physics definition of “wave” from Webster'"'"'s Encyclopedic Unabridged Dictionary is: “11. Physics. A progressive disturbance propagated from point to point in a medium or space without progress or advance of the points themselves, . . . ”. In a capacitor, the time that it takes for the disturbance (a change in voltage) to propagate through space, the dielectric and to the opposite plate is generally small and neglected but it is not zero. In space, this velocity of propagation is the speed of light. As the frequency driving the capacitor increases and the distance separating the plates increases, this transmission time as a percentage of the period of oscillation can become significant. Nevertheless, an observer between the plates will see the rise and fall of the electric field much like a person standing in the water of an ocean. The presence of a dielectric body between the plates causes the waves to get bigger as more electrons flow to and from the plates of the capacitor. Thus, an occupant affects the magnitude of these waves which is sensed by the capacitor circuit. Thus, the electromagnetic field is a material agent that carries information about a passenger'"'"'s position in both Kithil'"'"'s and a beam type electromagnetic wave sensor.
The following definitions are from the Encyclopedia Britannica:
“electromagnetic field”
“A property of space caused by the motion of an electric charge. A stationary charge will produce only an electric field in the surrounding space. If the charge is moving, a magnetic field is also produced. An electric field can be produced also by a changing magnetic field. The mutual interaction of electric and magnetic fields produces an electromagnetic field, which is considered as having its own existence in space apart from the charges or currents (a stream of moving charges) with which it may be related . . . ” (Copyright 1994–1998 Encyclopedia Britannica).
“displacement current”
“ . . . in electromagnetism, a phenomenon analogous to an ordinary electric current, posited to explain magnetic fields that are produced by changing electric fields. Ordinary electric currents, called conduction currents, whether steady or varying, produce an accompanying magnetic field in the vicinity of the current. [ . . . ]
“As electric charges do not flow through the insulation from one plate of a capacitor to the other, there is no conduction current; instead, a displacement current is said to be present to account for the continuity of the magnetic effects. In fact, the calculated size of the displacement current between the plates of a capacitor being charged and discharged in an alternating-current circuit is equal to the size of the conduction current in the wires leading to and from the capacitor. Displacement currents play a central role in the propagation of electromagnetic radiation, such as light and radio waves, through empty space. A traveling, varying magnetic field is everywhere associated with a periodically changing electric field that may be conceived in terms of a displacement current. Maxwell'"'"'s insight on displacement current, therefore, made it possible to understand electromagnetic waves as being propagated through space completely detached from electric currents in conductors.” Copyright 1994–1998 Encyclopedia Britannica.
“electromagnetic radiation”
“ . . . energy that is propagated through free space or through a material medium in the form of electromagnetic waves, such as radio waves, visible light, and gamma rays. The term also refers to the emission and transmission of such radiant energy. [ . . . ]
“It has been established that time-varying electric fields can induce magnetic fields and that time-varying magnetic fields can in like manner induce electric fields. Because such electric and magnetic fields generate each other, they occur jointly, and together they propagate as electromagnetic waves. An electromagnetic wave is a transverse wave in that the electric field and the magnetic field at any point and time in the wave are perpendicular to each other as well as to the direction of propagation. [ . . . ]
“Electromagnetic radiation has properties in common with other forms of waves such as reflection, refraction, diffraction, and interference. [ . . . ]” Copyright 1994–1998 Encyclopedia Britannica
The main part of the Kithil “circuit means” is an oscillator, which is as necessary in the system as the capacitor itself to make the capacitive coupling effect be detectable. An oscillator by nature creates waves. The system can operate as a sensor only if an alternating current flows through the sensor capacitor, which, in fact, is a detector from which an informative signal is acquired. Then, this current (or, more exactly, the integral of the current over time—charge) is measured and the result is a measure of the sensor capacitance value. The latter in turn depends on the passenger presence that affects the magnitude of the waves that travel between the plates of the capacitor making the Kithil sensor a wave sensor by the definition herein.
An additional relevant definition is:
(Telecom Glossary, atis.org/tg2k/_capacitive_coupling.html)
“capacitive coupling: The transfer of energy from one circuit to another by means of the mutual capacitance between the circuits. (188) Note 1: The coupling may be deliberate or inadvertent. Note 2: Capacitive coupling favors transfer of the higher frequency components of a signal, whereas inductive coupling favors lower frequency components, and conductive coupling favors neither higher nor lower frequency components.”
Another similarity between one embodiment of the sensor of at least one of the inventions disclosed herein and the Kithil sensor is the use of a voltage-controlled oscillator (VCO).
9. Telematics
One key invention disclosed here and in the current assignee'"'"'s above-referenced patents is that once an occupancy has been categorized one of the many ways that the information can be used is to transmit all or some of it to a remote location, e.g., via a telematics link. This link can be a cell phone, Wi-Fi Wi-Mobile or other Internet connection or a satellite (LEO or geo-stationary). The recipient of the information can be a governmental authority, a company or an EMS organization.
9.1 Transmission of Occupancy Information
For example, vehicles can be provided with a standard cellular phone as well as the Global Positioning System (GPS), an automobile navigation or location system with an optional connection to a manned assistance facility, which is now available on a number of vehicle models. In the event of an accident, the phone may automatically call 911 for emergency assistance and report the exact position of the vehicle. If the vehicle also has a system as described herein for monitoring each seat location, the number and perhaps the condition of the occupants could also be reported. In that way, the emergency service (EMS) would know what equipment and how many ambulances to send to the accident site. Moreover, a communication channel can be opened between the vehicle and a monitoring facility/emergency response facility or personnel to enable directions to be provided to the occupant(s) of the vehicle to assist in any necessary first aid prior to arrival of the emergency assistance personnel.
One existing service is OnStar® provided by General Motors that automatically notifies an OnStar® operator in the event that the airbags deploy. By adding the teachings of the inventions herein, the service can also provide a description on the number and category of occupants, their condition and the output of other relevant information including a picture of a particular seat before and after the accident if desired. There is not believed to be any prior art for these added services.
9.2 Low Cost Automatic Crash Notification
9.3 Cell Phone Improvements
9.4 Children Trapped in a Vehicle
9.5 Telematics with Non-Automotive Vehicles
10. Display
10.1 Heads-Up Display (HUD)
Heads-up displays are normally projected onto the windshield. In a few cases, they can appear on a visor that is placed in front of the driver or vehicle passenger. The use of the term heads-up display or HUD herein will generally encompass both systems as well as other equivalent systems such as an OLED display.
Various manufacturers have attempted to provide information to a driver through the use of a heads-up display. In some cases, the display is limited to information that would otherwise appear on the instrument panel. In more sophisticated cases, there is an attempt to display information about the environment that would be useful to the driver. Night vision cameras can record that there is a person or an object ahead on the road that the vehicle might run into if the driver is not aware of its presence. Present day systems of this type provide a display at the bottom of the windshield of the scene sensed by the night vision camera. No attempt is made to superimpose this onto the windshield such that the driver would see it at the location that he would normally see it if the object were illuminated. This confuses the driver and in one study the driver actually performed worse than he would have in the absence of the night vision information.
The ability to find the eyes of the driver, as taught here, permits the placement of the night vision image exactly where the driver expects to see it. An enhancement is to categorize and identify the objects that should be brought to the attention of the driver and then place an icon at the proper place in the driver'"'"'s field of view. There is no known prior art of these inventions. There is of course much prior art on night vision. See for example, M. Aguilar, D. A. Fay, W. D. Ross, A. M. Waxman, D. B. Ireland, J. P. Racamato, “Real-time fusion of low-light CCD and uncooled IR imagery for color night vision”, SPIE Vol. 3364 (1998).
The University of Minnesota attempts to show the driver of a snow plow where the snow covered road edges are on a LCD display that is placed in front of the windshield. Needless to say this also can confuse the driver and a preferable approach, as disclosed herein, is to place the edge markings on the windshield as they would appear if the driver could see the road. This again requires knowledge of the location of the eyes of the driver which is not present in the Minnesota system.
Many other applications of display technology come to mind including aids to a lost driver from the route guidance system. An arrow, lane markings or even a pseudo-colored lane can be properly placed in his field of view when he should make a turn, for example or direct the driver to the closest McDonalds or gas station. For the passenger, objects of interest along with short descriptions (written or oral) can be highlighted on the HUD if the locations of the eyes of the passenger are known. In fact, all of the windows of the vehicle can become semi-transparent computer screens and be used as a virtual reality or augmented reality system guiding the driver and providing information about the environment that is generated by accurate maps, sensors and inter-vehicle communication and vehicle-to-infrastructure communication. This becomes easier with the development of organic displays that comprise a thin film that can be manufactured as part of the window or appear as part of a transparent visor. Again, there is not believed to be any prior art on these features.
10.2 Adjust HUD Based on Driver Seating Position
A simpler system that can be implemented without an occupant sensor is to base the location of the HUD display on the expected location of the eyes of the driver that can be calculated from other sensor information such as the position of the rear view mirror, seat position and weight of the occupant. Once an approximate location for the display is determined, a knob of another system can be provided to permit the driver to fine tune that location. There is not believed to be any prior art for this concept. Some relevant patents are U.S. Pat. No. 5,668,907 and WO0235276.
10.3 HUD on Rear Window
In some cases, it might be desirable to project the HUD onto the rear window or in some cases even the side windows. For the rear window, the position of the mirror and the occupant'"'"'s eyes would be useful in determining where to place the image. The position of the eyes of the driver or passenger would be useful for a HUD display on the side windows. Finally, for an entertainment system, the positions of the eyes of a passenger can allow the display of three-dimensional images onto any in-vehicle display. In this regard, see for example U.S. Pat. No. 6,291,906.
10.4 Plastic Electronics
Heads-up displays previously have been based on projection systems. With the development of plastic electronics, the possibility now exists to eliminate the projection system and to create the image directly on the windshield. Relevant patents for this technology include U.S. Pat. No. 5,661,553, U.S. Pat. No. 5,796,454, U.S. Pat. No. 5,889,566, and U.S. Pat. No. 5,933,203. A relevant paper is “Polymer Material Promises an Inexpensive and Thin Full-Color Light-Emitting Plastic Display”, Electronic Design Magazine, Jan. 9, 1996. This display material can be used in conjunction with SPD, for example, to turn the vehicle windows into a multicolored display. Also see “Bright Future for Displays”, MIT Technology Review, pp 82–3, April 2001.
11. Pattern Recognition
Many of the teachings of the inventions herein are based on pattern recognition technologies as taught in numerous textbooks and technical papers. For example, an important part of the diagnostic teachings of at least one of the inventions disclosed herein is the manner in which the diagnostic module determines a normal pattern from an abnormal pattern and the manner in which it decides what data to use from the vast amount of data available. This is accomplished using pattern recognition technologies, such as artificial neural networks, combination neural networks, support vector machines, cellular neural networks etc.
The present invention relating to occupant sensing can use sophisticated pattern recognition capabilities such as fuzzy logic systems, neural networks, neural-fuzzy systems or other pattern recognition computer-based algorithms with the occupant position measurement system disclosed in the above referenced patents and/or patent applications.
The pattern recognition techniques used can be applied to the preprocessed data acquired by various transducers or to the raw data itself depending on the application. For example, as reported in the current assignee'"'"'s patent publications, there is frequently information in the frequencies present in the data and thus a Fourier transform of the data can be inputted into the pattern recognition algorithm. In optical correlation methods, for example, a very fast identification of an object can be obtained using the frequency domain rather than the time domain. Similarly, when analyzing the output of weight sensors, the transient response is usually more accurate that the static response, as taught in the current assignee'"'"'s patents and patent applications, and this transient response can be analyzed in the frequency domain or in the time domain. An example of the use of a simple frequency analysis is presented in U.S. Pat. No. 6,005,485 to Kursawe.
Pattern recognition technology is important to the development of smart airbags that the occupant identification and position determination systems described in the above-referenced patents and patent applications and to the methods described herein for adapting those systems to a particular vehicle model and for solving particular subsystem problems discussed in this section. To complete the development of smart airbags, an anticipatory crash detecting system such as disclosed in U.S. Pat. No. 6,343,810 is also desirable. Prior to the implementation of anticipatory crash sensing, the use of a neural network smart crash sensor, which identifies the type of crash and thus its severity based on the early part of the crash acceleration signature, should be developed and thereafter implemented.
U.S. Pat. No. 5,684,701 describes a crash sensor based on neural networks. This crash sensor, as with all other crash sensors, determines whether or not the crash is of sufficient severity to require deployment of the airbag and, if so, initiates the deployment. A smart airbag crash sensor based on neural networks can also be designed to identify the crash and categorize it with regard to severity, thus permitting the airbag deployment to be matched not only to the characteristics and position of the occupant but also to the severity and timing of the crash itself as described in more detail in U.S. RE37260 (a reissue of U.S. Pat. No. 5,943,295).
The applications for this technology are numerous as described in the current assignee'"'"'s patents and patent applications listed herein. They include, among others: (i) the monitoring of the occupant for safety purposes to prevent airbag deployment induced injuries, (ii) the locating of the eyes of the occupant (driver) to permit automatic adjustment of the rear view mirror(s), (iii) the location of the seat to place the occupant'"'"'s eyes at the proper position to eliminate the parallax in a heads-up display in night vision systems, (iv) the location of the ears of the occupant for optimum adjustment of the entertainment system, (v) the identification of the occupant for security or other reasons, (vi) the determination of obstructions in the path of a closing door or window, (vii) the determination of the position of the occupant'"'"'s shoulder so that the seat belt anchorage point can be adjusted for the best protection of the occupant, (viii) the determination of the position of the rear of the occupants head so that the headrest or other system can be adjusted to minimize whiplash injuries in rear impacts, (ix) anticipatory crash sensing, (x) blind spot detection, (xi) smart headlight dimmers, (xii) sunlight and headlight glare reduction and many others. In fact, over forty products alone have been identified based on the ability to identify and monitor objects and parts thereof in the passenger compartment of an automobile or truck. In addition, there are many other applications of the apparatus and methods described herein for monitoring the environment exterior to the vehicle.
Unless specifically stated otherwise below, there is no known prior art for any of the applications listed in this section.
11.1 Neural Networks
The theory of neural networks including many examples can be found in several books on the subject including. See references 16 through 33. An example of such a pattern recognition system using neural networks using sonar is discussed in two papers by Gorman, R. P. and Sejnowski, T. J. “Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets”, Neural Networks, Vol. 1. pp. 75–89, 1988, and “Learned Classification of Sonar Targets Using a Massively Parallel Network”, IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 36, No. 7, July 1988. A more recent example using cellular neural networks is: M. Milanove, U. Büker, “Object recognition in image sequences with cellular neural networks”, Neurocomputing 31 (2000) 124–141, Elsevier. Another recent example using support vector machines, a form of neural network, is: E. Destéfanis, E. Kienzle, L. Canali, “Occupant Detection Using Support Vector Machines With a Polynomial Kernel Function”, SPIE Vol. 4192 (2000).
Japanese Patent No. 342337 (A) to Ueno describes a device for detecting the driving condition of a vehicle driver comprising a light emitter for irradiating the face of the driver and a means for picking up the image of the driver and storing it for later analysis. Means are provided for locating the eyes of the driver and then the irises of the eyes and then determining if the driver is looking to the side or sleeping. Ueno determines the state of the eyes of the occupant rather than determining the location of the eyes relative to the other parts of the vehicle passenger compartment. Such a system can be defeated if the driver is wearing glasses, particularly sunglasses, or another optical device which obstructs a clear view of his/her eyes. Pattern recognition technologies such as neural networks are not used. The method of finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,008,946 to Ando uses a complicated set of rules to isolate the eyes and mouth of a driver and uses this information to permit the driver to control the radio, for example, or other systems within the vehicle by moving his eyes and/or mouth. Ando uses visible light and illuminates only the head of the driver. He also makes no use of trainable pattern recognition systems such as neural networks, nor is there any attempt to identify the contents neither of the vehicle nor of their location relative to the vehicle passenger compartment. Rather, Ando is limited to control of vehicle devices by responding to motion of the driver'"'"'s mouth and eyes. As with Ueno, a method of finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,298,732 and U.S. Pat. No. 5,714,751 to Chen also concentrate on locating the eyes of the driver so as to position a light filter in the form of a continuously repositioning small sun visor or liquid crystal shade between a light source, such as the sun or the lights of an oncoming vehicle, and the driver'"'"'s eyes. Chen does not explain in detail how the eyes are located but does supply a calibration system whereby the driver can adjust the filter so that it is at the proper position relative to his or her eyes as long as the eyes remain at the particular position. Chen references the use of automatic equipment for determining the location of the eyes but does not describe how this equipment works. In any event, in Chen, there is no mention of illumination of the occupant, monitoring the position of the occupant, other than the eyes, determining the position of the eyes relative to the passenger compartment, or identifying any other object in the vehicle other than the driver'"'"'s eyes. Also, there is no mention of the use of a trainable pattern recognition system. A method for finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,305,012 to Faris also describes a system for reducing the glare from the headlights of an oncoming vehicle. Faris locates the eyes of the occupant by using two spaced-apart infrared cameras using passive infrared radiation from the eyes of the driver. Again, Faris is only interested in locating the driver'"'"'s eyes relative to the sun or oncoming headlights and does not identify or monitor the occupant or locate the occupant, a rear facing child seat or any other object for that matter, relative to the passenger compartment or the airbag. Also, Faris does not use trainable pattern recognition techniques such as neural networks. Faris, in fact, does not even say how the eyes of the occupant are located but refers the reader to a book entitled Robot Vision (1991) by Berthold Horn, published by MIT Press, Cambridge, Mass. A review of this book did not appear to provide the answer to this question. Also, Faris uses the passive infrared radiation rather than illuminating the occupant with ultrasonic or electromagnetic radiation as in some implementations of the instant invention. A method for finding the eyes of the occupant is described but not a method of adapting the system to a particular vehicle model.
The use of neural networks, or neural fuzzy systems, and in particular combination neural networks, as the pattern recognition technology and the methods of adapting this to a particular vehicle, such as the training methods, is important to some of the inventions herein since it makes the monitoring system robust, reliable and accurate. The resulting algorithm created by the neural network program is usually short with a limited number of lines of code written in the C or C++ computer language as opposed to typically a very large algorithm when the techniques of the above patents to Ando, Chen and Faris are implemented. As a result, the resulting systems are easy to implement at a low cost, making them practical for automotive applications. The cost of the ultrasonic transducers, for example, is expected to be less than about $1 in quantities of one million per year and the cost of the CCD and CMOS arrays, which have been prohibitively expensive until recently, currently are estimated to cost less than about $5 each in similar quantities also rendering their use practical. Similarly, the implementation of the techniques of the above-referenced patents requires expensive microprocessors while the implementation with neural networks and similar trainable pattern recognition technologies permits the use of low cost microprocessors typically costing less than about $10 in large quantities.
The present invention is best implemented using sophisticated software that develops trainable pattern recognition algorithms such as neural networks and combination neural networks. Usually, the data is preprocessed, as discussed below, using various feature extraction techniques and the results post-processed to improve system accuracy. Examples of feature extraction techniques can be found in U.S. Pat. No. 4,906,940 entitled “Process and Apparatus for the Automatic Detection and Extraction of Features in Images and Displays” to Green et al. Examples of other more advanced and efficient pattern recognition techniques can be found in U.S. Pat. No. 5,390,136 entitled “Artificial Neuron and Method of Using Same” and U.S. Pat. No. 5,517,667 entitled “Neural Network That Does Not Require Repetitive Training” to S. T. Wang. Other examples include U.S. Pat. No. 5,235,339 (Morrison et al.), U.S. Pat. No. 5,214,744 (Schweizer et al), U.S. Pat. No. 5,181,254 (Schweizer et al), and U.S. Pat. No. 4,881,270 (Knecht et al). Neural networks as used herein include all types of neural networks including modular neural networks, cellular neural networks and support vector machines and all combinations as described in detail in U.S. Pat. No. 6,445,988 and referred to therein as “combination neural networks”
11.2 Combination Neural Networks
A “combination neural network” as used herein will generally apply to any combination of two or more neural networks that are either connected together or that analyze all or a portion of the input data. A combination neural network can be used to divide up tasks in solving a particular occupant problem. For example, one neural network can be used to identify an object occupying a passenger compartment of an automobile and a second neural network can be used to determine the position of the object or its location with respect to the airbag, for example, within the passenger compartment. In another case, one neural network can be used merely to determine whether the data is similar to data upon which a main neural network has been trained or whether there is something significantly different about this data and therefore that the data should not be analyzed. Combination neural networks can sometimes be implemented as cellular neural networks.
Consider a comparative analysis performed by neural networks to that performed by the human mind. Once the human mind has identified that the object observed is a tree, the mind does not try to determine whether it is a black bear or a grizzly. Further observation on the tree might center on whether it is a pine tree, an oak tree etc. Thus, the human mind appears to operate in some manner like a hierarchy of neural networks. Similarly, neural networks for analyzing the occupancy of the vehicle can be structured such that higher order networks are used to determine, for example, whether there is an occupying item of any kind present. Another neural network could follow, knowing that there is information on the item, with attempts to categorize the item into child seats and human adults etc., i.e., determine the type of item.
Once it has decided that a child seat is present, then another neural network can be used to determine whether the child seat is rear facing or forward facing. Once the decision has been made that the child seat is facing rearward, the position of the child seat relative to the airbag, for example, can be handled by still another neural network. The overall accuracy of the system can be substantially improved by breaking the pattern recognition process down into a larger number of smaller pattern recognition problems. Combination neural networks can now be applied to solving many other pattern recognition problems in and outside of a vehicle including vehicle diagnostics, collision avoidance, anticipatory sensing etc.
In some cases, the accuracy of the pattern recognition process can be improved if the system uses data from its own recent decisions. Thus, for example, if the neural network system had determined that a forward facing adult was present, then that information can be used as input into another neural network, biasing any results toward the forward facing human compared to a rear facing child seat, for example. Similarly, for the case when an occupant is being tracked in his or her forward motion during a crash, for example, the location of the occupant at the previous calculation time step can be valuable information to determining the location of the occupant from the current data. There is a limited distance an occupant can move in 10 milliseconds, for example. In this latter example, feedback of the decision of the neural network tracking algorithm becomes important input into the same algorithm for the calculation of the position of the occupant at the next time step.
What has been described above is generally referred to as modular neural networks with and without feedback. Actually, the feedback does not have to be from the output to the input of the same neural network. The feedback from a downstream neural network could be input to an upstream neural network, for example.
The neural networks can be combined in other ways, for example in a voting situation. Sometimes the data upon which the system is trained is sufficiently complex or imprecise that different views of the data will give different results. For example, a subset of transducers may be used to train one neural network and another subset to train a second neural network etc. The decision can then be based on a voting of the parallel neural networks, sometimes known as an ensemble neural network. In the past, neural networks have usually only been used in the form of a single neural network algorithm for identifying the occupancy state of an automobile. At least one of the inventions disclosed herein is primarily advancing the state of the art and using combination neural networks wherein two or more neural networks are combined to arrive at a decision.
The applications for this technology are numerous as described in the patents and patent applications listed above. However, the main focus of some of the instant inventions is the process and resulting apparatus of adapting the system in the patents and patent applications referenced above and using combination neural networks for the detection of the presence of an occupied child seat in the rear facing position or an out-of-position occupant and the detection of an occupant in a normal seating position. The system is designed so that in the former two cases, deployment of the occupant protection apparatus (airbag) may be controlled and possibly suppressed, and in the latter case, it will be controlled and enabled.
One preferred implementation of a first generation occupant sensing system, which is adapted to various vehicle models using the teachings presented herein, is an ultrasonic occupant position sensor, as described below and in the current assignee'"'"'s above-referenced patents. This system uses a Combination Artificial Neural Network (CANN) to recognize patterns that it has been trained to identify as either airbag enable or airbag disable conditions. The pattern can be obtained from four ultrasonic transducers that cover the front passenger seating area. This pattern consists of the ultrasonic echoes bouncing off of the objects in the passenger seat area. The signal from each of the four transducers includes the electrical representation of the return echoes, which is processed by the electronics. The electronic processing can comprise amplification, logarithmic compression, rectification, and demodulation (band pass filtering), followed by discretization (sampling) and digitization of the signal. The only software processing required, before this signal can be fed into the combination artificial neural network, is normalization (i.e., mapping the input to a fixed range such as numbers between 0 and 1). Although this is a fair amount of processing, the resulting signal is still considered “raw”, because all information is treated equally.
A further important application of CANN is where optical sensors such as cameras are used to monitor the inside or outside of a vehicle in the presence of varying illumination conditions. At night, artificial illumination usually in the form of infrared radiation is frequently added to the scene. For example, when monitoring the interior of a vehicle, one or more infrared LEDs are frequently used to illuminate the occupant and a pattern recognition system is trained under such lighting conditions. In bright daylight, however, unless the infrared illumination is either very bright or in the form of a scanning laser with a narrow beam, the reflections of the sun off of an object can overwhelm the infrared. However, in daylight there is no need for artificial illumination but the patterns of reflected radiation differ significantly from the infrared case. Thus, a separate pattern recognition algorithm is frequently trained to handle this case. Furthermore, depending on the lighting conditions, more than two algorithms can be trained to handle different cases. If CANN is used for this case, the initial algorithm can determine the category of illumination that is present and direct further processing to a particular neural network that has been trained under similar conditions. Another example would be the monitoring of objects in the vicinity of the vehicle. There is no known prior art on the use on neural networks, pattern recognition algorithms or, in particular, CANN for systems that monitor either the interior or the exterior of a vehicle.
11.3 Interpretation of Other Occupant States—Inattention, Drowsiness, Sleep
Another example of an invention herein involves the monitoring of the driver'"'"'s behavior over time that can be used to warn a driver if he or she is falling asleep, or to stop the vehicle if the driver loses the capacity to control it.
A paper entitled “Intelligent System for Video Monitoring of Vehicle Cockpit” by S. Boverie et al., SAE Technical Paper Series No. 980613, Feb. 23–26, 1998, describes the installation of an optical/retina sensor in the vehicle and several uses of this sensor. Possible uses are said to include observation of the driver'"'"'s face (eyelid movement) and the driver'"'"'s attitude to allow analysis of the driver'"'"'s vigilance level and warn him/her about critical situations and observation of the front passenger seat to allow the determination of the presence of somebody or something located on the seat and to value the volumetric occupancy of the passenger for the purpose of optimizing the operating conditions for airbags.
11.4 Combining Occupant Monitoring and Car Monitoring
As discussed above and in the current assignee'"'"'s above-referenced patents and in particular in U.S. Pat. No. 6,532,408, the vehicle and the occupant can be simultaneously monitored in order to optimize the deployment of the restraint system, for example, using pattern recognition techniques such as CANN. Similarly, the position of the head of an occupant can be monitored while at the same time, the likelihood of a side impact or a rollover can be monitored by a variety of other sensor systems such as an IMU, gyroscopes, radar, laser radar, ultrasound, cameras etc. and deployment of the side curtain airbag initiated if the occupant'"'"'s head is getting too close to the side window. There are of course many other examples where the simultaneous monitoring of two environments can be combined, preferably using pattern recognition, to cause an action that would not be warranted by an analysis of only one environment. There is no known prior art, except the current assignee'"'"'s, of monitoring more than one environment to render a decision that would not have been made based on the monitoring of a single environment and particularly through the use of pattern recognition, trained pattern recognition, neural networks or combination neural networks in the automotive field.
CANN, as well as the other pattern recognition systems discussed herein, can be implemented in either software or in hardware through the use of cellular neural networks, support vector machines, ASIC, systems on a chip, or FPGAs depending on the particular application and the quantity of units to be made. In particular, for many applications where the volume is large but not huge, a rapid and relatively low cost implementation could be to use a field programmable gate array (FPGA). This technology lends itself well to the implementation of multiple connected networks such as some implementations of CANN.
11.5 Continuous Tracking
During the process of adapting an occupant monitoring system to a vehicle, the actual position of the occupant can be an important input during the training phase of a trainable pattern recognition system. Thus, for example, it might be desirable to associate a particular pattern of data from one or more cameras to the measured location of the occupant relative to the airbag. It is frequently desirable to positively measure the location of the occupant with another system while data collection is taking place. Systems for performing this measurement function include string potentiometers attached to the head or chest of the occupant, for example, inertial sensors such as an IMU attached to the occupant, laser optical systems using any part of the spectrum such as the far, mid or near infrared, visible and ultraviolet, radar, laser radar, stereo or focusing cameras, RF emitters attached to the occupant, or any other such measurement system. There is no known prior art for continuous tracking systems to be used in data collection when adapting a system for monitoring the interior or exterior of a vehicle.
11.6 Preprocessing
There are many preprocessing techniques that are and can be used to prepare the data for input into a pattern recognition or other analysis system in an interior or exterior monitoring system. The simplest systems involve subtracting one image from another to determine motion of the object of interest and to subtract out the unchanging background, removing some data that is known not to contain any useful information such as the early and late portions of an ultrasonic reflected signal, scaling, smoothing of filtering the data etc. More sophisticated preprocessing algorithms involve applying a Fourier transform, combining data from several sources using “sensor fusion” techniques, finding edges of objects and their orientation and elimination of non-edge data, finding areas having the same color or pattern and identifying such areas, image segmentation and many others. Very little preprocessing prior art exists other than that of the current assignee. The prior art is limited to the preprocessing techniques of Ando, Chen and Faris for eye detection and the sensor fusion techniques of Corrado, all discussed above.
11.7 Post Processing
In some cases, after the system has made a decision that there is an out-of-position adult occupying the passenger seat, for example, it is useful to compare that decision with another recent decision to see it they are consistent. If a previous decision made 10 milliseconds ago indicates that the adult was safely in position, and then thermal gradients or some other anomaly perhaps corrupted the data and thus the decision, then the new decision should be ignored unless subsequently confirmed. Post processing can involve a number of techniques including averaging the decisions with a 5 decision moving average, applying other more sophisticated filters, applying limits to the decision and/or to the change from the previous decision, comparing data point by data point in the input data that lead to the changed decision and correcting data points that appear to be in error etc. A goal of post-processing is to apply a reasonableness test to the decision and thus to improve the accuracy of the decision or eliminate erroneous decisions. There appears to be no known prior art for post-processing in the automotive monitoring field other than that of the current assignee.
12. Optical Correlators
Optical methods for data correlation analysis are utilized in systems for military purpose such as target tracking, missile self-guidance, aerospace reconnaissance data processing etc. Advantages of these methods are the possibility of parallel processing of the elements of images being recognized providing high speed recognition and the ability to use advanced optical processors created by means of integrated optics technologies.
Some prior art includes the following technical papers:
- 1. I. Mirkin, L. Singher “Adaptive Scale Invariant Filters”, SPIE Vol. 3159, 1997
- 2. B. Javidi “Non-linear Joint Transform Correlators”, University of Conn.
- 3. A. Awwal, H. Michel “Single Step Joint Fourier Transform Correlator”, SPIE Vol. 3073, 1997
- 4. M. O'"'"'Callaghan, D. Ward, S. Perlmuter, L. Ji, C. Walker “A highly integrated single-chip optical correlator” SPIE Vol. 3466, 1998
These papers describe the use of optical methods and tools (optical correlators and spectral analyzers) for image recognition. Paper (1) discusses the use of an optical correlation technique for transforming an initial image to a form invariant to displacements of the respective object in the view. The very recognition of the object is done using a sectoring mask that is built by training with a genetic algorithm similar to methods of neural network training. The system discussed in the paper (2) includes an optical correlator that performs projection of the spectra of the target and the sample images onto a CCD matrix which functions as a detector. The consistent spectrum image at its output is used to detect the maximum of the correlation function by the median filtration method. Papers (3), (4) discuss some designs of optical correlators.
The following should be noted in connection with the discussion on the use of optical correlators for a vehicle compartment occupant position sensing task:
- 1) Making use of optical correlators to detect and classify objects in presence of noise is efficient when the amount of possible alternatives of the object'"'"'s shape and position is comparatively small with respect to the number of elements in the scene. This is apparent from the character of demonstration samples in papers (1), (2) where there were only a few sample scenes and their respective scale factors involved.
- 2) The effectiveness of making use of optical correlation methods in systems of military purpose can be explained by a comparatively small number of classes of military objects to be recognized and a low probability of catching several objects of this kind with a single view.
- 3) In their principles of operation and capabilities, optical correlators are similar to neural associative memories.
In the task of occupant'"'"'s position sensing in a car compartment, for example, the description of the sample object is represented by a training set that can include hundreds of thousands of various images. This situation is fundamentally different from those discussed in the mentioned papers. Therefore, the direct use of the optical correlation methods appears to be difficult and expensive.
Nevertheless, making use of the correlation centering technique in order to reduce the image description'"'"'s redundancy can be a valuable technique. This task could involve a contour extraction technique that does not require excessive computational effort but may have limited capabilities as to the reduction of redundancy. The correlation centering can demand significantly more computational resources, but the spectra obtained in this way will be invariant to objects'"'"' displacements and, possibly, will maintain the classification features needed by the neural network for the purpose of recognition.
Once again, no prior art is believed to exist on the application of optical correlation techniques to the monitoring of either the interior or the exterior of the vehicle other than that of the current assignee.
13. Vehicle Diagnostics and Prognostics
Communications between a vehicle and a remote assistance facility are also important for the purpose of diagnosing problems with the vehicle and forecasting problems with the vehicle, called prognostics. Motor vehicles contain complex mechanical systems that are monitored and regulated by computer systems such as electronic control units (ECUs) and the like. Such ECUs monitor various components of the vehicle including engine performance, carburetion, speed/acceleration control, transmission, exhaust gas recirculation (EGR), braking systems, etc. However, vehicles perform such monitoring typically only for the vehicle driver and without communication of any impending results, problems and/or vehicle malfunction to a remote site for trouble-shooting, diagnosis or tracking for data mining. They also do not inform the driver about future problems.
In the past, systems that provide for remote monitoring did not provide for automated analysis and communication of problems or potential problems and recommendations to the driver. As a result, the vehicle driver or user is often left stranded, or irreparable damage occurs to the vehicle as a result of neglect or driving the vehicle without the user knowing the vehicle is malfunctioning until it is too late, such as low oil level and a malfunctioning warning light, fan belt about to fail, failing radiator hose etc.
In this regard, U.S. Pat. No. 5,400,018 (Scholl et al.) describes a system for relaying raw sensor output from an off road work site relating to the status of a vehicle to a remote location over a communications data link. The information consists of fault codes generated by sensors and electronic control modules indicating that a failure has occurred rather than forecasting a failure. The vehicle does not include a system for performing diagnosis. Rather, the raw sensor data is processed at an off-vehicle location in order to arrive at a diagnosis of the vehicle'"'"'s operating condition. Bi-directional communications are described in that a request for additional information can be sent to the vehicle from the remote location with the vehicle responding and providing the requested information but no such communication takes place with the vehicle operator and not with an operator of a vehicle traveling on a road. Also, Scholl et al. does not teach the diagnostics of the problem or potential problem on the vehicle itself nor does it teach the automatic diagnostics or any prognostics. In Scholl et al., the determination of the problem occurs at the remote site by human technicians.
U.S. Pat. No. 5,754,965 (Hagenbuch) describes an apparatus for diagnosing the state of health of a vehicle and providing the operator of the vehicle with a substantially real-time indication of the efficiency of the vehicle in performing as assigned task with respect to a predetermined goal. A processor in the vehicle monitors sensors that provide information regarding the state of health of the vehicle and the amount of work the vehicle has done. The processor records information that describes events leading up to the occurrence of an anomaly for later analysis. The sensors are also used to prompt the operator to operate the vehicle at optimum efficiency.
U.S. Pat. No. 5,955,642 (Slifkin et al.) describes a method for monitoring events in vehicles in which electrical outputs representative of events in the vehicle are produced, the characteristics of one event are compared with the characteristics of other events accumulated over a given period of time and departures or variations of a given extent from the other characteristics are determined as an indication of a significant event. A warning is sent in response to the indication, including the position of the vehicle as determined by a global positioning system on the vehicle. For example, for use with a railroad car, a microprocessor responds to outputs of an accelerometer by comparing acceleration characteristics of one impact with accumulated acceleration characteristics of other impacts and determines departures of a given magnitude from the other characteristics as a failure indication which gives rise of a warning.
Every automobile driver fears that his or her vehicle will breakdown at some unfortunate time, e.g., when he or she is traveling at night, during rush hour, or on a long trip away from home. To help alleviate that fear, certain luxury automobile manufacturers provide roadside service in the event of a breakdown. Nevertheless, unless the vehicle is equipped with OnStar® or an equivalent service, the vehicle driver must still be able to get to a telephone to call for service. It is also a fact that many people purchase a new automobile out of fear of a breakdown with their current vehicle. At least one of the inventions disclosed herein is concerned with preventing breakdowns and with minimizing maintenance costs by predicting component failure that would lead to such a breakdown before it occurs.
When a vehicle component begins to fail, the repair cost is frequently minimal if the impending failure of the component is caught early, but increases as the repair is delayed. Sometimes if a component in need of repair is not caught in a timely manner, the component, and particularly the impending failure thereof, can cause other components of the vehicle to deteriorate. One example is where the water pump fails gradually until the vehicle overheats and blows a head gasket. It is desirable, therefore, to determine that a vehicle component is about to fail as early as possible so as to minimize the probability of a breakdown and the resulting repair costs.
There are various gages on an automobile which alert the driver to various vehicle problems. For example, if the oil pressure drops below some predetermined level, the driver is warned to stop his vehicle immediately. Similarly, if the coolant temperature exceeds some predetermined value, the driver is also warned to take immediate corrective action. In these cases, the warning often comes too late as most vehicle gages alert the driver after he or she can conveniently solve the problem. Thus, what is needed is a component failure warning system that alerts the driver to the impending failure of a component sufficiently in advance of the time when the problem gets to a catastrophic point.
Some astute drivers can sense changes in the performance of their vehicle and correctly diagnose that a problem with a component is about to occur. Other drivers can sense that their vehicle is performing differently but they don'"'"'t know why or when a component will fail or how serious that failure will be, or possibly even what specific component is the cause of the difference in performance. An invention disclosed herein will, in most cases, solve this problem by predicting component failures in time to permit maintenance and thus prevent vehicle breakdowns.
Presently, automobile sensors in use are based on specific predetermined or set levels, such as the coolant temperature or oil pressure, whereby an increase above the set level or a decrease below the set level will activate the sensor, rather than being based on changes in this level over time. The rate at which coolant heats up, for example, can be an important clue that some component in the cooling system is about to fail. There are no systems currently on automobiles to monitor the numerous vehicle components over time and to compare component performance with normal performance. Nowhere in the vehicle is the vibration signal of a normally operating front wheel stored, for example, or for that matter, any normal signal from any other vehicle component. Additionally, there is no system currently existing on a vehicle to look for erratic behavior of a vehicle component and to warn the driver or the dealer that a component is misbehaving and is therefore likely to fail in the very near future.
Sometimes, when a component fails, a catastrophic accident results. In the Firestone tire case, for example, over 100 people were killed when a tire of a Ford Explorer blew out which caused the Ford Explorer to rollover. Similarly, other component failures can lead to loss of control of the vehicle and a subsequent accident. It is thus very important to accurately forecast that such an event will take place but furthermore, for those cases where the event takes place suddenly without warning, it is also important to diagnose the state of the entire vehicle, which in some cases can lead to automatic corrective action to prevent unstable vehicle motion or rollovers resulting in an accident. Finally, an accurate diagnostic system for the entire vehicle can determine much more accurately the severity of an automobile crash once it has begun by knowing where the accident is taking place on the vehicle (e.g., the part of or location on the vehicle which is being impacted by an object) and what is colliding with the vehicle based on a knowledge of the force deflection characteristics of the vehicle at that location. Therefore, in addition to a component diagnostic, the teachings of at least one of the inventions disclosed herein also provide a diagnostic system for the entire vehicle prior to and during accidents. In particular, at least one of the inventions disclosed herein is concerned with the simultaneous monitoring of multiple sensors on the vehicle so that the best possible determination of the state of the vehicle can be determined. Current crash sensors operate independently or at most one sensor may influence the threshold at which another sensor triggers a deployable restraint. In the teachings of at least one of the inventions disclosed herein, two or more sensors, frequently accelerometers, are monitored simultaneously and the combination of the outputs of these multiple sensors are combined continuously in making the crash severity analysis.
Marko et al. (U.S. Pat. No. 5,041,976) is directed to a diagnostic system using pattern recognition for electronic automotive control systems and particularly for diagnosing faults in the engine of a motor vehicle after they have occurred. For example, Marko et al. is interested in determining cylinder specific faults after the cylinder is operating abnormally. More specifically, Marko et al. is directed to detecting a fault in a vehicular electromechanical system indirectly, i.e., by means of the measurement of parameters of sensors which are affected by that system, and after that fault has already manifested itself in the system. In order to form the fault detecting system, the parameters from these sensors are input to a pattern recognition system for training thereof. Then known faults are introduced and the parameters from the sensors are inputted into the pattern recognition system with an indicia of the known fault. Thus, during subsequent operation, the pattern recognition system can determine the fault of the electromechanical system based on the parameters of the sensors, assuming that the fault was “trained” into the pattern recognition system and has already occurred.
When the electromechanical system is an engine, the parameters input into the pattern recognition system for training thereof, and used for fault detection during operation, all relate to the engine. (If the electromechanical system is other than the engine, then the parameters input into the pattern recognition system would relate to that system.) In other words, each parameter will be affected by the operation of the engine and depend thereon and changes in the operation of the engine will alter the parameter, e.g., the manifold absolute pressure is an indication of the airflow into the engine. In this case, the signal from the manifold absolute pressure sensor may be indicative of a fault in the intake of air into the engine, e.g., the engine is drawing in too much or too little air, and is thus affected by the operation of the engine. Similarly, the mass air flow is the airflow into the engine and is an alternative to the manifold absolute pressure. It is thus a parameter that is directly associated with, related to and dependent on the engine. The exhaust gas oxygen sensor is also affected by the operation of the engine, and thus directly associated therewith, since during normal operation, the mixture of the exhaust gas is neither rich or lean whereas during abnormal engine operation, the sensor will detect an abrupt change indicative of the mixture being too rich or too lean.
Thus, the system of Marko et al. is based on the measurement of sensors which affect or are affected by, i.e., are directly associated with, the operation of the electromechanical system for which faults are to be detected. However, the system of Marko et al. does not detect faults in the sensors that are conducting the measurements, e.g., a fault in the exhaust gas oxygen sensor, or faults that are only developing but have not yet manifested themselves or faults in other systems. Rather, the sensors are used to detect a fault in the system after it has occurred.
Asami et al. (U.S. Pat. No. 4,817,418) is directed to a failure diagnosis system for a vehicle including a failure display means for displaying failure information to a driver. This system only reports failures after they have occurred and does not predict them.
Tiernan et al. (U.S. Pat. No. 5,313,407) is directed, inter alia, to a system for providing an exhaust active noise control system, i.e., an electronic muffler system, including an input microphone which senses exhaust noise at a first location in an exhaust duct. An engine has exhaust manifolds feeding exhaust air to the exhaust duct. The exhaust noise sensed by the microphone is processed to obtain an output from an output speaker arranged downstream of the input microphone in the exhaust path in order to cancel the noise in the exhaust duct.
Haramaty et al. (U.S. Pat. No. 5,406,502) describes a system that monitors a machine in a factory and notifies maintenance personnel remote from the machine (not the machine operator) that maintenance should be scheduled at a time when the machine is not in use. Haramaty et al. does not expressly relate to vehicular applications.
NASA Technical Support Package MFS-26529 “Engine Monitoring Based on Normalized Vibration Spectra”, describes a technique for diagnosing engine health using a neural network based system.
A paper “Using acoustic emission signals for monitoring of production processes” by H. K. Tonshoff et al. also provides a good description of how acoustic signals can be used to predict the state of machine tools.
Based on the monitoring of vehicular components, systems and subsystems as well as to the measurement of physical and chemical characteristics relating to the vehicle or its components, systems and subsystems, it becomes possible to control and/or affect one or more vehicular system.
An important component or system which is monitored is the tires as failure of one or more of the tires can often lead to a fatal accident. Indeed, tire monitoring is extremely important since NHTSA (National Highway Traffic Safety Administration) has recently linked 148 deaths and more than 525 injuries in the United States to separations, blowouts and other tread problems in Firestone'"'"'s ATX, ATX II and Wilderness AT tires, 5 million of which were recalled in 2000. Many of the tires were standard equipment on the Ford Explorer. Ford recommends that the Firestone tires on the Explorer sport utility vehicle be inflated to 26 psi, while Firestone recommends 30 psi. It is surprising that a tire can go from a safe condition to an unsafe condition based on an under inflation of 4 psi.
Recent studies in the United States conducted by the Society of Automotive Engineers show that low tire pressure causes about 260,000 accidents annually. Another finding is that about 75% of tire failures each year are preceded by slow air leaks or inadequate tire inflation. Nissan, for example, warns that incorrect tire pressures can compromise the stability and overall handling of a vehicle and can contribute to an accident. Additionally, most non-crash auto fatalities occur while drivers are changing flat tires. Thus, tire failures are clearly a serious automobile safety problem that requires a solution.
About 16% of all car accidents are a result of incorrect tire pressure. Thus, effective pressure and wear monitoring is extremely important. Motor Trend magazine stated that one of the most overlooked maintenance areas on a car is tire pressure. An estimated 40 to 80 percent of all vehicles on the road are operating with under-inflated tires. When under-inflated, a tire tends to flex its sidewall more, increasing its rolling resistance which decreases fuel economy. The extra flex also creates excessive heat in the tire that can shorten its service life.
The Society of Automotive Engineers reports that about 87 percent of all flat tires have a history of under-inflation. About 85% of pressure-loss incidents are slow punctures caused either by small-diameter objects trapped in the tire or by larger diameter nails. The leak will be minor as long as the nail is trapped. If the nail comes out, pressure can decrease rapidly. Incidents of sudden pressure loss are potentially the most dangerous for drivers and account for about 15% of all cases.
A properly inflated tire loses approximately 1 psi per month. A defective time can lose pressure at a more rapid rate. About 35 percent of the recalled Bridgestone tires had improper repairs.
Research from a variety of sources suggests that under-inflation can be significant to both fuel economy and tire life. Industry experts have determined that tires under-inflated by a mere 10% wear out about 15% faster. An average driver with an average set of tires can drive an extra 5,000 to 7,000 miles before buying new tires by keeping the tire properly inflated.
The American Automobile Association has determined that under inflated tires cut a vehicle'"'"'s fuel economy by as much as 2% per psi below the recommended level. If each of a car'"'"'s tires is supposed to have a pressure of 30 psi and instead has a pressure of 25 psi, the car'"'"'s fuel efficiency drops by about 10%. Depending on the vehicle and miles driven, that could cost from $100 to $500 a year.
The ability to control a vehicle is strongly influenced by tire pressure. When the tire pressure is kept at proper levels, optimum vehicle braking, steering, handling and stability are accomplished. Low tire pressure can also lead to damage to both the tires and wheels.
A Michelin study revealed that the average driver doesn'"'"'t recognize a low tire until it is 14 psi too low. One of the reasons is that today'"'"'s radial tire is hard to judge visually because the sidewall flexes even when properly inflated.
Despite all the recent press about keeping tires properly inflated, new research shows that most drivers do not know the correct inflation pressure. In a recent survey, only 45 percent of respondents knew where to look to find the correct pressure, even though 78 percent thought they knew. Twenty-seven percent incorrectly believed the sidewall of the tire carries the correct information and did not know that the sidewall only indicates the maximum pressure for the tire, not the optimum pressure for the vehicle. In another survey, about 60% of the respondents reported that they check tire pressure but only before going on a long trip. The National Highway Traffic Safety Administration estimates that at least one out of every five tires is not properly inflated.
The problem is exacerbated with the new run-flat tires where a driver may not be aware that a tire is flat until it is destroyed. Run-flat tires can be operated at air pressures below normal for a limited distance and at a restricted speed (125 miles at a maximum of 55 mph). The driver must therefore be warned of changes in the condition of the tires so that she can adapt her driving to the changed conditions.
One solution to this problem is to continuously monitor the pressure and perhaps the temperature in the tire. Pressure loss can be automatically detected in two ways: by directly measuring air pressure within the tire or by indirect tire rotation methods. Various indirect methods are based on the number of revolutions each tire makes over an extended period of time through the ABS system, and others are based on monitoring the frequency changes in the sound emitted by the tire. In the direct detection case, a sensor is mounted into each wheel or tire assembly, each with its own identity. An on-board computer collects the signals, processes and displays the data and triggers a warning signal in the case of pressure loss.
Under-inflation isn'"'"'t the only cause of sudden tire failure. A variety of mechanical problems including a bad wheel bearing or a “dragging” brake can cause the tire to heat up and fail. In addition, as may have been a contributing factor in the Firestone case, substandard materials can lead to intra-tire friction and a buildup of heat. The use of re-capped truck tires is another example of heat caused failure as a result by intra-tire friction. An overheated tire can fail suddenly without warning.
As discussed in more detail below, tire monitors, such as those disclosed below, permit the driver to check the vehicle tire pressures from inside the vehicle, or even from a remote location.
The Transportation Recall Enhancement, Accountability, and Documentation Act, (H.R. 5164, or Public Law No. 106-414) known as the TREAD Act, was signed by President Clinton on Nov. 1, 2000. Section 12, TIRE PRESSURE WARNING, states that: “Not later than one year after the date of enactment of this Act, the Secretary of Transportation, acting through the National Highway Traffic Safety Administration, shall complete a rulemaking for a regulation to require a warning system in a motor vehicle to indicate to the operator when a tire is significantly under-inflated. Such requirement shall become effective not later than 2 years after the date of the completion of such rulemaking.” Thus, it is expected that a rule requiring continuous tire monitoring will take effect for the 2004 model year.
This law will dominate the first generation of such systems as automobile manufacturers move to satisfy the requirement. In subsequent years, more sophisticated systems that in addition to pressure will monitor temperature, tire footprint, wear, vibration, etc. Although the Act requires that the tire pressure be monitored, it is believed by the inventors that other parameters are as important as the tire pressure or even more important than the tire pressure as described in more detail below.
Consumers are also in favor of tire monitors. Johnson Controls'"'"' market research showed that about 80 percent of consumers believe a low tire pressure warning system is an important or extremely important vehicle feature. Thus, as with other safety products such as airbags, competition to meet customer demands will soon drive this market.
Although, as with most other safety products, the initial introductions will be in the United States, speed limits in the United States and Canada are sufficiently low that tire pressure is not as critical an issue as in Europe, for example, where the drivers often drive much faster.
The advent of microelectromechanical (MEMS) pressure sensors, especially those based on surface acoustical wave (SAW) technology, has now made the wireless and powerless monitoring of tire pressure feasible. This is the basis of the tire pressure monitors described below. According to a Frost and Sullivan report on the U.S. Micromechanical Systems (MEMS) market (June 1997): “A MEMS tire pressure sensor represents one of the most profound opportunities for MEMS in the automotive sector.”
There are many wireless tire temperature and pressure monitoring systems disclosed in the prior art patents such as for example, U.S. Pat. No. 4,295,102, U.S. Pat. No. 4,296,347, U.S. Pat. No. 4,317,372, U.S. Pat. No. 4,534,223, U.S. Pat. No. 5,289,160, U.S. Pat. No. 5,612,671, U.S. Pat. No. 5,661,651, U.S. Pat. No. 5,853,020 and U.S. Pat. No. 5,987,980 and International Publication No. WO 01/07271(A1), all of which are illustrative of the state of the art of tire monitoring.
Devices for measuring the pressure and/or temperature within a vehicle tire directly can be categorized as those containing electronic circuits and a power supply within the tire, those which contain electronic circuits and derive the power to operate these circuits either inductively, from a generator or through radio frequency radiation, and those that do not contain electronic circuits and receive their operating power only from received radio frequency radiation. For the reasons discussed above, the discussion herein is mainly concerned with the latter category. This category contains devices that operate on the principles of surface acoustic waves (SAW) and the disclosure below is concerned primarily with such SAW devices.
International Publication No. WO 01/07271 describes a tire pressure sensor that replaces the valve and valve stem in a tire.
U.S. Pat. No. 5,231,827 contains a good description and background of the tire-monitoring problem. The device disclosed, however, contains a battery and electronics and is not a SAW device. Similarly, the device described in U.S. Pat. No. 5,285,189 contains a battery as do the devices described in U.S. Pat. No. 5,335,540 and U.S. Pat. No. 5,559,484. U.S. Pat. No. 5,945,908 applies to a stationary tire monitoring system and does not use SAW devices.
One of the first significant SAW sensor patents is U.S. Pat. No. 4,534,223. This patent describes the use of SAW devices for measuring pressure and also a variety of methods for temperature compensation but does not mention wireless transmission.
U.S. Pat. No. 5,987,980 describes a tire valve assembly using a SAW pressure transducer in conjunction with a sealed cavity. This patent does disclose wireless transmission. The assembly includes a power supply and thus this also distinguishes it from a preferred system of at least one of the inventions disclosed herein. It is not a SAW system and thus the antenna for interrogating the device in this design must be within one meter, which is closer than needed for a preferred device of at least one of the inventions disclosed herein.
U.S. Pat. No. 5,698,786 relates to the sensors and is primarily concerned with the design of electronic circuits in an interrogator. U.S. Pat. No. 5,700,952 also describes circuitry for use in the interrogator to be used with SAW devices. In neither of these patents is the concept of using a SAW device in a wireless tire pressure monitoring system described. These patents also do not describe including an identification code with the temperature and/or pressure measurements in the sensors and devices.
U.S. Pat. No. 5,804,729 describes circuitry for use with an interrogator in order to obtain more precise measurements of the changes in the delay caused by the physical or chemical property being measured by the SAW device. Similar comments apply to U.S. Pat. No. 5,831,167. Other related prior art includes U.S. Pat. No. 4,895,017.
Other patents disclose the placement of an electronic device in the sidewall or opposite the tread of a tire but they do not disclose either an accelerometer or a surface acoustic wave device. In most cases, the disclosed system has a battery and electronic circuits.
One method of measuring pressure that is applicable to at least one of the inventions disclosed herein is disclosed in V. V. Varadan, Y. R. Roh and V. K. Varadan “Local/Global SAW Sensors for Turbulence”, IEEE 1989 Ultrasonics Symposium p. 591–594 makes use of a Polyvinylidene fluoride (PVDF) piezoelectric film to measure pressure. Mention is made in this article that other piezoelectric materials can also be used. Experimental results are given where the height of a column of oil is measured based on the pressure measured by the piezoelectric film used as a SAW device. In particular, the speed of the surface acoustic wave is determined by the pressure exerted by the oil on the SAW device. For the purposes of the instant invention, air pressure can also be measured in a similar manner by first placing a thin layer of a rubber material onto the surface of the SAW device which serves as a coupling agent from the air pressure to the SAW surface. In this manner, the absolute pressure of a tire, for example, can be measured without the need for a diaphragm and reference pressure greatly simplifying the pressure measurement. Other examples of the use of PVDF film as a pressure transducer can be found in U.S. Pat. No. 4,577,510 and 5,341,687, although they are not used as SAW devices.
The following U.S. patents provide relevant information to at least one of the inventions disclosed herein, and to the extent necessary: U.S. Pat. No. 4,361,026, U.S. Pat. No. 4,620,191, U.S. Pat. No. 4,703,327, U.S. Pat. No. 4,724,443, U.S. Pat. No. 4,725,841, U.S. Pat. No. 4,734,698, U.S. Pat. No. 5,691,698, U.S. Pat. No. 5,841,214, U.S. Pat. No. 6,060,815, U.S. Pat. No. 6,107,910, U.S. Pat. No. 6,114,971 and U.S. Pat. No. 6,144,332.
In recent years, SAW devices have been used as sensors in a broad variety of applications. Compared with sensors utilizing alternative technologies, SAW sensors possess outstanding properties, such as high sensitivity, high resolution, and ease of manufacturing by microelectronic technologies. However, the most attractive feature of SAW sensors is that they can be interrogated wirelessly.
U.S. Pat. No. 5,641,902, U.S. Pat. No. 5,819,779 and U.S. Pat. No. 4,103,549 illustrate a valve cap pressure sensor where a visual output is provided. Other related prior art includes U.S. Pat. No. 4,545,246.
14. Other Products, Outputs, Features
14.1 Inflator Control
Inflators now exist which will adjust the amount of gas flowing to or from the airbag to account for the size and position of the occupant and for the severity of the accident. The vehicle identification and monitoring system (VIMS) discussed in U.S. Pat. No. 5,829,782, and U.S. RE37260 (a reissue of U.S. Pat. No. 5,943,295) among others, can control such inflators based on the presence and position of vehicle occupants or of a rear facing child seat. Some of the inventions herein are concerned with the process of adapting the vehicle interior monitoring systems to a particular vehicle model and achieving a high system accuracy and reliability as discussed in greater detail below. The automatic adjustment of the deployment rate of the airbag based on occupant identification and position and on crash severity has been termed “smart airbags” and is discussed in great detail in U.S. Pat. No. 6,532,408.
14.2 Seat, Seatbelt, Steering Wheel and Pedal Adjustment and Resonators
The adjustment of an automobile seat occupied by a driver of the vehicle is now accomplished by the use of either electrical switches and motors or by mechanical levers. As a result, the driver'"'"'s seat is rarely placed at the proper driving position which is defined as the seat location which places the eyes of the driver in the so-called “eye ellipse” and permits him or her to comfortably reach the pedals and steering wheel. The “eye ellipse” is the optimum eye position relative to the windshield and rear view mirror of the vehicle.
There are a variety of reasons why the eye ellipse, which is actually an ellipsoid, is rarely achieved by the actions of the driver. One reason is the poor design of most seat adjustment systems particularly the so-called “4-way-seat”. It is known that there are three degrees of freedom of a seat bottom, namely vertical, longitudinal, and rotation about the lateral or pitch axis. The 4-way-seat provides four motions to control the seat: (1) raising or lowering the front of the seat, (2) raising or lowering the back of the seat, (3) raising or lowering the entire seat, (4) moving the seat fore and aft. Such a seat adjustment system causes confusion since there are four control motions for three degrees of freedom. As a result, vehicle occupants are easily frustrated by such events as when the control to raise the seat is exercised, the seat not only is raised but is also rotated. Occupants thus find it difficult to place the seat in the optimum location using this system and frequently give up trying leaving the seat in an improper driving position. This problem could be solved by the addition of a microprocessor and the elimination of one switch.
Many vehicles today are equipped with a lumbar support system that is almost never used by most occupants. One reason is that the lumbar support cannot be preset since the shape of the lumbar for different occupants differs significantly, for example a tall person has significantly different lumbar support requirements than a short person. Without knowledge of the size of the occupant, the lumbar support cannot be automatically adjusted.
As discussed in the current assignee'"'"'s above-referenced '"'"'320 patent, in approximately 95% of the cases where an occupant suffers a whiplash injury, the headrest is not properly located to protect him or her in a rear impact collision. Thus, many people are needlessly injured. Also, the stiffness and damping characteristics of a seat are fixed and no attempt is made in any production vehicle to adjust the stiffness and damping of the seat in relation to either the size or weight of an occupant or to the environmental conditions such as road roughness. All of these adjustments, if they are to be done automatically, require knowledge of the morphology of the seat occupant. The inventions disclosed herein provide that knowledge. Other than that of the current assignee, there is no known prior art for the automatic adjustment of the seat based on the driver'"'"'s morphology. U.S. Pat. No. 4,797,824 to Sugiyama uses visible colored light to locate the eyes of the driver with the assistance of the driver. Once the eye position is determined, the headrest and the seat are adjusted for optimum protection.
U.S. Pat. No. 4,698,571 to Mizuta et al. shows a system for automatically adjusting parts of the vehicle to a predetermined optimum setting for the driver. Buttons are provided with each button controlling a directional movement of the parts of the vehicle, e.g., the seat or rear view mirror. By depressing the button, movement of the part is thus effected. No mention is made of adjusting the steering wheel or enabling adjustment of vehicle parts automatically without manual intervention by the driver.
U.S. Pat. No. 4,811,226 to Shinohara describes an angle adjusting apparatus for adjusting parts of the vehicle in which a seat adjustment switch is provided to enable movement of the seat upon depression of the switch. No mention is made of adjusting the steering wheel or enabling adjustment of vehicle parts automatically without manual intervention by the driver.
14.3 Side Impacts
Side impact airbag systems began appearing on 1995 vehicles. The danger of deployment-induced injuries will exist for side impact airbags as they now do for frontal impact airbags. A child with his head against the airbag is such an example. The system of at least one of the inventions disclosed herein will minimize such injuries. This fact has been also realized, subsequent to its disclosure by the current assignee, by NEC and such a system now appears on Honda vehicles. There is no other known prior art.
14.4 Children and Animals Left Alone
It is a problem in vehicles that children, infants and pets are sometimes left alone, either intentionally or inadvertently, and the temperature in the vehicle rises or falls. The child, infant or pet then suffocates in view of the lack of oxygen in the vehicle or freezes. This problem can be solved by the inventions disclosed herein since the existence of the occupant can be determined as well as the temperature, and even oxygen content if desired, and preventative measures automatically taken. Similarly, children and pets die every year from suffocation after being locked in a vehicle trunk. The sensing of a life form in the trunk is discussed below.
14.5 Vehicle Theft
Another problem relates to the theft of vehicles. With an interior monitoring system, or a variety of other sensors as disclosed herein, connected with a telematics device, the vehicle owner could be notified if someone attempts to steal the vehicle while the owner is away.
14.6 Security, Intruder Protection
There have been incidents when a thief waits in a vehicle until the driver of the vehicle enters the vehicle and then forces the driver to provide the keys and exit the vehicle. Using the inventions herein, a driver can be made aware that the vehicle is occupied before he or she enters and thus he or she can leave and summon help. Motion of an occupant in the vehicle who does not enter the key into the ignition can also be sensed and the vehicle ignition, for example, can be disabled. In more sophisticated cases, the driver can be identified and operation of the vehicle enabled. This would eliminate the need even for a key.
14.7 Entertainment System Control
Once an occupant sensor is operational, the vehicle entertainment system can be improved if the number, size and location of occupants and other objects are known. However, prior to the inventions disclosed herein engineers have not thought to determine the number, size and/or location of the occupants and use such determination in combination with the entertainment system. Indeed, this information can be provided by the vehicle interior monitoring system disclosed herein to thereby improve a vehicle'"'"'s entertainment system. Once one considers monitoring the space in the passenger compartment, an alternate method of characterizing the sonic environment comes to mind which is to send and receive a test sound to see what frequencies are reflected, absorbed or excite resonances and then adjust the spectral output of the entertainment system accordingly.
As the internal monitoring system improves to where such things as the exact location of the occupants'"'"' ears and eyes can be determined, even more significant improvements to the entertainment system become possible through the use of noise canceling sound. It is even possible to beam sound directly to the ears of an occupant using hypersonic-sound if the ear location is known. This permits different occupants to enjoy different programming at the same time.
14.8 HVAC
Similarly to the entertainment system, the heating, ventilation and air conditioning system (HVAC) could be improved if the number, attributes and location of vehicle occupants were known. This can be used to provide a climate control system tailored to each occupant, for example, or the system can be turned off for certain seat locations if there are no occupants present at those locations.
U.S. Pat. No. 5,878,809 to Heinle, describes an air-conditioning system for a vehicle interior comprising a processor, seat occupation sensor devices, and solar intensity sensor devices. Based on seat occupation and solar intensity data, the processor provides the air-conditioning control of individual air-conditioning outlets and window-darkening devices which are placed near each seat in the vehicle. A residual air-conditioning function device maintains air conditioning operation after vehicle ignition switch-off, which allows specific climate conditions to be maintained after vehicle ignition switch-off for a certain period of time provided at least one seat is occupied. The advantage of this design is the allowance for occupation of certain seats in the vehicle. The drawbacks include the lack of some important sensors of vehicle interior and environment condition (such as temperature or air humidity). It is not possible to set climate conditions individually at locations of each passenger seat.
U.S. Pat. No. 6,454,178 to Fusco, et al. describes an adaptive controller for an automotive HVAC system which controls air temperature and flow at each of locations that conform to passenger seats based on individual settings manually set by passengers at their seats. If the passenger corrects manual settings for his location, this information will be remembered, allowing for climate conditions taking place at other locations and further, will be used to automatically tune the air temperature and flow at the locations allowing for climate conditions at other locations. The device does not use any sensors of the interior vehicle conditions or the exterior environment, nor any seat occupation sensing.
14.9 Obstruction Sensing
In some cases, the position of a particular part of the occupant is of interest such as his or her hand or arm and whether it is in the path of a closing window or sliding door so that the motion of the window or door needs to be stopped. Most anti-trap systems, as they are called, are based on the current flow in a motor. When the window, for example, is obstructed, the current flow in the window motor increases. Such systems are prone to errors caused by dirt or ice in the window track, for example. Prior art on window obstruction sensing is essentially limited to the Prospect Corporation anti-trap system described in U.S. Pat. No. 5,054,686 and U.S. Pat. No. 6,157,024. Anti-trap systems are discussed in detail in the current assignee'"'"'s pending U.S. patent application Ser. No. 10/152,160 filed May 21, 2002, incorporated by reference herein.
Closures for apertures such as vehicle windows, sunroofs and sliding doors, and soon swinging doors, are now commonly motor-driven. As a further convenience to an operator or passenger of a vehicle, such power windows are frequently provided with control features for the automatic closing and opening of an aperture following a simple, short command from the operator or passenger. For instance, a driver'"'"'s side window may be commanded to rise from any lowered position to a completely closed position simply by momentarily elevating a portion of a window control switch, then releasing the switch. This is sometimes referred to as an “express close” feature. This feature is commonly provided in conjunction with vehicle sunroofs. Auto manufacturers may also provide these features in conjunction with power doors, hatches or the like. Such automated aperture closing features may also be utilized in various other home or industrial settings.
Other convenience features now being offered for use on vehicles include environmental venting modes, in which vehicle windows are automatically lowered or opened a prescribed distance once a control system determines a certain temperature threshold, internal or external, has been met or exceeded. In addition, a precipitation detection system may be provided for sensing the advent of precipitation and for automatically closing a sunroof, windows or an automatic door. These specific examples pertain to vehicles, though other instances of automatic aperture adjustment are known to one skilled in the art.
In addition to providing added convenience, however, such features introduce a previously unencountered safety hazard. Body parts or inanimate objects may be present within an aperture when a command is given to automatically close the aperture. For example, an automatic window closing feature may be activated due to rain while a pet in the vehicle has its head outside a window. A further example includes a child who has placed his or her head through a window or sunroof and then he or she accidentally initiates an express close operation.
In order to avoid tragic and damaging accidents involving obstacles entrapped by a power window, some vehicles are now provided with systems which detect a condition where a window has been commanded to express close, but which has not completed the operation after a given period of time. As an example, a system may monitor the time it takes for a window to reach a closed state. If a time threshold is exceeded, the window is automatically lowered. Another system monitors the current drain attributed to the motor driving the window. If it exceeds a threshold at an inappropriate time during the closing operation, the window is again lowered.
The problem with such safety systems is that an obstacle must first be entrapped and subject to the closing force of the window or other closure for a discrete period of time before the safety mechanism lowers the window. Limbs may be bruised and fragile objects may be broken by such systems. In addition, if a mechanical failure in the window driving system occurs or if a fuse is blown, the obstacle may remain entrapped.
To address these shortcomings, a system has been proposed which monitors the environment adjacent to or within an aperture, and which may be used as an obstacle detection system, among other applications. This system may be used in conjunction with a power window to prevent activation of an express close mode, to stop such a mode once in progress, or to exit an express close mode and automatically reverse the window motion. The system comprises an emitter positioned in proximity to the aperture to emit a field of radiation adjacent the aperture. A detector is also provided which normally receives radiation reflected from one or more surfaces proximate the aperture. When an obstacle enters the radiation field, it alters the amount of reflected radiation received at the detector. This alteration, if sufficient to meet or exceed a threshold value, can be used to prevent, stop or reverse an express close mode, to activate a warning annunciator, or to initiate some other action.
The economics of producing such a system dictate that it is not feasible to produce a system custom-tailored for the environment of every vehicle in which it is installed. This is also true if the system is installed for some other non-vehicle application. Therefore, depending upon the reflecting characteristics of the environment proximate the aperture, the system detector will provide varying degrees of sensitivity. In one embodiment where the detector registers a high degree of reflectivity from the environment and is triggered by an obstacle which decreases the reflected radiation, it is desirable that the environmental reflectance be maximized. In contrast, in an embodiment where the detector senses a minimum of reflected radiation normally and is triggered by a higher degree of reflectance from an obstacle, it is desired to minimize environmentally reflected radiation. In vehicle applications, radiation reflectance is likely to vary between vehicle manufacturers, between vehicle models and model years, and between individual vehicles, due to the physical orientation of surfaces adjacent an aperture and the materials comprising such surfaces.
Additionally, reflecting surfaces adjacent the aperture tend to alter over time. For vehicles, such alteration may be across manufacturers, models, model years and individual vehicles. Thus, a monitoring system initially optimized for a particular environment may not be optimized for the useful life of the system. In the worst case, environmental changes are sufficient to cause reflected energy to register in the system as an obstacle when no obstacle is present.
U.S. Pat. No. 6,157,024 (Chapdelaine et al.) describes a monitoring system for use in detecting the presence of an obstacle in or proximate to an aperture. Materials are applied to one or more reflecting surfaces adjacent the aperture, enabling the improvement of the signal-to-noise ratio in the system without requiring tuning of the system for the particular environment. The choice of specific materials depends upon the type of radiation used for aperture monitoring and whether an obstacle is detected as an increase or decrease in reflected radiation. A calibration LED within the monitoring system enables predictable performance over a range of temperatures. The monitoring system is also provided with the capacity to adjust to variations in the background-reflected radiation, either automatically by monitoring trends in system performance or by external command. The latter case includes the use of a further element for communicating to the monitoring system directly or indirectly.
The device of Chapdelaine et al. suffers from the problem that its performance depends on the known and calibrated reflectivity of the reflecting edge surface of the aperture. These are special materials that are applied to such reflective surfaces. The reflection properties of such surfaces can change over the life of the vehicle and although some effort is made to compensate for this change, if the properties of such surfaces change, the system can fail. Thus, a system that does not depend on the reflective properties of the aperture edges would not require the application of special materials to such surfaces and would also remove this failure mode. A calibration LED is used in the Chapdelaine et al. device that is also a source of additional failure modes and thus the elimination of this device will improve the reliability of the system.
Winner et al. (U.S. Pat. No. 6,031,600) describes a method for determining the presence and distance of an object within a resolution cell. A comparison is made of the phase difference between a reflected electromagnetic wave signal (Se) and an electronically generated reference signal (Ss) whose phase relationship is independent of distance. The measured value is compared to predetermined stored values for which distances are known. To generate signal Ss, the output signal of a clock generator is conveyed through an output stage 37, an LED 38, a fiber optic cable 39, a photodiode 40 and a preamplifier 41 (see
14.10 Rear Impacts
The largest use of hospital beds in the United States is by automobile accident victims. The largest use of these hospital beds is for victims of rear impacts. The rear impact is the most expensive accident in America. The inventions herein teach a method of determining the position of the rear of the occupants head so that the headrest can be adjusted to minimize whiplash injuries in rear impacts.
Approximately 100,000 rear impacts per year result in whiplash injuries to the vehicle occupants. Most of these injuries could be prevented if the headrest were properly positioned behind the head of the occupant and if it had the correct contour to properly support the head and neck of the occupant. Whiplash injuries are the most expensive automobile accident injury even though these injuries are usually are not life-threatening and are usually classified as minor.
A good discussion of the causes of whiplash injuries in motor vehicle accidents can be found in Dellanno et al, U.S. Pat. No. 5,181,763 and U.S. Pat. No. 5,290,091, and Dellanno patents U.S. Pat. No. 5,580,124, U.S. Pat. No. 5,769,489 and U.S. Pat. No. 5,961,182, as well as many other technical papers. These patents discuss a novel automatic adjustable headrest to minimize such injuries. However, these patents assume that the headrest is properly positioned relative to the head of the occupant. A survey has shown that as many as 95% of automobiles do not have the headrest properly positioned. These patents also assume that all occupants have approximately the same contour of the neck and head. Observations of humans, on the other hand, show that significant differences occur where the back of some people'"'"'s heads is almost in the same plane as that of their neck and shoulders, while other people have substantially the opposite case, that is, their neck extends significantly forward of their head back and shoulders.
One proposed attempt at solving the problem where the headrest is not properly positioned uses a conventional crash sensor which senses the crash after impact and a headrest composed of two portions, a fixed portion and a movable portion. During a rear impact, a sensor senses the crash and pyrotechnically deploys a portion of the headrest toward the occupant. This system has the following potential problems:
1) An occupant can get a whiplash injury in fairly low velocity rear impacts; thus, either the system will not protect occupants in such accidents or there will be a large number of low velocity deployments with the resulting significant repair expense.
2) If the portion of the headrest which is propelled toward the occupant has significant mass, that is if it is other than an airbag type device, there is a risk that it will injure the occupant. This is especially true if the system has no method of sensing and adjusting for the position of the occupant.
3) If the system does not also have a system which pre-positions the headrest to the proximity of the occupant'"'"'s head, it will also not be effective when the occupant'"'"'s head has moved forward due to pre-crash braking, for example, or for different-sized occupants.
A variation of this approach uses an airbag positioned in the headrest which is activated by a rear impact crash sensor. This system suffers the same problems as the pyrotechnically deployed headrest portion. Unless the headrest is pre-positioned, there is a risk for the out-of-position occupant.
U.S. Pat. No. 5,833,312 to Lenz describes several methods for protecting an occupant from whiplash injuries using the motion of the occupant loading the seat back to stretch a canvas or deploy an airbag using fluid contained within a bag inside the seat back. In the latter case, the airbag deploys out of the top of the seat back and between the occupant'"'"'s head and the headrest. The system is based on the proposed fact that: “[F]irstly the lower part of the body reacts and is pressed, by a heavy force, against the lower part of the seat back, thereafter the upper part of the body trunk is pressed back, and finally the back of the head and the head is thrown back against the upper part of the seat back . . . ” (Col. 2 lines 47–53). Actually this does not appear to be what occurs. Instead, the vehicle, and thus the seat that is attached to it, begins to decelerate while the occupant continues at its pre-crash velocity. Those parts of the occupant that are in contact with the seat experience a force from the seat and begin to slow down while other parts, the head for example, continue moving at the pre-crash velocity. In other words, all parts of the body are “thrown back” at the same time. That is, they all have the same relative velocity relative to the seat until acted on by the seat itself. Although there will be some mechanical advantage due to the fact that the area in contact with the occupant'"'"'s back will generally be greater than the area needed to support his or her head, there generally will not be sufficient motion of the back to pump sufficient gas into the airbag to cause it to be projected in between the headrest and the head that is not rapidly moving toward the headrest. In some cases, the occupant'"'"'s head is very close to the headrest and in others it is far away. For all cases except when the occupant'"'"'s head is very far away, there is insufficient time for motion of the occupant'"'"'s back to pump air and inflate the airbag and position it between the head and the headrest. Thus, not only will the occupant impact the headrest and receive whiplash injuries, but it will also receive an additional impact from the deploying airbag.
Lenz also suggests that for those cases where additional deployment speed is required, the output from a crash sensor could be used in conjunction with a pyrotechnic element. Since he does not mention anticipatory crash sensor, which were not believed to be available at the time of the filing of the Lenz patent application, it must be assumed that a conventional crash sensor is contemplated. As discussed herein, this is either too slow or unreliable since if it is set so sensitive that it will work for low speed impacts where many whiplash injuries occur, there will be many deployments and the resulting high repair costs. For higher speed crashes, the deployment time will be too slow based on the close position of the occupant to the airbag. Thus, if a crash sensor is used, it must be an anticipatory crash sensor as disclosed herein.
14.11 Combined with SDM and Other Systems
The above applications illustrate the wide range of opportunities, which become available if the identity and location of various objects and occupants, and some of their parts, within the vehicle are known. Once the system is operational, it would be logical for the system to also incorporate the airbag electronic sensor and diagnostics system (SDM) since it needs to interface with SDM anyway and since they could share a power supply, some circuitry and computer capabilities, which will result in a significant cost saving to the auto manufacturer. For the same reasons, it would be logical for a monitoring system to include the side impact sensor and diagnostic system. As the monitoring system improves to where such things as the exact location of the occupants'"'"' ears and eyes can be determined, even more significant improvements to the entertainment system become possible through the use of noise canceling sound, and the rear view mirror can be automatically adjusted for the driver'"'"'s eye location. Another example involves the monitoring of the driver'"'"'s behavior over time, which can be used to warn a driver if he or she is falling asleep, or to stop the vehicle if the driver loses the capacity to control it.
14.13 Monitoring of Other Vehicles Such as Cargo Containers, Truck Trailers and Railroad Cars
The following is from “Occupational Health & Safety” Publication date: 2003-08-01”: “Each year, $12.5 trillion of merchandise is traded worldwide, using more than 200 million intermodal containers. Ninety percent of these shipments are between seaports. Unsecured freight represents a global security threat, both in terms of potentially lost merchandise value and the crippling of the global trading economy. Additionally, containerized freight provides a means of directly transporting harmful biological, chemical, and radioactive materials into both the United States and its allies. A Brookings Institute study estimated the Gross Domestic Product impact of a shipment, via container, of weapons of mass destruction at a major port “ . . . would cause extended shutdown in deliveries, physical destruction and lost production in contaminated areas; massive loss of life; and medical treatment of survivors. Potential cost: up to $1 trillion.”
The technology disclosed herein can be used to minimize this threat. Electronic seals now exist that provide assurance the container has not been opened once it has been sealed. This is not a complete solution as it is still possible to introduce hazardous cargo into the container prior to sealing or the container could be violated during transit and the seal reinstalled. Better protection of course comes from monitoring the contents of the container with radiation, chemical, and other sensors as described below coupled with an appropriate telematics system.
Many issues are now arising that render a low power remote asset monitoring system desirable. Some of these issues developed from the terrorist threat to the United States since Sep. 11, 2001, and the concern of anti-terrorist personnel with the relatively free and unmonitored transportation of massive amounts of material throughout the United States by trains, trucks, and ships. A system that permits monitoring of the contents of these shipping containers could substantially reduce this terrorist threat.
The FBI has recently stated that cargo crime is conservatively estimated at about $12 billion per year. It is the fastest growing crime problem in the United States. Other areas of criminal activity involve shipments imported into the United States that are used to conceal illegal goods including weapons, illegal immigrants, narcotics, and products that violate trademarks and patents. The recent concern on the potential use of cargo containers as weapons of mass destruction is also causing great pressure to improve information, inspection, tracking and monitoring technologies. Furthermore, the movement of hazardous cargo and the potential for sabotage is also causing increased concern among law enforcement agencies and resulting in increasing demands for security for such hazardous cargo shipments.
A low cost low power monitoring system of cargo containers and their contents could substantially solve these problems.
Cargo security is defined as the safe and reliable intermodal movement of goods from the shipper to the eventual destination with no loss due to theft or damage. Cargo security is concerned with the key assets that move the cargo including containers, trailers, chassis, tractors, vessels and rail cars as well as the cargo itself. Modern manufacturing methods requiring just-in-time delivery further place a premium on cargo security.
The recent increase in cargo theft and the concern for homeland security are thus placing new demands on cargo security and because of the large number of carriers and storage locations, inexpensive systems are needed to continuously monitor the status of cargo from the time that it leaves the shipper until it reaches its final destination. Technological advancements such as the global positioning system (GPS), and improved communication systems, including wireless telecommunications via satellites, and the Internet have created a situation where such an inexpensive system is now possible.
To partially respond to these concerns, projects are underway to remotely monitor the geographic location of shipping containers as well as the tractors and chassis, boats, planes and railroad cars that move these containers or cargo in general. The ability exists now for communicating limited amounts of information from shipping containers directly to central computers and the Internet using satellites and other telematics communication devices.
In some prior art systems, cargo containers are sealed with electronic cargo seals, the integrity of which can be remotely monitored. Knowledge of the container'"'"'s location as well as the seal integrity are vital pieces of information that can contribute to solving the problems mentioned above. However, this is not sufficient and the addition of various sensors and remote monitoring of these sensors is now not only possible but necessary.
Emerging technology now permits the monitoring of some safety and status information on the chassis such as tire pressures, brake system status, lights, geographical location, generator performance, and container security and this information can now be telecommunicated to a remote location. At least one of the inventions disclosed herein is concerned with these additional improvements to the remote reporting system.
Additionally, biometric information can be used to validate drivers of vehicles containing hazardous cargo to minimize terrorist activities involving these materials. This data needs to be available remotely especially if there is a sudden change in drivers. Similarly, any deviation from the authorized route can now be detected and this also needs to be remotely reported. Much of the above-mentioned prior art activity is in bits and pieces, that is, it is available on the vehicle and sometimes to the dispatching station while the vehicle is on the premises. It now needs to be available to a central monitoring location at all times. Homeland security issues arising out the components that make up the cargo transportation system including tractors, trailers, chassis, containers and railroad cars, will only be eliminated when the contents of all such elements are known, monitored, and thus the misappropriation of such assets eliminated. The shipping system or process that takes place in the United States should guarantee that all shipping containers contain only the appropriate contents and are always on the proper route from their source to their destination and on schedule. At least one of the inventions disclosed herein is concerned with achieving this 100 percent system primarily through low power remote monitoring of the assets that make up the shipping system.
The system that is described herein for monitoring shipping assets and the contents of shipping containers can also be used for a variety of other asset monitoring problems including the monitoring of unattended boats, cabins, summer homes, private airplanes, sheds, warehouses, storage facilities and other remote unattended facilities. With additional sensors, the quality of the environment, the integrity of structures, the presence of unwanted contaminants etc. can also now be monitored and reported on an exception basis through a low power, essentially maintenance-free monitoring and reporting system in accordance with the invention as described herein.
15. Definitions
Preferred embodiments of the invention are described below and unless specifically noted, it is the applicants'"'"' intention that the words and phrases in the specification and claims be given the ordinary and accustomed meaning to those of ordinary skill in the applicable art(s). If the applicants intend any other meaning, they will specifically state they are applying a special meaning to a word or phrase.
Likewise, applicants'"'"' use of the word “function” here is not intended to indicate that the applicants seek to invoke the special provisions of 35 U.S.C. §112, sixth paragraph, to define their invention. To the contrary, if applicants wish to invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, they will specifically set forth in the claims the phrases “means for” or “step for” and a function, without also reciting in that phrase any structure, material or act in support of the function. Moreover, even if applicants invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, it is the applicants'"'"' intention that their inventions not be limited to the specific structure, material or acts that are described in the preferred embodiments herein. Rather, if applicants claim their inventions by specifically invoking the provisions of 35 U.S.C. §112, sixth paragraph, it is nonetheless their intention to cover and include any and all structure, materials or acts that perform the claimed function, along with any and all known or later developed equivalent structures, materials or acts for performing the claimed function.
“Pattern recognition” as used herein will generally mean any system which processes a signal that is generated by an object (e.g., representative of a pattern of returned or received impulses, waves or other physical property specific to and/or characteristic of and/or representative of that object) or is modified by interacting with an object, in order to determine to which one of a set of classes that the object belongs. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally a series of electrical signals coming from transducers that are sensitive to acoustic (ultrasonic) or electromagnetic radiation (e.g., visible light, infrared radiation, capacitance or electric and/or magnetic fields), although other sources of information are frequently included. Pattern recognition systems generally involve the creation of a set of rules that permit the pattern to be recognized. These rules can be created by fuzzy logic systems, statistical correlations, or through sensor fusion methodologies as well as by trained pattern recognition systems such as neural networks, combination neural networks, cellular neural networks or support vector machines.
A trainable or a trained pattern recognition system as used herein generally means a pattern recognition system that is taught to recognize various patterns constituted within the signals by subjecting the system to a variety of examples. The most successful such system is the neural network used either singly or as a combination of neural networks. Thus, to generate the pattern recognition algorithm, test data is first obtained which constitutes a plurality of sets of returned waves, or wave patterns, or other information radiated or obtained from an object (or from the space in which the object will be situated in the passenger compartment, i.e., the space above the seat) and an indication of the identify of that object. A number of different objects are tested to obtain the unique patterns from each object. As such, the algorithm is generated, and stored in a computer processor, and which can later be applied to provide the identity of an object based on the wave pattern being received during use by a receiver connected to the processor and other information. For the purposes here, the identity of an object sometimes applies to not only the object itself but also to its location and/or orientation in the passenger compartment. For example, a rear facing child seat is a different object than a forward facing child seat and an out-of-position adult can be a different object than a normally seated adult. Not all pattern recognition systems are trained systems and not all trained systems are neural networks. Other pattern recognition systems are based on fuzzy logic, sensor fusion, Kalman filters, correlation as well as linear and non-linear regression. Still other pattern recognition systems are hybrids of more than one system such as neural-fuzzy systems.
The use of pattern recognition, or more particularly how it is used, is important to many embodiments of the instant invention. In the above-cited prior art, except that assigned to the current assignee, pattern recognition which is based on training, as exemplified through the use of neural networks, is not mentioned for use in monitoring the interior passenger compartment or exterior environments of the vehicle in all of the aspects of the invention disclosed herein. Thus, the methods used to adapt such systems to a vehicle are also not mentioned.
A pattern recognition algorithm will thus generally mean an algorithm applying or obtained using any type of pattern recognition system, e.g., a neural network, sensor fusion, fuzzy logic, etc.
To “identify” as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing, for example, all rear facing child seats, one containing all human occupants, or all human occupants not sitting in a rear facing child seat, or all humans in a certain height or weight range depending on the purpose of the system. In the case where a particular person is to be recognized, the set or class will contain only a single element, i.e., the person to be recognized.
To “ascertain the identity of” as used herein with reference to an object will generally mean to determine the type or nature of the object (obtain information as to what the object is), i.e., that the object is an adult, an occupied rear facing child seat, an occupied front facing child seat, an unoccupied rear facing child seat, an unoccupied front facing child seat, a child, a dog, a bag of groceries, a car, a truck, a tree, a pedestrian, a deer etc.
An “object” in a vehicle or an “occupying item” of a seat may be a living occupant such as a human or a dog, another living organism such as a plant, or an inanimate object such as a box or bag of groceries or an empty child seat.
A “rear seat” of a vehicle as used herein will generally mean any seat behind the front seat on which a driver sits. Thus, in minivans or other large vehicles where there are more than two rows of seats, each row of seats behind the driver is considered a rear seat and thus there may be more than one “rear seat” in such vehicles. The space behind the front seat includes any number of such rear seats as well as any trunk spaces or other rear areas such as are present in station wagons.
An “optical image” will generally mean any type of image obtained using electromagnetic radiation including X-ray, ultraviolet, visual, infrared, terahertz and radar radiation.
In the description herein on anticipatory sensing, the term “approaching” when used in connection with the mention of an object or vehicle approaching another will usually mean the relative motion of the object toward the vehicle having the anticipatory sensor system. Thus, in a side impact with a tree, the tree will be considered as approaching the side of the vehicle and impacting the vehicle. In other words, the coordinate system used in general will be a coordinate system residing in the target vehicle. The “target” vehicle is the vehicle that is being impacted. This convention permits a general description to cover all of the cases such as where (i) a moving vehicle impacts into the side of a stationary vehicle, (ii) where both vehicles are moving when they impact, or (iii) where a vehicle is moving sideways into a stationary vehicle, tree or wall.
“Vehicle” as used herein includes any container that is movable either under its own power or using power from another vehicle. It includes, but is not limited to, automobiles, trucks, railroad cars, ships, airplanes, trailers, shipping containers, barges, etc. The term “container” will frequently be used interchangeably with vehicle however a container will generally mean that part of a vehicle that separate from and in some cases may exist separately and away from the source of motive power. Thus, a shipping container may exist in a shipping yard and a trailer may be parked in a parking lot without the tractor. The passenger compartment or a trunk of an automobile, on the other hand, are compartments of a container that generally only exists attaches to the vehicle chassis that also has an associated engine for moving the vehicle. Note, a container can have one or a plurality of compartments.
“Out-of-position” as used for an occupant will generally mean that the occupant, either the driver or a passenger, is sufficiently close to an occupant protection apparatus (airbag) prior to deployment that he or she is likely to be more seriously injured by the deployment event itself than by the accident. It may also mean that the occupant is not positioned appropriately in order to attain the beneficial, restraining effects of the deployment of the airbag. As for the occupant being too close to the airbag, this typically occurs when the occupant'"'"'s head or chest is closer than some distance, such as about 5 inches, from the deployment door of the airbag module. The actual distance where airbag deployment should be suppressed depends on the design of the airbag module and is typically farther for the passenger airbag than for the driver airbag.
“Dynamic out-of-position” refers to the situation where a vehicle occupant, either driver or passenger, is in position at a point in time prior to an accident but becomes out-of-position, (that is, too close to the airbag module so that he or she could be injured or killed by the deployment of the airbag) prior to the deployment of the airbag due to pre-crash braking or other action which causes the vehicle to decelerate prior to a crash.
“Transducer” or “transceiver” as used herein will generally mean the combination of a transmitter and a receiver. In come cases, the same device will serve both as the transmitter and receiver while in others two separate devices adjacent to each other will be used. In some cases, a transmitter is not used and in such cases transducer will mean only a receiver. Transducers include, for example, capacitive, inductive, ultrasonic, electromagnetic (antenna, CCD, CMOS arrays), electric field, weight measuring or sensing devices. In some cases, a transducer will be a single pixel either acting alone, in a linear or an array of some other appropriate shape. In some cases, a transducer may comprise two parts such as the plates of a capacitor or the antennas of an electric field sensor. Sometimes, one antenna or plate will communicate with several other antennas or plates and thus for the purposes herein, a transducer will be broadly defined to refer, in most cases, to any one of the plates of a capacitor or antennas of a field sensor and in some other cases, a pair of such plates or antennas will comprise a transducer as determined by the context in which the term is used.
“Thermal instability” or “thermal gradients” refers to the situation where a change in air density causes a change in the path of ultrasonic waves from what the path would be in the absence of the density change. This density change ordinarily occurs due to a change in the temperature of a portion of the air through which the ultrasonic waves travel. The high speed flow of air (wind) through the passenger compartment can cause a similar effect. Thermal instability is generally caused by the sun beating down on the top of a closed vehicle (“long-term thermal instability”) of through the operation of the heater or air conditioner (“short-term thermal instability”). Of course, other heat sources can cause a similar effect and thus the term as used herein is not limited to the examples provided.
“Adaptation” as used here will generally represent the method by which a particular occupant or object sensing system is designed and arranged for a particular vehicle model. It includes such things as the process by which the number, kind and location of various transducers are determined. For pattern recognition systems, it includes the process by which the pattern recognition system is designed and then taught or made to recognize the desired patterns. In this connection, it will usually include (1) the method of training when training is used, (2) the makeup of the databases used, testing and validating the particular system, or, in the case of a neural network, the particular network architecture chosen, (3) the process by which environmental influences are incorporated into the system, and (4) any process for determining the pre-processing of the data or the post processing of the results of the pattern recognition system. The above list is illustrative and not exhaustive. Basically, adaptation includes all of the steps that are undertaken to adapt transducers and other sources of information to a particular vehicle to create the system that accurately identifies and/or determines the location of an occupant or other object in a vehicle.
For the purposes herein, a “neural network” is defined to include all such learning systems including cellular neural networks, support vector machines and other kernel-based learning systems and methods, cellular automata and all other pattern recognition methods and systems that learn. A “combination neural network” as used herein will generally apply to any combination of two or more neural networks as most broadly defined that are either connected together or that analyze all or a portion of the input data. “Neural network” can also be defined as a system wherein the data to be processed is separated into discrete values which are then operated on and combined in at least a two-stage process and where the operation performed on the data at each stage is in general different for each of the discrete values and where the operation performed is at least determined through a training process. The operation performed is typically a multiplication by a particular coefficient or weight and by different operation, therefore is meant in this example, that a different weight is used for each discrete value.
A “morphological characteristic” will generally mean any measurable property of a human such as height, weight, leg or arm length, head diameter, skin color or pattern, blood vessel pattern, voice pattern, finger prints, iris patterns, etc.
A “wave sensor” or “wave transducer” is generally any device which senses either ultrasonic or electromagnetic waves. An electromagnetic wave sensor, for example, includes devices that sense any portion of the electromagnetic spectrum from ultraviolet down to a few hertz. The most commonly used kinds of electromagnetic wave sensors include CCD and CMOS arrays for sensing visible and/or infrared waves, millimeter wave and microwave radar, and capacitive or electric and/or magnetic field monitoring sensors that rely on the dielectric constant of the object occupying a space but also rely on the time variation of the field, expressed by waves as defined below, to determine a change in state.
A “CCD” will be generally defined to include all devices, including CMOS arrays, APS arrays, focal plane arrays, QWIP arrays or equivalent, artificial retinas and particularly HDRC arrays, which are capable of converting light frequencies, including infrared, visible and ultraviolet, into electrical signals. The particular CCD array used for many of the applications disclosed herein is implemented on a single chip that is less than two centimeters on a side. Data from the CCD array is digitized and sent serially to an electronic circuit containing a microprocessor for analysis of the digitized data. In order to minimize the amount of data that needs to be stored, initial processing of the image data takes place as it is being received from the CCD array, as discussed in more detail elsewhere herein. In some cases, some image processing can take place on the chip such as described in the Kage et al. artificial retina article referenced above.
The “windshield header” as used herein generally includes the space above the front windshield including the first few inches of the roof.
A “sensor” as used herein can be a single receiver or the combination of two transducers (a transmitter and a receiver) or one transducer which can both transmit and receive.
The “headliner” is the trim which provides the interior surface to the roof of the vehicle and the A-pillar is the roof-supporting member which is on either side of the windshield and on which the front doors are hinged.
An “occupant protection apparatus” is any device, apparatus, system or component which is actuatable or deployable or includes a component which is actuatable or deployable for the purpose of attempting to reduce injury to the occupant in the event of a crash, rollover or other potential injurious event involving a vehicle
As used herein, a diagnosis of the “state of the vehicle” generally means a diagnosis of the condition of the vehicle with respect to its stability and proper running and operating condition. Thus, the state of the vehicle could be normal when the vehicle is operating properly on a highway or abnormal when, for example, the vehicle is experiencing excessive angular inclination (e.g., two wheels are off the ground and the vehicle is about to rollover), the vehicle is experiencing a crash, the vehicle is skidding, and other similar situations. A diagnosis of the state of the vehicle could also be an indication that one of the parts of the vehicle, e.g., a component, system or subsystem, is operating abnormally.
As used herein, an “occupant restraint device” generally includes any type of device which is deployable in the event of a crash involving the vehicle for the purpose of protecting an occupant from the effects of the crash and/or minimizing the potential injury to the occupant. Occupant restraint devices thus include frontal airbags, side airbags, seatbelt tensioners, knee bolsters, side curtain airbags, externally deployable airbags and the like.
As used herein, a “part” of the vehicle generally includes any component, sensor, system or subsystem of the vehicle such as the steering system, braking system, throttle system, navigation system, airbag system, seatbelt retractor, air bag inflation valve, air bag inflation controller and airbag vent valve, as well as those listed below in the definitions of “component” and “sensor”.
As used herein, a “sensor system” generally includes any of the sensors listed below in the definition of “sensor” as well as any type of component or assembly of components which detect, sense or measure something.
The term “gage” or “gauge” is used herein interchangeably with the terms “sensor” and “sensing device”.
The following references are potentially relevant to the subject matter of the claimed invention and relevant to the disclosure herein.
- 1. Jacob, R. J. K. (1995). Eye tracking in advanced interface design. In Barøeld, W., & Furness, T. (Eds.), Advanced Interface Design and Virtual Environments, pp. 258288. Oxford University Press, Oxford. http://citeseer.nj.nec.com/jacob95eye.html
- 2. Mirkin, Irina; Singher, Liviu “Adaptive scale-invariant filters”; Proceedings of SPIE Volume:
3159 Algorithms, Devices, and Systems for Optical Information Processing Editor(s): Javidi, Bahram; Psaltis, Demetri Published: October 1997
- 3. O'"'"'Callaghan, Michael J.; Ward, David J.; Perlmutter, Stephen H.; Ji, Lianhua; Walker, Christopher M.; “Highly integrated single-chip optical correlator”, Proceedings of SPIE Volume: 3466 Algorithms, Devices, and Systems for Optical Information Processing II Editor(s): Javidi, Bahram; Psaltis, Demetri, Published: October 1998
- 4. Awwal, Abdul Ahad S.; Michel, Howard E., “Single-step joint Fourier transform correlator”,
Proceedings of SPIE Volume: 3073 Optical Pattern Recognition VIII Editor(s): Casasent, David P.; Chao, Tien-Hsin, Published: February 1997
- 5. Javidi, Bahram, “Nonlinear joint transform correlators”, Real-Time Optical Information Processing, B. Javidi, and J. L. Horner, eds., Academic, N.Y., (1994)
- 6. M. Böhm, “Imagers Using Amorphous Silicon Thin Film on ASIC (TFA) Technology”, Journal of Non-Crystalline Solids, 266–269, pp. 1145–1151, 2000.
- 7. A. Eckhardt, F. Blecher, B. Schneider, J. Sterzel, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, K. Seibel, F. Mütze, M. Böhm, “Image Sensors in TFA (Thin Film on ASIC) Technology with Analog Image Pre-Processing”, H. Reichl, E. Obermeier (eds.), Proc. Micro System Technologies 98, Potsdam, Germany, pp. 165–170, 1998.
- 8. T. Lulé, B. Schneider, M. Böhm, “Design and Fabrication of a High Dynamic Range Image Sensor in TFA Technology”, invited paper for IEEE Journal of Solid-State Circuits, Special Issue on 1998 Symposium on VLSI Circuits, 1999.
- 9. M. Böhm, F. Blecher, A. Eckhardt, B. Schneider, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, R. C. Lind, L. Humm, M. Daniels, N. Wu, H. Yen, “High Dynamic Range Image Sensors in Thin Film on ASIC—Technology for Automotive Applications”, D. E. Ricken, W. Gessner (eds.), Advanced Microsystems for Automotive Applications, Springer-Verlag, Berlin, pp. 157–172, 1998.
- 10. Lake, D. W. “TFA Technology: The Coming Revolution in Photography”, pp 34–49, Advanced Imagining Magazine, Apr. 2, 2002.
- 11. M. Böhm, F. Blecher, A. Eckhardt, K. Seibel, B. Schneider, J. Sterzel, S. Benthien, H. Keller, T. Lulé, P. Rieve, M. Sommer, B. van Uffel, F. Librecht, R. C. Lind, L. Humm, U. Efron, E. Roth, “Image Sensors in Thin Film on ASIC Technology—Status & Future Trends”, Mat. Res. Soc. Symp. Proc., vol. 507, pp. 327–338, 1998.
- 12. Schwarte, R. “A New Powerful Sensory Tool in Automotive Safety Systems Based on PMD-Technology, S-TEC GmbH Proceedings of the AMAA 2000 can be ordered at your local bookseller: “Advanced Microsystems for Automotive Applications 2000” Eds. S. Krueger, W. Gessner, Springer Verlag; Berlin, Heidelberg, N.Y., ISBN 3-540-67087-4
- 13. Nayar, S. K. and Mitsunaga, T., “High Dynamic Range Imaging: Spatially Varying Pixel Exposures” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, S.C., June 2000.
- 14. Zorpette, G, “Working Knowledge: Focusing in a Flash”, Scientific American Magazine, August, 2000.
- 15. Smeraldi, F., Carmona, J. B., “Saccadic search with Garbor features applied to eye detection and real-time head tracking”, Image and Vision Computing 18 (2000) 323–329, Elsevier Science B.V.
- 16. Wang, Y., Yuan, B., “Human Eye Location Using Wavelet and Neural Network”, Proceedings of the IEEE Internal Conference on Signal Processing 2000, p 1233–1236.
- 17. Sirohey, S. A., Rosenfeld, A., “Eye detection in a face using linear and nonlinear filters”, Pattern Recognition 34 (2001) p 1367–1391, Elsevier Science Ltd.
- 18. Richards, A., Alien Vision, p. 6–9, 2001, SPIE Press, Bellingham, Wash.
- 19. Aguilar, M., Fay, D. A., Ross, W. D., Waxman, M., Ireland, D. B., and Racamato, J. P., “Rear-time fusion of low-light CCD and uncooled IR imagery for color night vision” SPIE Conference on Enhanced and Synthetic Vision 1998, Orlando, Fla. SPIE Vol. 3364 p. 124–133.
- 20. Fletcher, P., “Polymer material promises as inexpensive and thin full-color light-emitting plastic display”, Electronic Design Magazine, Jan. 8, 1996
- 21 . . . . “Organic light-emitting diodes represent the only display technology poised to meet third-generation mobile phone standards”, p. 82–85 MIT Technology Review, April 2001.
- 22. Robinson, A. “New ‘smart’ glass darkens, lightens in a flash”, p. 22F, Automotive news, Aug. 31, 1998.
- 23. “Markets for SPD technology”, refr-spd.com/markets.html
- 24. Feiner, S. “Augmented Reality: a new way of seeing”, Scientific American Magazine, April 2002.
- 25. “Sigma SD9 Digital Camera Preview and Foveon Discussion”, http://www.photo.net/sigma/sd9 (May 8, 2002)
- 26. Techniques And Application Of Neural Networks, edited by Taylor, M. and Lisboa, P., Ellis Horwood, West Sussex, England, 1993.
- 27. Naturally Intelligent Systems, by Caudill, M. and Butler, C., MIT Press, Cambridge Mass., 1990.
- 28. J. M. Zaruda, Introduction to Artificial Neural Systems, West publishing Co., N.Y., 1992.
- 29. Digital Neural Networks, by Kung, S. Y., PTR Prentice Hall, Englewood Cliffs, N.J., 1993 Eberhart, R., Simpson, P.
- 30. Dobbins, R., Computational Intelligence PC Tools, Academic Press, Inc., 1996, Orlando, Fla.
- 31. Cristianini, N. and Shawe-Taylor, J. An Introduction to Support Vector Machines and other kernel-based learning methods, Cambridge University Press, Cambridge England, 2000.
- 32. Proceedings of the 2000 6th IEEE International Workshop on Cellular Neural Networks and their Applications (CNNA 2000), IEEE, Piscataway N.J.
- 33. Sinha, N. K. and Gupta, M. M. Soft Computing & Intelligent Systems, Academic Press 2000 San Diego, Calif.
1. General Occupant Sensors
Briefly, the claimed inventions are methods and arrangements for obtaining information about an object in a vehicle as vehicle is defined above. This determination is used in various methods and arrangements for, for example, controlling occupant protection devices in the event of a vehicle crash and/or adjusting various vehicle components.
At least one of the inventions disclosed herein includes a system to sense the presence, position and/or type of an occupying item such as a child seat in a passenger compartment of a motor vehicle and more particularly, to identify and monitor the occupying items and their parts and other objects in the passenger compartment of a motor vehicle, such as an automobile or truck, by processing one or more signals received from the occupying items and their parts and other objects using one or more of a variety of pattern recognition techniques and illumination technologies. The received signal(s) may be a reflection of a transmitted signal, the reflection of some natural signal within the vehicle, or may be some signal emitted naturally by the object. Information obtained by the identification and monitoring system is then used to affect the operation of some other system in the vehicle.
At least one of the inventions disclosed herein is also a system designed to identify, locate and/or monitor occupants, including their parts, and other objects in the passenger compartment and in particular an occupied child seat in the rear facing position or an out-of-position occupant, by illuminating the contents of the vehicle with ultrasonic or electromagnetic radiation, for example, by transmitting radiation waves, as broadly defined above to include capacitors and electric or magnetic fields, from a wave generating apparatus into a space above the seat, and receiving radiation modified by passing through the space above the seat using two or more transducers properly located in the vehicle passenger compartment, in specific predetermined optimum locations.
More particularly, at least one of the inventions disclosed herein relates to a system including a plurality of transducers appropriately located and mounted and which analyze the received radiation from any object which modifies the waves or fields, or which analyze a change in the received radiation caused by the presence of the object (e.g., a change in the dielectric constant), in order to achieve an accuracy of recognition previously not possible to achieve in the past. Outputs from the receivers are analyzed by appropriate computational means employing trained pattern recognition technologies, and in particular combination neural networks, to classify, identify and/or locate the contents, and/or determine the orientation of, for example, a rear facing child seat.
In general, the information obtained by the identification and monitoring system is used to affect the operation of some other system, component or device in the vehicle and particularly the passenger and/or driver airbag systems, which may include a front airbag, a side airbag, a knee bolster, or combinations of the same. However, the information obtained can be used for controlling and/or affecting the operation of a multitude of other vehicle or in some cases, non-vehicle resident systems.
When the vehicle interior monitoring system in accordance with the invention is installed in the passenger compartment of an automotive vehicle equipped with an occupant protection apparatus, such as an inflatable airbag, and the vehicle is subjected to a crash of sufficient severity that the crash sensor has determined that the airbag is to be deployed, the system has determined (usually prior to the deployment) whether a child placed in the child seat in the rear facing position is present and if so, a signal has been sent to the control circuitry that the airbag should be controlled and most likely disabled and not deployed in the crash.
It must be understood though that instead of suppressing deployment, it is possible that the deployment may be controlled so that it might provide some meaningful protection for the occupied rear-facing child seat. The system developed using the teachings of at least one of the inventions disclosed herein also determines the position of the vehicle occupant relative to the airbag and controls and possibly disables deployment of the airbag if the occupant is positioned so that he or she is likely to be injured by the deployment of the airbag. As before, the deployment is not necessarily disabled but may be controlled to provide protection for the out-of-position occupant.
The invention also includes methods and arrangements for obtaining information about an object in a vehicle. This determination is used in various methods and arrangements for, e.g., controlling occupant protection devices in the event of a vehicle crash. The determination can also used in various methods and arrangements for, e.g., controlling heating and air-conditioning systems to optimize the comfort for any occupants, controlling an entertainment system as desired by the occupants, controlling a glare prevention device for the occupants, preventing accidents by a driver who is unable to safely drive the vehicle and enabling an effective and optimal response in the event of a crash (either oral directions to be communicated to the occupants or the dispatch of personnel to aid the occupants). Thus, one objective of the invention is to obtain information about occupancy of a vehicle and convey this information to remotely situated assistance personnel to optimize their response to a crash involving the vehicle and/or enable proper assistance to be rendered to the occupant(s) after the crash.
Accordingly, it is a principal object of the present invention to provide new and improved apparatus for obtaining information about an occupying item on a vehicle seat which apparatus may be integrated into vehicular component adjustment apparatus and methods which evaluate the occupancy of the seat and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat.
Some other objects related to general occupant sensors are:
To provide a new and improved system for identifying the presence, position and/or orientation of an object in a vehicle.
To provide a system for accurately detecting the presence of an occupied rear-facing child seat in order to prevent an occupant protection apparatus, such as an airbag, from deploying, when the airbag would impact against the rear-facing child seat if deployed.
To provide a system for accurately detecting the presence of an out-of-position occupant in order to prevent one or more deployable occupant protection apparatus such as airbags from deploying when the airbag(s) would impact against the head or chest of the occupant during its initial deployment phase causing injury or possible death to the occupant.
To provide an interior monitoring system that utilizes reflection, scattering, absorption or transmission of waves including capacitive or other field based sensors.
To determine the presence of a child in a child seat based on motion of the child.
To recognize the presence of a human on a particular seat of a motor vehicle and then to determine his or her velocity relative to the passenger compartment and to use this velocity information to affect the operation of another vehicle system.
To determine the presence of a life form anywhere in a vehicle based on motion of the life form.
To provide an occupant sensing system which detects the presence of a life form in a vehicle and under certain conditions, activates a vehicular warning system or a vehicular system to prevent injury to the life form.
To recognize the presence of a human on a particular seat of a motor vehicle and then to determine his or her position and to use this position information to affect the operation of another vehicle system.
To provide a reliable system for recognizing the presence of a rear-facing child seat on a particular seat of a motor vehicle.
To provide a reliable system for recognizing the presence of a human being on a particular seat of a motor vehicle.
To provide a reliable system for determining the position, velocity or size of an occupant in a motor vehicle.
To provide a reliable system for determining in a timely manner that an occupant is out-of-position, or will become out-of-position, and likely to be injured by a deploying airbag.
To provide an occupant vehicle interior monitoring system which has high resolution to improve system accuracy and permits the location of body parts of the occupant to be determined.
To provide a new and improved steering wheel or steering wheel assembly including a position and/or velocity sensor for use in determining the position of the occupant relative to the steering wheel or steering wheel assembly.
To provide a new and improved airbag module for mounting in a vehicle and which includes a position and/or velocity sensor for use in determining the position of the occupant to enable the airbag to be operationally controlled depending on the position of the occupant.
To provide new and improved methods and apparatus for controlling deployment of an airbag in which the distance between the occupant to be protected by the airbag and the steering wheel, in the case of the driver, or instrument panel, in the case of the front-seated passenger, are determined by a position and/or velocity sensor mounted on or in connection with the airbag module.
To provide a warning to a driver if he/she is falling asleep.
To sense that a driver is inebriated or otherwise suffering from a reduced capacity to operate a motor vehicle and to take appropriate action.
To provide a simplified system for determining the approximate location and velocity of a vehicle occupant and to use this system to control the deployment of a passive restraint. This occupant position and velocity determining system can be based on the position of the vehicle seat, the position of the seat back, the state of the seatbelt buckle switch, a seatbelt payout sensor or a combination thereof.
To provide new and improved adjustment apparatus and methods that evaluate the occupancy of the seat without the problems mentioned above.
To provide a method for accurately detecting the presence of an out-of-position occupant, and particularly one who becomes out-of-position during a high speed crash, in order to prevent one or more airbags from deploying, which airbag(s) would impact against the head or chest of the occupant during its initial deployment phase causing injury or possible death to the occupant.
1.1 Ultrasonics
Some objects mainly related to ultrasonic sensors are:
To provide adjustment apparatus and methods that evaluate the occupancy of the seat by a combination of ultrasonic sensors and additional sensors and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat.
To provide an occupant vehicle interior monitoring system this is not affected by temperature or thermal gradients. At least one of the inventions disclosed herein provides improvements to a system to sense the presence, position and/or type of an occupant in a passenger compartment of a motor vehicle in the presence of thermal gradients and more particularly, to identify and monitor occupants and their parts and other objects in the passenger compartment of a motor vehicle, such as an automobile or truck, by processing one or more signals received from the occupants and their parts and other objects using one or more of a variety of pattern recognition techniques and ultrasonic illumination technologies. The received signals are generally reflections of a transmitted signal. Information obtained by the identification and monitoring system is then used to affect the operation of some other system in the vehicle.
To enable the presence, position and type of occupying item in a passenger compartment to be detected even in the presence of thermal gradients.
To provide a method for reducing the effects of thermal gradients that occur when the sun beats down on a closed vehicle or from the operation of the heater or air conditioner, such gradients causing the ultrasonic or electromagnetic waves to be diffracted and thereby changing the received wave pattern.
To provide a reliable method using a single transducer for both sending and receiving ultrasonic or electromagnetic waves while permitting objects to be detected that are less than 4 inches from the transducer.
To provide a reliable method for dynamically determining the location of a vehicle occupant who is moving toward the airbag module due to vehicle decelerations caused by, for example, pre-crash braking and to use this information to control another vehicle system such as the airbag system.
To provide a reliable method for compensating for the effects of the change in the speed of sound due to temperature changes within the vehicle, such method based on the variation of a measurable property of the transducer such as its capacitance, inductance or natural frequency with temperature.
To provide a reliable method for determining in a timely manner, such as every 10–20 milliseconds, that an occupant is out of position, or will become out of position, and likely to be injured by a deploying airbag and to then output a signal to suppress the deployment of the airbag and to do so in sufficient time that the airbag deployment can be suppressed even in the case of a poorly designed or malfunctioning crash sensor which triggers late on a short duration crash.
To provide a method of controlling the wave pattern emitted from the transducer assembly so as to more precisely illuminate the area of interest.
To provide apparatus which permits speed of sound compensation to be achieved even when each transducer in the system operates at a different tuned frequency.
To provide apparatus which detect objects that are very close to the transducer assembly.
1.2 Optics
It is an object of at least one of the inventions disclosed herein to provide for the use of naturally occurring and artificial electromagnetic radiation in the visual, IR and ultraviolet portions of the electromagnetic spectrum. Such systems can employ, among others, cameras, CCD and CMOS arrays, Quantum Well Infrared Photodetector arrays, focal plane arrays and other imaging and radiation detecting devices and systems.
1.3 Ultrasonics and Optics
It is an object of at least one of the inventions disclosed herein to employ a combination of optical systems and ultrasonic systems to exploit the advantages of each system.
1.4 Other Transducers
It is an object of at least one of the inventions disclosed herein to also employ other transducers such as seat position, temperature, acceleration, pressure and other sensors and antennas.
2. Adaptation
It is an object of at least one of the inventions disclosed herein to provide for the adaptation of a system comprising a variety of transducers such as seatbelt payout sensors, seatbelt buckle sensors, seat position sensors, seatback position sensors, and weight sensors and which is adapted so as to constitute a highly reliable occupant presence and position system when used in combination with electromagnetic, ultrasonic or other radiation or field sensors.
3. Mounting Locations for and Quantity of Transducers
It is an object of at least one of the inventions disclosed herein to provide for one or a variety of transducer mounting locations in and on the vehicle including the headliner, A-Pillar, B-Pillar, C-Pillar, instrument panel, rear view mirror assembly, windshield, doors, windows and other appropriate locations for the particular application.
3.1 Single Camera, Dual Camera with Single Light Source
It is an object of at least one of the inventions disclosed herein to provide a single camera system that satisfies the requirements of FMVSS-208.
3.2 Location of the Transducers
It is an object of at least one of the inventions disclosed herein to provide for a driver monitoring system using an imaging transducer mounted on the rear view mirror assembly.
It is an object of at least one of the inventions disclosed herein to provide a system in which transducers are located within the passenger compartment at specific locations such that a high reliability of classification of objects and their position is obtained from the signals generated by the transducers.
3.3 Color Cameras—Multispectral Imaging
It is an object of at least one of the inventions disclosed herein to, where appropriate, use all frequencies or selected frequencies of the Radar, terahertz, infrared, visual, ultraviolet and X-ray portions of the electromagnetic spectrum.
3.4 High Dynamic Range Cameras
It is an object of at least one of the inventions disclosed herein to provide an imaging system that has sufficient dynamic range for the application. This may include the use of a high dynamic range camera (such as 120 db) or the use a lower dynamic range (such as 70 db or less) along with a method of adjusting the exposure either through use of an iris, a spatial light monitor or shutter control.
3.5 Fisheye Lens, Pan and Zoom
It is an object of at least one of the inventions disclosed herein, where appropriate, to provide for the use of a fisheye or similar very wide angle or otherwise distorting lens and to thereby achieve wide coverage and, in some cases, a pan and zoom capability.
It is a further object of at least one of the inventions disclosed herein to provide for a low-cost single element lens that can mount directly on the imaging chip.
4. 3D Cameras
It is a further object of at least one of the inventions disclosed herein to provide an interior monitoring system which provides three-dimensional information about an occupying item from a single transducer mounting location.
4.1 Stereo Vision
It is a further object of at least one of the inventions disclosed herein for some applications, where appropriate, to achieve a three-dimensional representation of objects in the passenger compartment through the use of at least two cameras. When two cameras are used, they may or may not be located near each other.
4.2 Distance by Focusing
It is a further object of at least one of the inventions disclosed herein to provide a method of measuring the distance from a sensor to an occupant or part thereof using calculations based of the degree of focus of an image.
4.3 Ranging
Further objects of at least one of the inventions disclosed herein are:
To provide a vehicle monitoring system using modulated radiation to aid in the determining of the distance from a transducer (either ultrasonic or electromagnetic) to an occupying item of a vehicle.
To provide a system of frequency domain modulation of the illumination of an object interior and/or exterior of a vehicle.
To utilize code modulation such as with a pseudo random code to permit the unambiguous monitoring of the vehicle exterior in the presence of other vehicles with the same system.
To use a chirp frequency modulation technique to aid in determining the distance to an object interior and/or exterior of a vehicle.
To use a beat frequency technique to aid in determining the distance to an object interior and/or exterior of a vehicle.
To utilize a correlation pattern modulation in a form of code division modulation for determining the distance of an object interior and/or exterior of a vehicle.
4.4 Pockel or Kerr Cell for Determining Range
It is a further object of at least one of the inventions disclosed herein to utilize a Pockel cell, Kerr cell or other spatial light monitor or equivalent to aid in determining the distance to an object in the interior or exterior of a vehicle.
4.5 Thin Film on ASIC (TFA)
It is a further object of at least one of the inventions disclosed herein to incorporate TFA technology in such a manner as to provide a three-dimensional image of the interior and/or exterior of a vehicle.
5. Glare Control
Further objects of at least one of the inventions disclosed herein are:
To determine the location of the eyes of a vehicle occupant and the direction of a light source such as the headlights of an oncoming vehicle or the sun and to cause a filter to be placed in a position to reduce the intensity of the light striking the eyes of the occupant.
To determine the location of the eyes of a vehicle occupant and the direction of a light source such as the headlights of a rear approaching vehicle or the sun and to cause a filter to be placed in a position to reduce the intensity of the light reflected from the rear view mirrors and striking the eyes of the occupant.
To provide a glare filter for a glare reduction system that uses semiconducting or metallic (organic) polymers to provide a low cost system, which may reside in the windshield, visor, mirror or special device.
To provide a glare filter based on electronic Venetian blinds, polarizers or spatial light monitors.
5.1 Windshield
It is a further object of at least one of the inventions disclosed herein to determine the location of the eyes of a vehicle occupant and the direction of a light source such as the headlights of an oncoming vehicle or the sun and to cause a filter to be placed in a position to reduce the intensity of the light striking the eyes of the occupant.
It is a further object of at least one of the inventions disclosed herein to provide a windshield where a substantial part of the area is covered by a plastic electronics film for a display and/or glare control.
5.2 Glare in Rear View Mirrors
It is an additional object of at least one of the inventions disclosed herein to determine the location of the eyes of a vehicle occupant and the direction of a light source such as the headlights of a rear approaching vehicle or the sun and to cause a filter to be placed in a rear view mirror to reduce the intensity of the light striking the eyes of the occupant.
5.3 Visor for Glare Control and HUD
It is a further object of at least one of the inventions disclosed herein to provide an occupant vehicle interior monitoring system which reduces the glare from sunlight and headlights by imposing a filter between the eyes of an occupant and the light source wherein the filter is placed in a visor.
6. Weight Measurement and Biometrics
Further objects of at least one of the inventions disclosed herein are:
To provide a system and method wherein the weight of an occupant is determined utilizing sensors located on the seat structure.
To provide apparatus and methods for measuring the weight of an occupying item on a vehicle seat which may be integrated into vehicular component adjustment apparatus and methods which evaluate the occupancy of the seat and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat.
To provide vehicular seats including a weight measuring feature and weight measuring methods for implementation in connection with vehicular seats.
To provide vehicular seats in which the weight applied by an occupying item to the seat is measured based on capacitance between conductive and/or metallic members underlying the seat cushion.
To provide adjustment apparatus and methods that evaluate the occupancy of the seat and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat and on a measurement of the occupant'"'"'s weight or a measurement of a force or pressure exerted by the occupant on the seat.
To provide seat pressure or weight measurement systems in order to improve the accuracy of another apparatus or system that utilizes measured seat pressure or weight as input, e.g., a component adjustment apparatus.
To provide a system where the morphological characteristics of an occupant are measured by sensors located within the seat.
To provide a system for recognizing the identity of a particular individual in the vehicle.
To provide an improved seat pressure or weight measurement system and thereby improve the accuracy of another apparatus or system which utilizes measured seat pressure or weight as input, e.g., a component adjustment apparatus.
To provide a system for passively and automatically adjusting the position of a vehicle component to an optimum or near optimum location based on the size of an occupant.
To provide a system for recognizing a particular occupant of a vehicle and thereafter adjusting various components of the vehicle in accordance with the preferences of the recognized occupant.
To provide a pattern recognition system to permit more accurate location of an occupant'"'"'s head and the parts thereof and to use this information to adjust a vehicle component.
To provide a method of determining whether a seat is occupied and, if not, leaving the seat at a neutral position.
6.1 Strain Gage Weight Sensors
It is a further object of at least one of the inventions disclosed herein to provide a seat pressure or weight measuring system based on the use of one or more strain gages.
6.2 Bladder Weight Sensors
It is a further object of at least one of the inventions disclosed herein to provide a seat pressure or weight measuring system based on the use of one or more fluid-filled bladders.
6.3 Dynamic Weight Measurement
It is a further object of at least one of the inventions disclosed herein:
To provide an occupant weight measuring system that utilizes the dynamic motion of the vehicle to determine the seat pressure applied by or weight of occupying items that is independent of seatbelt forces or residual stresses or other hysteretic effects in the seat pressure or weight measuring system.
To obtain a measurement of the weight of an occupying item in a seat of a vehicle while compensating for effects caused by a seatbelt, road roughness, steering maneuvers and a vehicle suspension system.
To classify an occupying item in a seat based on dynamic forces measured by a seat pressure or weight sensor associated with the seat, with an optional compensation for effects caused by the seatbelt, road roughness, etc.
To determine whether an occupying item is belted based on dynamic forces measured by a seat pressure or weight sensor associated with the seat, with an optional compensation for effects caused by the seatbelt, road roughness, etc.
To determine whether an occupying item in the seat is alive or inanimate based on dynamic forces measured by a seat pressure or weight sensor associated with the seat, with an optional compensation for effects caused by the seatbelt, road roughness, etc.
To determine the location of the occupying item on a seat based on dynamic forces measured by a seat pressure or weight sensor associated with the seat, with an optional compensation for effects caused by the seatbelt, road roughness, etc.
6.4 Combined Spatial and Weight
It is a further object of at least one of the inventions disclosed herein:
To provide an occupant sensing system that comprises both a seat pressure or weight measuring system and a special sensing system. To provide new and improved adjustment apparatus and methods that evaluate the occupancy of the seat by a combination of ultrasonic sensors and additional sensors and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based on the evaluated occupancy of the seat.
To provide new and improved adjustment apparatus and methods that reliably discriminate between a normally seated passenger and a forward facing child seat, between an abnormally seated passenger and a rear facing child seat, and whether or not the seat is empty and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based thereon.
6.5 Face Recognition (Face and Iris IR Scans)
It is a further object of at least one of the inventions disclosed herein to recognize a particular driver based on such factors as facial characteristics, physical appearance or other attributes and to use this information to control another vehicle system such as the vehicle ignition, a security system, seat adjustment, or maximum permitted vehicle velocity, among others.
Further objects of at least one of the inventions disclosed herein are:
To determine the approximate location of the eyes of a driver and to use that information to control the position of the rear view mirrors of the vehicle and/or adjust the seat.
To control a vehicle component using eye tracking techniques.
To provide systems for approximately locating the eyes of a vehicle driver to thereby permit the placement of the driver'"'"'s eyes at a particular location in the vehicle.
To provide systems for approximately locating the eyes of a vehicle driver to thereby permit the placement of the driver'"'"'s eyes at a particular location in the vehicle.
6.6 Heartbeat and Health State
Further objects of at least one of the inventions disclosed herein are:
To provide a system using radar which detects a heartbeat of life forms in a vehicle.
To provide an occupant sensor which determines the presence and health state of any occupants in a vehicle. The presence of the occupants may be determined using an animal life or heartbeat sensor.
To provide an occupant sensor that determines whether any occupants of the vehicle are breathing by analyzing the occupant'"'"'s motion. It can also be determined whether an occupant is breathing with difficulty.
To provide an occupant sensor which determines whether any occupants of the vehicle are breathing by analyzing the chemical composition of the air/gas in the vehicle, e.g., in proximity of the occupant'"'"'s mouth.
To provide an occupant sensor that determines whether any occupants of the vehicle are conscious by analyzing movement of their eyes.
To provide an occupant sensor which determines whether any occupants of the vehicle are wounded to the extent that they are bleeding by analyzing air/gas in the vehicle, e.g., directly around each occupant.
To provide an occupant sensor which determines the presence and health state of any occupants in the vehicle by analyzing sounds emanating from the passenger compartment. Such sounds can be directed to a remote, manned site for consideration in dispatching response personnel.
6.7 Other Inputs
7. Illumination
7.1 Infrared Light
It is a further object of at least one of the inventions disclosed herein provide for infrared illumination in one or more of the near IR, SWIR, MWIR or LWIR regions of the infrared portion of the electromagnetic spectrum for illuminating the environment inside or outside of a vehicle.
7.2 Structured Light
It is a further object of at least one of the inventions disclosed herein to use structured light to help determine the distance to an object from a transducer.
7.3 Color and Natural Light
It is a further object of at least one of the inventions disclosed herein to provide a system that uses colored light and natural light in monitoring the interior and/or exterior of a vehicle.
7.4 Radar
Further objects of at least one of the inventions disclosed herein are:
To provide an occupant sensor which determines whether any occupants of the vehicle are moving using radar systems, e.g., micropower impulse radar (MIR), which can also detect the heartbeats of any occupants.
To provide an occupant sensor which determines whether any occupants of the vehicle are moving using radar systems, such as micropower impulse radar (MIR), which can also detect the heartbeats of any occupants and, optionally, to send this information by telematics to one or more remote sites.
7.5 Frequency or Spectrum Considerations
8. Field Sensors and Antennas
It is a further object of at least one of the inventions disclosed herein to provide a very low cost monitoring and presence detection system that uses the property that water in the near field of an antenna changes the antenna'"'"'s loading or impedance matching or resonant properties.
9. Telematics
The occupancy determination can also be used in various methods and arrangements for, controlling heating and air-conditioning systems to optimize the comfort for any occupants, controlling an entertainment system as desired by the occupants, controlling a glare prevention device for the occupants, preventing accidents by a driver who is unable to safely drive the vehicle and enabling an effective and optimal response in the event of a crash (either oral directions to be communicated to the occupants or the dispatch of personnel to aid the occupants) as well as many others. Thus, one objective of the invention is to obtain information about occupancy of a vehicle before, during and/or after a crash and convey this information to remotely situated assistance personnel to optimize their response to a crash involving the vehicle and/or enable proper assistance to be rendered to the occupants after the crash.
It is an object of the present invention is to provide a new and improved method and system for obtaining information about occupancy of a vehicle and conveying this information to remotely situated assistance personnel after a crash involving the vehicle.
It is another object of the present invention is to provide a new and improved method and system for obtaining information about occupancy of a vehicle and conveying this information to remotely situated assistance personnel to optimize their response to a crash involving the vehicle and/or enable proper assistance to be rendered to the occupant(s) after the crash.
It is another object of the present invention to provide a new and improved method and system for determining the presence of an object on a particular seat of a motor vehicle and conveying this information over a wireless data link system or cellular phone.
It is another object of the present invention to provide a new and improved method and system for determining the presence of an object on a particular seat of a motor vehicle and using this information to affect the operation of a wireless data link system or cellular phone.
It is still another object of the present invention to provide a new and improved method and system for determining the presence of and total number of occupants of a vehicle and, in the event of an accident, transmitting that information, as well as other information such as the condition of the occupants, to a receiver site remote from the vehicle.
It is yet another object of the present invention to provide a new and improved occupant sensor which determines the presence and health state of any occupants in the vehicle by analyzing sounds emanating from the passenger compartment and directing directed such sounds to a remote, manned site for consideration in dispatching response personnel.
Still another object of the present invention is to provide a new and improved vehicle monitoring system which provides a communications channel between the vehicle (possibly through microphones distributed throughout the vehicle) and a manned assistance facility to enable communications with the occupants after a crash or whenever the occupants are in need of assistance particularly when the communication is initiated from the remote facility in response to a condition that the operator may not know exists (e.g., if the occupants are lost, then data forming maps as a navigational aid would be transmitted to the vehicle).
Further objects of at least one of the inventions disclosed herein are:
To determine the total number of occupants of a vehicle and in the event of an accident to transmit that information, as well as other information such as the condition of the occupants, to a receiver remote from the vehicle.
To determine the total number of occupants of a vehicle and in the event of an accident to transmit that information, as well as other information such as the condition of the occupants before, during and/or after a crash, to a receiver remote from the vehicle, such information may include images.
To provide an occupant sensor which determines the presence and health state of any occupants in a vehicle and, optionally, to send this information by telematics to one or more remote sites. The presence of the occupants may be determined using an animal life or heartbeat sensors.
To provide an occupant sensor which determines whether any occupants of the vehicle are breathing or breathing with difficulty by analyzing the occupant'"'"'s motion and, optionally, to send this information by telematics to one or more remote sites.
To provide an occupant sensor which determines whether any occupants of the vehicle are breathing by analyzing the chemical composition of in the vehicle and, optionally, to send this information by telematics to one or more remote sites.
To provide an occupant sensor which determines whether any occupants of the vehicle are conscious by analyzing movement of their eyes, eyelids or other parts and, optionally, to send this information by telematics to one or more remote sites.
To provide an occupant sensor which determines whether any occupants of the vehicle are wounded to the extent that they are bleeding by analyzing the gas/air in the vehicle and, optionally, to send this information by telematics to one or more remote sites.
To provide an occupant sensor which determines the presence and health state of any occupants in the vehicle by analyzing sounds emanating from the passenger compartment and, optionally, to send this information by telematics to one or more remote sites. Such sounds can be directed to a remote, manned site for consideration in dispatching response personnel.
10. Display
10.1 Heads-Up Display
It is a further object of at least one of the inventions disclosed herein to provide a heads-up display that positions the display on the windshield based of the location of the eyes of the driver so as to place objects at the appropriate location in the field of view.
10.2 Adjust HUD Based on Driver Seating Position
It is a further object of at least one of the inventions disclosed herein to provide a heads-up display that positions the display on the windshield based of the seating position of the driver so as to place objects at the appropriate location in the field of view.
10.3 HUD on Rear Window
It is a further object of at least one of the inventions disclosed herein to provide a heads-up display that positions the display on a rear window.
10.4 Plastic Electronics
It is a further object of at least one of the inventions disclosed herein to provide a heads-up display that uses plastic electronics rather than a projection system.
11. Pattern Recognition
It is a further object of at least one of the inventions disclosed herein to use pattern recognition techniques for determining the identity or location of an occupant or object in a vehicle.
It is a further object of at least one of the inventions disclosed herein to use pattern recognition techniques for analyzing three-dimensional image data of occupants of a vehicle and objects exterior to the vehicle.
11.1 Neural Networks
It is a further object of at least one of the inventions disclosed herein to use pattern recognition techniques comprising neural networks.
11.2 Combination Neural Networks
It is a further object of at least one of the inventions disclosed herein to use combination neural networks.
11.3 Interpretation of Other Occupant States—Inattention, Drowsiness, Sleep
Further objects of at least one of the inventions disclosed herein are:
To monitor the position of the head of the vehicle driver and determine whether the driver is falling asleep or otherwise impaired and likely to lose control of the vehicle and to use that information to affect another vehicle system.
To monitor the position of the eyes and/or eyelids of the vehicle driver and determine whether the driver is falling asleep or otherwise impaired and likely to lose control of the vehicle, or is unconscious after an accident, and to use that information to affect another vehicle system.
To monitor the position of the head and/or other parts of the vehicle driver and determine whether the driver is falling asleep or otherwise impaired and likely to lose control of the vehicle and to use that information to affect another vehicle system.
11.4 Combining Occupant Monitoring and Car Monitoring
It is a further object of at least one of the inventions disclosed herein to use a combination of occupant monitoring and vehicle monitoring to aid in determining if the driver is about to lose control of the vehicle.
11.5 Continuous Tracking
It is a further object of at least one of the inventions disclosed herein to provide an occupant position determination in a sufficiently short time that the position of an occupant can be tracked during a vehicle crash.
It is a further object of at least one of the inventions disclosed herein that the pattern recognition system is trained on the position of the occupant relative to the airbag rather than what zone the occupant occupies.
11.6 Preprocessing
Further objects of at least one of the inventions disclosed herein are:
To determine the presence of a child in a child seat based on motion of the child.
To determine the presence of a life form anywhere in a vehicle based on motion of the life form.
To provide a system using electromagnetics or ultrasonics to detect motion of objects in a vehicle and enable the use of the detection of the motion for control of vehicular components and systems.
11.7 Post-Processing
It is another object of at least one of the inventions disclosed herein to apply a filter to the output of the pattern recognition system that is based on previous decisions as a test of reasonableness.
13. Diagnostics and Prognostics
Principal objects and advantages of at least one of the inventions disclosed herein or other inventions disclosed herein are thus:
1. To prevent vehicle breakdowns.
2. To alert the driver of the vehicle that a component of the vehicle is functioning differently than normal and might be in danger of failing.
3. To alert the dealer, or repair facility, that a component of the vehicle is functioning differently than normal and is in danger of failing.
4. To provide an early warning of a potential component failure and to thereby minimize the cost of repairing or replacing the component.
5. To provide a device which will capture available information from signals emanating from vehicle components for a variety of uses such as current and future vehicle diagnostic purposes.
6. To provide a device that uses information from existing sensors for new purposes thereby increasing the value of existing sensors and, in some cases, eliminating the need for sensors that provide redundant information.
7. To provide a device which is trained to recognize deterioration in the performance of a vehicle component, or of the entire vehicle, based on information in signals emanating from the component or from vehicle angular and linear accelerations.
8. To provide a device which analyzes vibrations from various vehicle components that are transmitted through the vehicle structure and sensed by existing vibration sensors such as vehicular crash sensors used with airbag systems or by special vibration sensors, accelerometers, or gyroscopes.
9. To provide a device which provides information to the vehicle manufacturer of the events leading to a component failure.
10. To apply pattern recognition techniques based on training to diagnose potential vehicle component failures.
11. To apply component diagnostic techniques in combination with intelligent or smart highways wherein vehicles may be automatically guided without manual control in order to permit the orderly exiting of the vehicle from a restricted roadway prior to a breakdown of the vehicle.
12. To apply trained pattern recognition techniques using multiple sensors to provide an early prediction of the existence and severity of an accident.
13. To utilize pattern recognition techniques and the output from multiple sensors to determine at an early stage that a vehicle rollover might occur and to take corrective action through control of the vehicle acceleration, brakes and/or steering to prevent the rollover or if it is not preventable, to deploy side head protection airbags to attempt to reduce injuries.
14. To use the output from multiple sensors to determine that the vehicle is skidding or sliding and to send messages to the various vehicle control systems to activate the throttle, brakes and/or steering to correct for the vehicle sliding or skidding motion.
15. To provide a new and improved method and system for diagnosing components in a vehicle and the operating status of the vehicle and alerting the vehicle'"'"'s dealer, or another repair facility, via a telematics link that a component of the vehicle is functioning abnormally and may be in danger of failing.
16. To provide a new and improved method and apparatus for obtaining information about a vehicle system and components in the vehicle in conjunction with failure of the component or the vehicle and sending this information to the vehicle manufacturer.
17. To provide a new and improved method and system for diagnosing components in a vehicle by monitoring the patterns of signals emitted from the vehicle components and, through the use of pattern recognition technology, forecasting component failures before they occur. Vehicle component behavior is thus monitored over time in contrast to systems that wait until a serious condition occurs. The forecast of component failure can be transmitted to a remote location via a telematics link.
18. To provide a new and improved on-board vehicle diagnostic module utilizing pattern recognition technologies which are trained to differentiate normal from abnormal component behavior. The diagnosis of component behavior can be transmitted to a remote location via a telematics link.
19. To provide a diagnostic module that determines whether a component is operating normally or abnormally based on a time series of data from a single sensor or from multiple sensors that contain a pattern indicative of the operating status of the component. The diagnosis of component operation can be transmitted to a remote location via a telematics link.
20. To provide a diagnostic module that determines whether a component is operating normally or abnormally based on data from one or more sensors that are not directly associated with the component, i.e., do not depend on the operation of the component. The diagnosis of component operation can be transmitted to a remote location via a telematics link.
21. To simultaneously monitor several sensors, primarily accelerometers, gyroscopes and strain gages, to determine the state of the vehicle and optionally its occupants and to determine that a vehicle is out of control and possibly headed for an accident, for example. If so, then a signal can be sent to a part of the vehicle control system to attempt to re-establish stability. If this is unsuccessful, then the same system of sensors can monitor the early stages of a crash to make an assessment of the severity of the crash and what occupant protection systems should be deployed and how such occupant protection systems should be deployed.
22. To provide new and improved sensors for a vehicle which wirelessly transmits information about a state measured or detected by the sensor.
23. To incorporate surface acoustic wave technology into sensors on a vehicle with the data obtained by the sensors being transmittable via a telematics link to a remote location.
24. To provide new and improved sensors for measuring the pressure, temperature and/or acceleration of tires with the data obtained by the sensors being transmittable via a telematics link to a remote location.
25. To provide new and improved weight or load measuring sensors, switches, temperature sensors, acceleration sensors, angular position sensors, angular rate sensors, angular acceleration sensors, proximity sensors, rollover sensors, occupant presence and position sensors, strain sensors and humidity sensors which utilize wireless data transmission, wireless power transmission, and/or surface acoustic wave technology with the data obtained by the sensors being transmittable via a telematics link to a remote location.
26. To provide new and improved sensors for detecting the presence of fluids or gases which utilize wireless data transmission, wireless power transmission, and/or surface acoustic wave technology with the data obtained by the sensors being transmittable via a telematics link to a remote location.
27. To provide new and improved sensors for detecting the condition or friction of a road surface which utilize wireless data transmission, wireless power transmission, and/or surface acoustic wave technology with the data obtained by the sensors being transmittable via a telematics link to a remote location.
28. To provide new and improved sensors for detecting chemicals which utilize wireless data transmission, wireless power transmission, and/or surface acoustic wave technology with the data obtained by the sensors being transmittable via a telematics link to a remote location.
29. To utilize any of the foregoing sensors for a vehicular component control system in which a component, system or subsystem in the vehicle is controlled based on the information provided by the sensor. Additionally, the information provided by the sensor can be transmitted via a telematics link to one or more remote facilities for further analysis.
30. To provide new and improved sensors which obtain and provide information about the vehicle, about individual components, systems, vehicle occupants, subsystems, or about the roadway, ambient atmosphere, travel conditions and external objects with the data obtained by the sensors being transmittable via a telematics link to a remote location
14. Other Products, Outputs, Features
It is an object of the present invention to provide new and improved arrangements and methods for adjusting or controlling a component in a vehicle. Control of a component does not require an adjustment of the component if the operation of the component is appropriate for the situation.
It is another object of the present invention to provide new and improved methods and apparatus for adjusting a component in a vehicle based on occupancy of the vehicle. For example, an airbag system may be controlled based on the location of a seat and the occupant of the seat to be protected by the deployment of the airbag.
Further objects of at least one of the inventions disclosed herein related to additional capabilities are:
To recognize the presence of an object on a particular seat of a motor vehicle and to use this information to affect the operation of another vehicle system such as the entertainment system, airbag system, heating and air conditioning system, pedal adjustment system, mirror adjustment system, wireless data link system and cellular phone, among others.
To recognize the presence of an occupant on a particular seat of a motor vehicle and then to determine his/her position and to use this position information to affect the operation of another vehicle system.
To determine the approximate location of the eyes of a driver and to use that information to control the position of the rear view mirrors of the vehicle.
To recognize a particular driver based on such factors as physical appearance or other attributes and to use this information to control another vehicle system such as a security system, seat adjustment, or maximum permitted vehicle velocity, among others.
To recognize the presence of a human on a particular seat of a motor vehicle and then to determine his/her velocity relative to the passenger compartment and to use this velocity information to affect the operation of another vehicle system.
To provide a system using electric fields, electromagnetics or ultrasonics to detect motion of objects in a vehicle and enable the use of the detection of the motion for control of vehicular components and systems.
To provide a system for passively and automatically adjusting the position of a vehicle component to a near optimum location based on the size of an occupant.
To provide adjustment apparatus and methods that reliably discriminate between a normally seated passenger and a forward facing child seat, between an abnormally seated passenger and a rear facing child seat, and whether or not the seat is empty and adjust the location and/or orientation relative to the occupant and/or operation of a part of the component or the component in its entirety based thereon.
To provide a system for recognizing a particular occupant of a vehicle and thereafter adjusting various components of the vehicle in accordance with the preferences of the recognized occupant.
To provide a pattern recognition system to permit more accurate location of an occupant'"'"'s head and the parts thereof and to use this information to adjust a vehicle component.
To provide a system for automatically adjusting the position of various components of the vehicle to permit safer and more effective operation of the vehicle including the location of the pedals and steering wheel.
To provide new and improved apparatus and methods for automatically adjusting a steering wheel based on the morphology of the driver, e.g., to place the steering wheel in an optimum position for driving the vehicle.
To provide a new and improved method and apparatus for adjusting a steering wheel in which the occupancy of the driver'"'"'s seat is evaluated and the steering wheel adjusted automatically relative to the driver based on the evaluated occupancy of the driver'"'"'s seat.
To recognize the presence of a human on a particular seat of a motor vehicle and then to determine his or her position and to use this position information to affect the operation of another vehicle system.
14.1 Control of Passive Restraints
It is another object of the present invention to provide new and improved arrangements and methods for controlling an occupant protection device based on the morphology of an occupant to be protected by the actuation of the device and optionally, the location of a seat on which the occupant is sitting. Control of the occupant protection device can entail suppression of actuation of the device, or adjustment of the actuation parameters of the device if such adjustment is deemed necessary.
Further objects of at least one of the inventions disclosed herein related to control of passive restraints are:
To determine the position, velocity and/or size of an occupant in a motor vehicle and to utilize this information to control the rate of gas generation, or the amount of gas generated, by an airbag inflator system or otherwise control the flow of gas into and/or out of an airbag.
To determine the fact that an occupant is not restrained by a seatbelt and therefore to modify the characteristics of the airbag system. This determination can be done either by monitoring the position or motion of the occupant or through the use of a resonating device placed on the shoulder belt portion of the seatbelt.
To determine the presence and/or position of rear seated occupants in the vehicle and to use this information to affect the operation of a rear seat protection airbag for frontal, rear or side impacts, or rollovers.
To recognize the presence of a rear facing child seat on a particular seat of a motor vehicle and to use this information to affect the operation of another vehicle system such as the airbag system.
To provide a vehicle interior monitoring system for determining the location of occupants within the vehicle and to include within the same system various electronics for controlling an airbag system.
To provide an occupant sensing system which detects the presence of a life form in a vehicle and under certain conditions, activates a vehicular warning system or a vehicular system to prevent injury to the life form.
To determine whether an occupant is out-of-position relative to the airbag and if so, to suppress deployment of the airbag in a situation in which the airbag would otherwise be deployed.
To adjust the flow of gas into and/or out of the airbag based on the morphology and/or position of the occupant to improve the performance of the airbag in reducing occupant injury.
To provide an occupant position sensor which reliably permits, and in a timely manner, a determination to be made that the occupant is out-of-position, or will become out-of-position, and likely to be injured by a deploying airbag and to then output a signal to suppress the deployment of the airbag.
14.2 Seat, Seatbelt, Steering Wheel and Pedal Adjustment and Resonators
Further objects of at least one of the inventions disclosed herein related to control of a seat and related adjustments are:
To determine the position of a seat in the vehicle using sensors remote from the seat and to use that information in conjunction with a memory system and appropriate actuators to position the seat in a predetermined location.
To remotely determine the fact that a vehicle door is not tightly closed using an illumination transmitting and receiving system such as one employing electromagnetic or acoustic waves.
To determine the position of the shoulder of a vehicle occupant and to use that information to control the seatbelt anchorage point.
To obtain information about an object in a vehicle using resonators or reflectors arranged in association with the object, such as the position of the object and the orientation of the object.
To provide a system designed to determine the orientation of a child seat using resonators or reflectors arranged in connection with the child seat.
To provide a system designed to determine whether a seatbelt is in use using resonators and reflectors, for possible use in the control of a safety device such as an airbag.
To provide a system designed to determine the position of an occupying item of a vehicle using resonators or reflectors, for possible use in the control of a safety device such as an airbag.
To provide a system designed to determine the position of a seat using resonators or reflectors, for possible use in the control of a vehicular component or system which would be affected by different seat positions.
To obtain information about an object in a vehicle using resonators or reflectors arranged in association with the object, such as the position of the object and the orientation of the object.
To provide a system for automatically adjusting the position of various components of the vehicle to permit safer and more effective operation of the vehicle including the location of the pedals and steering wheel.
To provide a system where the morphological characteristics of an occupant are measured by sensors located within the seat.
To provide a system and method wherein the weight of an occupant is determined utilizing sensors located on the seat structure.
To provide a system and method wherein other morphological properties are used to identify an individual including facial features, iris patterns, voiceprints, fingerprints and handprints.
To provide new and improved vehicular seats including a seat pressure or weight measuring feature and seat pressure or weight measuring methods for implementation in connection with vehicular seats.
14.3 Side Impacts
It is a further object of at least one of the inventions disclosed herein to determine the presence and/or position of occupants relative to the side impact airbag systems and to use this information to affect the operation of a side impact protection airbag system.
14.4 Children and Animals Left Alone
It is a further object of at least one of the inventions disclosed herein to detect whether children or animals are left alone in a vehicle or vehicle trunk and the environment is placing such children or animals in danger.
14.5 Vehicle Theft
It is a further object of at least one of the inventions disclosed herein to prevent vehicle theft by warning the owner that the vehicle is being stolen.
14.6 Security, Intruder Protection
It is a further object of at least one of the inventions disclosed herein to provide a security system for a vehicle which determines the presence of an unexpected life form in a vehicle and conveys the determination prior to entry of a driver into the vehicle.
It is a further object of at least one of the inventions disclosed herein to recognize a particular driver based on such factors as physical appearance or other attributes and to use this information to control another vehicle system such as a security system, seat adjustment, or maximum permitted vehicle velocity, among others.
14.7 Entertainment System Control
Further objects of at least one of the inventions disclosed herein related to control of the entertainment system are:
To affect the vehicle entertainment system, e.g., the speakers, based on a determination of the number, size and/or location of various occupants or other objects within the vehicle passenger compartment.
To determine the location of the ears of one or more vehicle occupants and to use that information to control the entertainment system, e.g., the speakers, so as to improve the quality of the sound reaching the occupants'"'"' ears through such methods as noise canceling sound.
14.8 HVAC
Further objects of at least one of the inventions disclosed herein related to control of the HVAC system are:
To affect the vehicle heating, ventilation and air conditioning system based on a determination of the number, size and location of various occupants or other objects within the vehicle passenger compartment.
To determine the temperature of an occupant based on infrared radiation coming from that occupant and to use that information to control the heating, ventilation and air conditioning system.
To recognize the presence of a human on a particular seat of a motor vehicle and to use this information to affect the operation of another vehicle system such as the airbag, heating and air conditioning, or entertainment systems, among others.
14.9 Obstruction Sensing
Further objects of at least one of the inventions disclosed herein related to sensing of window and door obstructions are:
To determine the extent of openness of a vehicle window and to use that information to affect another vehicle system.
To determine the presence of an occupant'"'"'s hand or other object in the path of a closing window and to affect the window closing system.
To determine the presence of an occupant'"'"'s hand or other object in the path of a closing door and to affect the door closing system.
To provide a new and improved system for monitoring closure of apertures.
To provide a new and improved system for monitoring closure of apertures in vehicles such as windows, doors, sunroofs, convertible tops and trunks.
To provide a new and improved system for monitoring closure of apertures such as windows, doors, sunroofs, convertible tops and trunks in vehicles and to suppress closure of the same if an obstacle is detected.
To provide a new and improved aperture monitoring system that does not depend on the reflectivity of the edges of the aperture and does not require the application of special materials to such edges.
To provide a new and improved aperture monitoring system that does not require the use of a calibration system such as a calibration LED.
14.10 Rear Impacts
It is a further object of at least one of the inventions disclosed herein to determine the position of the rear of an occupant'"'"'s head and to use that information to control the position of the headrest.
It is an object of the present invention to provide new and improved headrests for seats in a vehicle which offer protection for an occupant in the event of a crash involving the vehicle.
It is another object of the present invention to provide new and improved seats for vehicles which offer protection for an occupant in the event of a crash involving the vehicle.
It is still another object of the present invention to provide new and improved cushioning arrangements for vehicles and protection systems including cushioning arrangements which provide protection for occupants in the event of a crash involving the vehicle.
It is yet another object of the present invention to provide new and improved cushioning arrangements for vehicles and protection systems including cushioning arrangements which provide protection for occupants in the event of a collision into the rear of the vehicle, i.e., a rear impact.
It is yet another object of the present invention to provide new and improved vehicular systems which reduce whiplash injuries from rear impacts of a vehicle by causing the headrest to be automatically positioned proximate to the occupant'"'"'s head.
It is yet another object of the present invention to provide new and improved vehicular systems to position a headrest proximate to the head of a vehicle occupant prior to a pending impact into the rear of a vehicle.
It is yet another object of the present invention to provide a simple anticipatory sensor system for use with an adjustable headrest, or other safety system, to predict a rear impact.
It is yet another object of the present invention to provide a method and arrangement for protecting an occupant in a vehicle during a crash involving the vehicle using an anticipatory sensor system and a cushioning arrangement including a fluid-containing bag which is brought closer toward the occupant or ideally in contact with the occupant prior to or coincident with the crash. The bag would then conform to the portion of the occupant with which it is in contact.
It is yet another object of the present invention to provide an automatically adjusting system which conforms to the head and neck geometry of an occupant regardless of the occupant'"'"'s particular morphology to properly support both the head and neck.
14.11 Combined with SDM and Other Systems
It is a further object of at least one of the inventions disclosed herein to provide for the combining of the electronics of the occupant sensor and the airbag control module into a single package.
14.12 Exterior Monitoring
Further objects of at least one of the inventions disclosed herein related to monitoring the exterior environment of the vehicle are:
To provide a system for monitoring the environment exterior of a vehicle in order to determine the presence and classification, identification and/or location of objects in the exterior environment.
To provide an anticipatory sensor that permits accurate identification of the about-to-impact object in the presence of snow and/or fog whereby the sensor is located within the vehicle.
To provide a smart headlight dimmer system which senses the headlights from an oncoming vehicle or the tail lights of a vehicle in front of the subject vehicle and identifies these lights differentiating them from reflections from signs or the road surface and then sends a signal to dim the headlights.
To provide a blind spot detector which detects and categorizes an object in the driver'"'"'s blind spot or other location in the vicinity of the vehicle, and warns the driver in the event the driver begins to change lanes, for example, or continuously informs the driver of the state of occupancy of the blind spot.
To use the principles of time of flight to measure the distance to an occupant or object exterior to the vehicle.
To provide a camera system for interior and exterior monitoring, which can adjust on a pixel by pixel basis for the intensity of the received light.
To provide for the use of an active pixel camera for interior and exterior vehicle monitoring.
14.13 Monitoring of Other Vehicles Such as Cargo Containers, Truck Trailers and Railroad Cars
It is an object of some embodiments of the present invention to provide new and improved systems for remotely monitoring transportation assets and other movable and/or stationary items which have very low power requirements.
It is another object of some embodiments of the present invention to provide new and improved systems for attachment to shipping containers and other transportation assets which enable remote monitoring of the location, contents and/or interior or exterior environment of shipping containers or other assets and transportation assets and since it has a low power requirement, lasts for years without needing maintenance. It is yet another object of some embodiments of the invention to provide new and improved tracking methods and systems for tracking shipping containers and other transportation assets and enabling recording of the travels of the shipping container or transportation asset.
15.1 Classification, Location and Identification
The occupant position sensor of at least one of the inventions disclosed herein is adapted for installation in the passenger compartment of an automotive vehicle equipped with a passenger passive protective device (also referred to herein as an occupant restraint device) such as an inflatable airbag. When the vehicle is subjected to a crash of sufficient magnitude as to require deployment of the passive protective device (airbag), and the crash sensor system has determined that the device is to be deployed, the occupant position sensor and associated electronic circuitry determines the position of the vehicle occupant relative to the airbag and the velocity of the occupant, and disables deployment of the airbag if the occupant is positioned and/or will be positioned so that he/she is likely to be injured by the deploying airbag.
In order to achieve some of the above objects, an optical classification method for classifying an occupant in a vehicle in accordance with the invention comprises the steps of acquiring images of the occupant from a single camera and analyzing the images acquired from the single camera to determine a classification of the occupant. The single camera may be a digital CMOS camera, a high-power near-infrared LED, and the LED control circuit. It is possible to detect brightness of the images and control illumination of an LED in conjunction with the acquisition of images by the single camera. The illumination of the LED may be periodic to enable a comparison of resulting images with the LED on and the LED off so as to determine whether a daytime condition or a nighttime condition is present. The position of the occupant can be monitored when the occupant is classified as a child, an adult or a forward-facing child restraint.
In one embodiment, analysis of the images entails pre-processing the images, compressing the data from the pre-processed images, determining from the compressed data or the acquired images a particular condition of the occupant and/or condition of the environment in which the images have been acquired, providing a plurality of trained neural networks, each designed to determine the classification of the occupant for a respective one of the conditions, inputting the compressed data into one of the neural networks designed to determine the classification of the occupant for the determined condition to thereby obtain a classification of the occupant and subjecting the obtained classification of the occupant to post-processing to improve the probability of the classification of the occupant corresponding to the actual occupant. The pre-processing step may involve removing random noise and enhancing contrast whereby the presence of unwanted objects other than the occupant are reduced. The presence of unwanted contents in the images other than the occupant may be detected and the camera adjusted to minimize the presence of the unwanted contents in the images.
The post-processing may involve filtering the classification of the occupant from the neural network to remove random noise and/or comparing the classification of the occupant from the neural network to a previously obtained classification of the occupant and determining whether any difference in the classification is possible.
The classification of the occupant from the neural network may be displayed in a position visible to the occupant and enabling the occupant to change or confirm the classification.
The position of the occupant may be monitored when the occupant is classified as a child, an adult or a forward-facing child restraint. One way to do this is to input the compressed data or acquired images into an additional neural network designed to determine a recommendation for control of a system in the vehicle based on the monitoring of the position of the occupant. Also, a plurality of additional neural networks may be used, each designed to determine a recommendation for control of a system in the vehicle for a particular classification of occupant. In this case, the compressed data or acquired images is input into one of the neural networks designed to determine the recommendation for control of the system for the obtained classification of the occupant to thereby obtain a recommendation for the control of the system for the particular occupant.
In another embodiment, the method also involves acquiring images of the occupant from an additional camera, pre-processing the images acquired from the additional camera, compressing the data from the pre-processed images acquired from the additional camera, determining from the compressed data or the acquired images from the additional camera a particular condition of the occupant or condition of the environment in which the images have been acquired, inputting the compressed data from the pre-processed images acquired by the additional camera into one of the neural networks designed to determine the classification of the occupant for the determined condition to thereby obtain a classification of the occupant, subjecting the obtained classification of the occupant to post-processing to improve the probability of the classification of the occupant corresponding to the actual occupant and comparing the obtained classification using the images acquired form the additional camera to the images acquired from the initial camera to ascertain any variations in classification.
To further improve the operation of the ultrasonic portion of the system, especially when thermal gradients are present, the received signal is processed using a pseudo logarithmic compression circuit. This circuit compresses high amplitude reflections in comparison to low amplitude reflections and thereby diminishes the effects of diffraction cause by thermal gradients.
A method for categorizing and determining the position of an object in a passenger compartment of a vehicle in accordance with the invention comprises the steps of mounting a plurality of wave-receiving transducers on the vehicle, training a first neural network on signals from at least some of the transducers representative of waves received by the transducers when different objects in different positions are situated in the passenger compartment such that the first neural network provides an output signal indicative of the categorization of the object, and training a second neural network on signals from at least some of the transducers representative of waves received by the transducers when different objects in different positions are situated in the passenger compartment such that the second neural network provides an output signal indicative of the position of the object.
Another method for identifying an object in a passenger compartment of a vehicle comprises the steps of mounting a plurality of wave-emitting and receiving transducers on the vehicle, each transducer being arranged to transmit and receive waves at a different frequency, controlling the transducers to simultaneously transmit waves at the different frequencies into the passenger compartment, and identifying the object based on the waves received by at least some of the transducers after being modified by passing through the passenger compartment. The spacing between the frequencies of the waves transmitted and received by the transducers is determined in order to reduce the possibility of each transducer receiving waves transmitted by another transducer. The position of the object is determined based on the waves received by at least some of the transducers after being modified by passing through the passenger compartment.
When ultrasonic transducers are used, motion of a respective vibrating element of at least one transducer can be electronically reduced in order to reduce ringing of the transducer. Also, at least one transducer may be mounted in a respective tube having an opening through which the waves are transmitted and received.
A processor may be coupled to the transducers for controlling the transducers to simultaneously transmit waves at the different frequencies into the passenger compartment and receive signals representative of the waves received by the transducers after being modified by passing through the passenger compartment. The processor would then identify the object and/or determine the position of the object based on the signals representative of the waves received by at least some of the transducers.
One embodiment of the interior monitoring system in accordance with the invention comprises a device for irradiating at least a portion of the compartment or other part of a vehicle in which an occupying item is situated, a receiver system for receiving radiation from the occupying item, e.g., a plurality of receivers, each arranged at a discrete location, a processor coupled to the receivers for processing the received radiation from each receiver in order to create a respective electronic signal characteristic of the occupying item based on the received radiation, each signal containing a pattern representative of the occupying item, a categorization unit coupled to the processor for categorizing the signals, and an output device coupled to the categorization unit for affecting another system within the vehicle based on the categorization of the signals characteristic of the occupying item. The categorization unit may use a pattern recognition technique for recognizing and thus identifying the class of the occupying item by processing the signals into a categorization thereof based on data corresponding to patterns of received radiation and associated with possible classes of occupying items of the vehicle. Each signal may comprise a plurality of data, all of which is compared to the data corresponding to patterns of received radiation and associated with possible classes of contents of the vehicle. In one specific embodiment, the system includes a location determining unit coupled to the processor for determining the location of the occupying item, e.g., based on the received radiation such that the output device coupled to the location determining unit, in addition to affecting the other system based on the categorization of the signals characteristic of the occupying item, affects the system based on the determined location of the occupying item. In another embodiment to determine the presence or absence of an occupant, the categorization unit comprises a pattern recognition system for recognizing the presence or absence of an occupying item in the compartment by processing each signal into a categorization thereof signal based on data corresponding to patterns of received radiation and associated with possible occupying items of the vehicle and the absence of such occupying items.
In a disclosed method for determining the occupancy of a seat in a passenger compartment of a vehicle in accordance with the invention, waves such as ultrasonic or electromagnetic waves are transmitted into the passenger compartment toward the seat, reflected waves from the passenger compartment are received by a component which then generates an output representative thereof, the weight applied onto the seat is measured and an output is generated representative thereof and then the seated-state of the seat is evaluated based on the outputs from the sensors and the weight measuring unit.
The evaluation of the seated-state of the seat may be accomplished by generating a function correlating the outputs representative of the received reflected waves and the measured weight and the seated-state of the seat, and incorporating the correlation function into a microcomputer. In the alternative, it is possible to generate a function correlating the outputs representative of the received reflected waves and the measured weight and the seated-state of the seat in a neural network, and execute the function using the outputs representative of the received reflected waves and the measured weight as input into the neural network.
To enhance the seated-state determination, the position of a seat track of the seat is measured and an output representative thereof is generated, and then the seated-state of the seat is evaluated based on the outputs representative of the received reflected waves, the measured weight and the measured seat track position. In addition to or instead of measuring the seat track position, it is possible to measure the reclining angle of the seat, i.e., the angle between the seat portion and the back portion of the seat, and generate an output representative thereof, and then evaluate the seated-state of the seat based on the outputs representative of the received reflected waves, the measured weight and the measured reclining angle of the seat (and seat track position, if measured).
Furthermore, the output representative of the measured weight may be compared with a reference value, and the occupying object of the seat identified, e.g., as an adult or a child, based on the comparison of the measured weight with the reference value.
In another method disclosed herein for determining the identification and position of objects in a passenger compartment of a vehicle in accordance with the invention, electromagnetic waves are transmitted into the passenger compartment from one or more locations, a plurality of images of the interior of the passenger compartment are obtained, each from a respective location, a three-dimensional representation of a portion of the interior of the passenger compartment or of the occupying item is created from the images, and a pattern recognition technique is applied to the representation in order to determine the identification and position of the objects in the passenger compartment. The pattern recognition technique may be a neural network, fuzzy logic or an optical correlator or combinations thereof. The representation may be obtained by utilizing a scanning laser radar system where the laser is operated in a pulse mode and determining the distance from the object being illuminated using range gating. (See, for example, H. Kage, W. Freemen, Y Miyke, E. Funstsu, K. Tanaka, K. Kyuma “Artificial retina chips as on-chip image processors and gesture-oriented interfaces”, Optical Engineering, December, 1999, Vol. 38, Number 12, ISSN 0091-3286)
Also, disclosed herein is a system to identify, locate and monitor occupants, including their parts, and other objects in the compartment and objects outside of a vehicle, such as an automobile, container or truck, by illuminating the contents of the vehicle and/or objects outside of the vehicle with electromagnetic radiation, and preferably infrared radiation, using natural illumination such as from the sun, or using radiation naturally emanating from the object, and using one or more lenses to focus images of the contents onto one or more arrays of charge coupled devices (CCD'"'"'s), CMOS or equivalent arrays. Outputs from the arrays are analyzed by appropriate computational devices employing trained pattern recognition technologies, to classify, identify or locate the contents and/or external objects. In general, the information obtained by the identification and monitoring system may be used to affect the operation of at least one other system in the vehicle.
In some implementations of the invention, several CCD, CMOS or equivalent arrays are placed such that the distance from, and the motion of the occupant toward, the airbag can be monitored as a transverse motion across the field of the array. In this manner, the need to measure the distance from the array to the object is obviated. In other implementations, the source of infrared light is a pulse-modulated laser which permits an accurate measurement of the distance to the point of reflection through the technique of range gating to measure the time of flight of the radiation pulse.
In some applications, a trained pattern recognition system, such as a neural network, sensor fusion or neural-fuzzy system is used to identify the occupancy of the vehicle or an object exterior to the vehicle. In some of these cases, the pattern recognition system determines which of a library of images most closely matches the seated state of a particular vehicle seat and thereby the location of certain parts of an occupant can be accurately estimated from stored data relating to the matched images, thus removing the requirement for the pattern recognition system to locate the head of an occupant, for example.
In yet another embodiment of the invention, the system for determining the occupancy state of a seat in a vehicle includes a plurality of transducers including at least two wave-receiving or electric field transducers arranged in the vehicle, each providing data relating to the occupancy state of the seat. One wave-receiving or electric field transducer is arranged on or adjacent to a ceiling of the vehicle and a second wave-receiving or electric field transducer is arranged at a different location in the vehicle such that an axis connecting these transducers is substantially parallel to a longitudinal axis of the vehicle, substantially parallel to a transverse axis of the vehicle or passes through a volume above the seat. A processor is coupled to the transducers for receiving data from the transducers and processing the data to obtain an output indicative of the current occupancy state of the seat. The processor comprises an algorithm which produces the output indicative of the current occupancy state of the seat upon inputting a data set representing the current occupancy state of the seat and being formed from data from the transducers.
Another measuring position arrangement comprises a light source capable of directing individual pulses of light, preferably infrared, into the environment, at least one array of light-receiving pixels arranged to receive light after reflection by any objects in the environment and a processor for determining the distance between any objects from which any pulse of light is reflected and the light source based on a difference in time between the emission of a pulse of light by the light source and the reception of light by the array. The light source can be arranged at various locations in the vehicle as described above to direct light into external and/or internal environments, relative to the vehicle.
The portion of the apparatus which includes the ultrasonic, optical or electromagnetic sensors, weight measuring unit and processor which evaluate the occupancy of the seat based on the measured weight of the seat and its contents and the returned waves from the ultrasonic, optical or electromagnetic sensors, may be considered to constitute a seated-state detecting unit. The seated-state detecting unit may further comprise a seat track position-detecting sensor. This sensor determines the position of the seat on the seat track in the forward and aft direction. In this case, the evaluation circuit evaluates the seated-state, based on a correlation function obtain from outputs of the ultrasonic sensors, an output of the weight sensor(s), and an output of the seat track position detecting sensor. With this structure, there is the advantage that the identification between the flat configuration of a detected surface in a state where a passenger is not sitting in the seat and the flat configuration of a detected surface which is detected when a seat is slid backwards by the amount of the thickness of a passenger, that is, of identification of whether a passenger seat is vacant or occupied by a passenger, can be reliably performed. Furthermore, the seated-state detecting unit may also comprise a reclining angle detecting sensor, and the evaluation circuit may also evaluate the seated-state based on a correlation function obtained from outputs of the ultrasonic, optical or electromagnetic sensors, an output of the weight sensor(s), and an output of the reclining angle detecting sensor. In this case, if the tilted angle information of the back portion of the seat is added as evaluation information for the seated-state, identification can be clearly performed between the flat configuration of a surface detected when a passenger is in a slightly slouching state and the configuration of a surface detected when the back portion of a seat is slightly tilted forward and similar difficult-to-discriminate cases.
This embodiment may even be combined with the output from a seat track position-detecting sensor to further enhance the evaluation circuit. Moreover, the seated-state detecting unit may comprise a comparison circuit for comparing the output of the weight sensor(s) with a reference value. In this case, the evaluation circuit identifies an adult and a child based on the reference value. Preferably, the seated-state detecting unit comprises: a plurality of ultrasonic, optical or electromagnetic sensors for transmitting ultrasonic or electromagnetic waves toward a seat and receiving reflected waves from the seat; one or more pressure or weight sensors for detecting seat pressure applied by or weight of a passenger in the seat; a seat track position detecting sensor; a reclining angle detecting sensor; and a neural network to which outputs of the ultrasonic or electromagnetic sensors and the pressure or weight sensor(s), an output of the seat track position detecting sensor, and an output of the reclining angle detecting sensor are inputted and which evaluates several kinds of seated-states, based on a correlation function obtained from the outputs. The kinds of seated-states that can be evaluated and categorized by the neural network include the following categories, among others, (i) a normally seated passenger and a forward facing child seat, (ii) an abnormally seated passenger and a rear-facing child seat, and (iii) a vacant seat. The seated-state detecting unit may further comprise a comparison circuit for comparing the output of the seat pressure or weight sensor(s) with a reference value and a gate circuit to which the evaluation signal and a comparison signal from the comparison circuit are input. This gate circuit, which may be implemented in software or hardware, outputs signals which evaluate several kinds of seated-states. These kinds of seated-states can include a (i) normally seated passenger, (ii) a forward facing child seat, (iii) an abnormally seated passenger, (iv) a rear facing child seat, and (v) a vacant seat. With this arrangement, the identification between a normally seated passenger and a forward facing child seat, the identification between an abnormally seated passenger and a rear facing child seat, and the identification of a vacant seat can be more reliably performed. The outputs of the plurality of ultrasonic or electromagnetic sensors, the output of the seat pressure or weight sensor(s), the outputs of the seat track position detecting sensor, and the outputs of the reclining angle detecting sensor are inputted to the neural network or other pattern recognition circuit, and the neural network determines the correlation function, based on training thereof during a training phase. The correlation function is then typically implemented in or incorporated into a microcomputer. For the purposes herein, neural network will be used to include both a single neural network, a plurality of neural networks, and other similar pattern recognition circuits or algorithms and combinations thereof including the combination of neural networks and fuzzy logic systems such as neural-fuzzy systems. To provide the input from the ultrasonic or electromagnetic sensors to the neural network, it is preferable that an initial reflected wave portion and a last reflected wave portion are removed from each of the reflected waves of the ultrasonic or electromagnetic sensors and then the output data is processed. This is a form of range gating. With this arrangement, the portions of the reflected ultrasonic or electromagnetic wave that do not contain useful information are removed from the analysis and the presence and recognition of an object on the passenger seat can be more accurately performed. The neural network determines the correlation function by performing a weighting process, based on output data from the plurality of ultrasonic or electromagnetic sensors, output data from the seat pressure or weight sensor(s), output data from the seat track position detecting sensor if present, and/or on output data from the reclining angle detecting sensor if present. Additionally, in advanced systems, outputs from the heartbeat and occupant motion sensors may be included.
With this arrangement, the portions of the reflected ultrasonic wave that do not contain useful information are removed from the analysis and the presence and recognition of an object on the passenger seat can be more accurately performed. Similar data pruning can take place with electromagnetic sensors on both a temporal or spatial basis.
One method described herein for determining the identification and position of objects in a passenger compartment of a vehicle in accordance with at least one invention herein comprises the steps of transmitting electromagnetic waves (optical or non-optical) into the passenger compartment from one or more locations, obtaining a plurality of images of the interior of the passenger compartment from several locations, and comparing the images of the interior of the passenger compartment with stored images representing different arrangements of objects in the passenger compartm