Portable vision system

• US 8,471,691 B2
• Filed: 06/15/2010
• Issued: 06/25/2013
• Est. Priority Date: 06/15/2010
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Naik, Sanjeev M., Saraydar, Cem U.
• Primary Examiner(s): Wu, Daniel
• Assistant Examiner(s): MA, KAM WAN
• Application Number:
  US12/816,070
• Publication Number:
  US 20110304444A1
• Time in Patent Office: 1,106 Days
• Field of Search: 340/438, 340/425.5, 700 90-306, 709201-203, 701/1
• US Class Current: 340/425.5
• CPC Class Codes: B60K 2360/184 Displaying the same information on different displays
• CPC Class Codes: B60K 2360/589 Wireless data transfers
• CPC Class Codes: B60K 2370/184 Displaying the same information on different displays
• CPC Class Codes: B60K 2370/52 Control of displays
• CPC Class Codes: B60K 2370/589 Wireless
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04M 1/724098 Interfacing with an on-board device of a vehicle for hands-free speaking in vehicles H04M1/6075

Abstract

A method and system for integrating a camera-equipped handheld mobile device with a vehicle'"'"'s electronic systems, such that the imaging capabilities of the mobile device can be utilized by the vehicle, and information from vehicle'"'"'s systems can be utilized by the mobile device. The mobile device is placed in a mounting fixture, which can provide power to the mobile device, and is designed to dynamically aim the mobile device'"'"'s camera based on driving conditions. The mobile device can communicate with the vehicle'"'"'s systems wirelessly, or via a wired connection. Images from the mobile device'"'"'s camera are provided to a processor onboard the vehicle, and can be used by vehicle safety and convenience systems. The mobile device can also receive information from the vehicle and use the information in applications on the mobile device. Computing workload can be shared between the vehicle'"'"'s processor and the mobile device.

Claims

1. A method for integrating a mobile device with electronic systems in a vehicle, said method comprising:

establishing communication between the mobile device and a processor in the vehicle;

sharing data between the mobile device and the processor;

evaluating a driving scenario based on input from a camera in the mobile device and input from sensors onboard the vehicle;

creating a saliency map by determining a priority of features to be employed based on the driving scenario, and employing the features based on the priority saliency map, where the features to be employed include one or more of lane departure warning, collision detection warning, backup warning, driver attentiveness, navigation assistance, road geometry estimation, traffic light state determination, and augmented reality;

optimizing processing based on the saliency map, including distributing computational tasks from the processor to the mobile device;

providing alerts and updating a display unit in the vehicle based on information from the features being employed; and

updating the driving scenario based on information from the features being employed.

2. The method of claim 1 wherein optimizing processing includes distributing computational tasks from the mobile device to the processor.

3. The method of claim 1 wherein establishing communication between the mobile device and the processor uses a wireless communication protocol.

4. The method of claim 1 further comprising optimizing capturing of images and data based on the features being employed.

5. The method of claim 4 wherein optimizing capturing of images and data includes designating images to capture by the camera in the mobile device and, if one or more cameras in the vehicle are available, designating images to use from the cameras in the vehicle, based on the features being employed.

6. The method of claim 4 wherein optimizing capturing of images and data includes adjusting an aiming direction and a field of view of the camera in the mobile device based on the features being employed.

7. The method of claim 1 wherein sharing data between the mobile device and the processor includes using the mobile device to connect via a cellular phone call to a data source, and providing data from the data source to the processor.

8. A method for integrating a mobile device with electronic systems in a vehicle, said method comprising:

establishing communication between the mobile device and a processor in the vehicle using a wireless communication protocol;

sharing data between the mobile device and the processor;

evaluating a driving scenario based on input from a camera in the mobile device and input from sensors onboard the vehicle;

determining a priority of features to be employed based on the driving scenario, and employing the features based on the priority, where the features to be employed include one or more of lane departure warning, collision detection warning, backup warning, driver attentiveness, navigation assistance, road geometry estimation, traffic light state determination, and augmented reality;

optimizing processing based on the features being employed;

optimizing capturing of images and data based on the features being employed, including automatically adjusting, using a mounting fixture controlled by the processor, an aiming direction and a field of view of the camera in the mobile device based on the features being employed;

providing alerts and updating a display unit in the vehicle based on information from the features being employed; and

updating the driving scenario based on information from the features being employed.

9. The method of claim 8 wherein optimizing processing includes distributing computational tasks from the processor to the mobile device.

10. The method of claim 8 wherein optimizing processing includes distributing computational tasks from the mobile device to the processor.

11. The method of claim 8 wherein optimizing capturing of images and data includes designating images to capture by the camera in the mobile device and, if one or more cameras in the vehicle are available, designating images to use from the cameras in the vehicle, based on the features being employed.

12. An integrated electronic system for a vehicle, said integrated electronic system comprising:

a mobile device including a camera;

a mounting fixture for mounting the mobile device in the vehicle;

a display unit in the vehicle;

a processor in the vehicle, said processor being configured to receive image data from the camera in the mobile device, evaluate a driving scenario, create a saliency map by determining a priority of features to be employed based on the driving scenario, employ the features based on the saliency map, where the features to be employed include one or more of lane departure warning, collision detection warning, backup warning, driver attentiveness, navigation assistance, road geometry estimation, traffic light state determination, and augmented reality, optimize image capturing and processing, provide alerts, and update the display unit, said processor also being configured to control the mounting fixture to adjust an aiming position of the camera in the mobile device based on the driving scenario; and

a communication link between the mobile device and the processor for sharing images and data between the mobile device and the processor.

13. The integrated electronic system of claim 12 wherein the mobile device is configured to receive data from the processor.

14. The integrated electronic system of claim 12 wherein the communication link uses a wireless communication protocol.

15. The integrated electronic system of claim 12 wherein the processor optimizes image capturing and processing on both the mobile device and the vehicle.

16. The integrated electronic system of claim 12 wherein the processor performs computational tasks for the mobile device, and the mobile device performs computational tasks for the processor.