Dynamic surround camera system

• US 10,358,088 B1
• Filed: 02/21/2017
• Issued: 07/23/2019
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Pertsel, Shimon
• Primary Examiner(s): Navas, Jr., Edemio
• Application Number:
  US15/437,655
• Time in Patent Office: 882 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/173 Reversing assist
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G02B 13/06 : Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30264 : Parking
• CPC Class Codes: G06T 3/18 : Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/20 : using local operators
• CPC Class Codes: G06T 5/80 : Geometric correction
• CPC Class Codes: G06T 7/80 : Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/268 : Signal distribution or switching
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A system comprising a plurality of camera sensors and a processor. The camera sensors may each be configured to generate a video signal having a view. The processor may be configured to (a) generate an output video signal by selecting one or more of the video signals and (b) present the selected output video signal to a display for viewing. The output video signal may be automatically selected in response to a selection signal. The selection signal may be generated in response to analysis of one or more images in the video signals. The analysis of the one or more images in the video signals may comprise determining an appropriate view for a driver. The processor may remove possible warping present in the video signals.

Claims

1. A system comprising:

a plurality of camera sensors each configured to generate a video signal having a view; and

a processor configured to (a) generate an output video signal by selecting one or more of said video signals and (b) present said selected output video signal to a display for viewing, wherein (i) said output video signal is automatically selected in response to a selection signal, (ii) said selection signal is generated in response to analysis of one or more images in said video signals, (iii) said analysis of said one or more images in said video signals comprises determining an appropriate view to present on said display to be viewed by a driver, (iv) said processor removes possible warping present in said video signals, (v) said selection signal is generated in further response to rules based on a priority of conditions from (a) detection of a hazard determined in said analysis, (b) signals from a vehicle, (c) movements of said driver captured by one of said camera sensors, and (d) previous habits of said driver stored from said analysis, and (vi) said processor lowers said priority of mild movements of said driver determined from said analysis when generating said selection signal based on said rules.

2. The system according to claim 1, wherein said appropriate view comprises at least one of a bird'"'"'s eye view and a view pointed in a direction of said detected hazard.

3. The system according to claim 1, wherein said selection signal is generated in further response to one or more external sensors comprising (i) an infra-red signal, (ii) a radar signal, (iii) a lidar signal, and (iv) a proximity sensor.

4. The system according to claim 1, wherein said system is configured to have a manual override to enable said driver to cause said selection signal to select one of said video signals not determined to be said appropriate view.

5. The system according to claim 1, wherein said system is configured to display both said appropriate view and a second view.

6. The system according to claim 1, wherein said processor is further configured to determine said appropriate view based on one or more driver preferences and said driver preferences include driver specific thresholds.

7. The system according to claim 1, wherein said selection signal is generated in further response to one or more global positioning system (GPS) coordinates.

8. The system according to claim 1, wherein said selection signal is generated in further response to characteristics of said vehicle.

9. The system according to claim 8, wherein said characteristics of said vehicle include a size of a blind spot and a side of the vehicle that a steering wheel is located.

10. The system according to claim 1, wherein (i) said signals from said vehicle are presented by a vehicle controller and (ii) said vehicle controller provides an acceleration of said vehicle.

11. The system according to claim 1, wherein said signals from said vehicle provide at least one of (i) an input received from a lane detection system and (ii) a predetermined distance calculated through a front collision warning system.

12. The system according to claim 1, wherein (A) said processor is further configured to (i) implement hysteresis and (ii) make assumptions based on previous input.

13. The system according to claim 1, wherein (i) said appropriate view is generated by said processor stitching together two or more smaller views and (ii) said smaller views are each generated by one of said camera sensors.

14. The system according to claim 1, wherein said rules are further based on a position of said driver.

15. A method for automatically selecting an output video signal to present to a driver of a vehicle, comprising the steps of:

(A) receiving a plurality of video signals from a respective plurality of camera sensors;

(B) generating said output video signal by selecting one or more of said video signals; and

(C) presenting said selected output video signal to a display for viewing, wherein (i) said output video signal is automatically selected in response to a selection signal, (ii) said selection signal is generated in response to analysis of one or more images in said video signals, (iii) said analysis of said one or more images in said video signals comprises determining an appropriate view to present on said display to be viewed by said driver, (iv) possible warping present in said video signals is removed, (v) said selection signal is generated in further response to rules based on a priority of conditions from (a) detection of a hazard determined in said analysis, (b) signals from said vehicle, (c) movements of said driver captured by one of said camera sensors, and (d) previous habits of said driver stored from said analysis, and (vi) said processor lowers said priority of mild movements of said driver determined from said analysis when generating said selection signal based on said rules.

16. The method according to claim 15, wherein said method selects one of the video signals pointed at a blindspot when said vehicle is moving forward.

17. The method according to claim 15, wherein said method displays a blindspot when the vehicle is moving forward at a predetermined speed.

18. The method according to claim 15, further comprising the steps of:

determining when a brake pedal is pressed;

determining when a turn signal is on;

when both said brake pedal is pressed and said turn signal is on, displaying a blindspot; and

when said brake pedal is pressed and said turn signal is not on, displaying a rear view.

19. The method according to claim 15, wherein analysis of the images in said video signals comprises defining a number of said rules used for selecting one of said video signals to present to said driver.

20. The method according to claim 15, wherein said method displays a bird'"'"'s eye view and a view in a direction of said hazard when said hazard is detected.