Vehicle vision system

• US 9,701,258 B2
• Filed: 07/07/2014
• Issued: 07/11/2017
• Est. Priority Date: 07/09/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Tiryaki, Riza
• Primary Examiner(s): Truong, Nguyen
• Application Number:
  US14/324,696
• Publication Number:
  US 20150015710A1
• Time in Patent Office: 1,100 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A vision system of a vehicle includes a camera in the vehicle with a field of view that encompasses at least a hand of a driver of the vehicle. A control includes an image processor operable to process image data captured by the camera. The control, responsive to image data processing, is operable to determine a hand of the driver of the vehicle and, responsive to determination of movement of the driver'"'"'s hand, is operable to determine if the movement is indicative of a gesture for controlling an accessory of the equipped vehicle. The control learns habitual movements of the driver'"'"'s hand and, responsive to (i) determination that a detected gesture is not indicative of a learned habitual movement and (ii) determination that the detected gesture is indicative of a gesture for controlling an accessory of the equipped vehicle, the control generates an output for controlling the accessory.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera disposed in an interior cabin of a vehicle equipped with said vision system, said camera having a field of view that encompasses at least a hand of a driver of the equipped vehicle;

a control comprising an image processor operable to process image data captured by said camera;

wherein said control, responsive to image processing of captured image data by said image processor, is operable to determine a movement of a hand of the driver of the equipped vehicle;

wherein, responsive to determination of movement of the driver'"'"'s hand, said control is operable to determine if the movement is indicative of a gesture for controlling an accessory of the equipped vehicle;

wherein said control learns habitual movements of the driver'"'"'s hand and is operable to distinguish intended gestures from learned habitual movements; and

wherein, responsive to (i) determination that a detected gesture is not indicative of a learned habitual movement and (ii) determination that the detected gesture is indicative of an intended gesture for controlling an accessory of the equipped vehicle, said control generates an output for controlling the accessory.

2. The vision system of claim 1, wherein said control is operable to recognize the learned habitual movements of the driver to reduce false determinations of gestures.

3. The vision system of claim 1, wherein said control learns habitual movements of multiple drivers and wherein said control, responsive to identification of the particular driver operating the equipped vehicle, is operable to determine whether a detected gesture of the particular driver is indicative of a learned habitual movement of that particular driver.

4. The vision system of claim 3, wherein identification of the particular driver is made via one of (i) a user input and (ii) image processing of captured image data by said image processor.

5. The vision system of claim 1, wherein the accessory comprises a plurality of cameras disposed at the equipped vehicle and having respective fields of view exterior of the equipped vehicle.

6. The vision system of claim 5, wherein said vision system controls at least one of (a) a viewing angle of individual cameras and (b) a viewing angle of a virtual camera.

7. The vision system of claim 1, comprising a display screen disposed in the interior cabin of the equipped vehicle and viewable by the driver of the equipped vehicle.

8. The vision system of claim 7, wherein said display screen is operable to display at least one virtual user input and wherein said control is operable to determine actuation of the virtual user input responsive to movement of the driver'"'"'s hands relative to a virtual location and type of user input of the displayed virtual user input.

9. The vision system of claim 1, wherein said control learns habitual movements of the driver'"'"'s hand responsive to image processing of multiple frames of captured image data by said image processor.

10. The vision system of claim 1, wherein said control, after learning habitual movements of the driver'"'"'s hand, is operable to discern an intentional gesture by the driver to control an accessory from a habitual movement of the driver'"'"'s hand.

11. The vision system of claim 1, wherein the learned habitual movement comprises at least one of (i) a repeating scratching movement, (ii) a repeating reach towards a region in the vehicle, (iii) a repeating movement corresponding to music playing in the vehicle.

12. A vision system of a vehicle, said vision system comprising:

a camera disposed in an interior cabin of a vehicle equipped with said vision system, said camera having a field of view that encompasses at least a hand of a driver of the equipped vehicle;

a control comprising an image processor operable to process image data captured by said camera;

wherein said control, responsive to image processing of captured image data by said image processor, is operable to determine a movement of a hand of the driver of the equipped vehicle;

wherein, responsive to determination of movement of the driver'"'"'s hand, said control is operable to determine if the movement is indicative of a gesture for controlling an accessory of the equipped vehicle;

wherein said control learns habitual movements of the driver'"'"'s hand for multiple drivers and stores habitual movement data for the multiple drivers, and wherein said control is operable to distinguish intended gestures of a driver from learned habitual movements of that driver;

wherein said control learns habitual movements of the driver'"'"'s hand responsive to image processing of multiple frames of captured image data by said image processor; and

wherein, responsive to (i) identification of the particular driver operating the equipped vehicle, (ii) determination that a detected gesture is not indicative of a learned habitual movement of that particular driver and (iii) determination that the detected gesture is indicative of an intended gesture for controlling an accessory of the equipped vehicle, said control generates an output for controlling the accessory.

13. The vision system of claim 12, wherein identification of the particular driver is made via one of (i) a user input and (ii) image processing of captured image data by said image processor.

14. The vision system of claim 12, wherein the accessory comprises a plurality of cameras disposed at the equipped vehicle and having respective fields of view exterior of the equipped vehicle, and wherein said vision system controls at least one of (a) a viewing angle of individual cameras and (b) a viewing angle of a virtual camera.

15. The vision system of claim 12, comprising a display screen disposed in the interior cabin of the equipped vehicle and viewable by the driver of the equipped vehicle, wherein said display screen is operable to display at least one virtual user input and wherein said control is operable to determine actuation of the virtual user input responsive to movement of the driver'"'"'s hands relative to a virtual location and type of user input of the displayed virtual user input.

16. The vision system of claim 12, wherein the learned habitual movement comprises at least one of (i) a repeating scratching movement, (ii) a repeating reach towards a region in the vehicle, (iii) a repeating movement corresponding to music playing in the vehicle.

17. A vision system of a vehicle, said vision system comprising:

a camera disposed in an interior cabin of a vehicle equipped with said vision system, said camera having a field of view that encompasses at least a hand of a driver of the equipped vehicle;

a control comprising an image processor operable to process image data captured by said camera;

wherein said control, responsive to image processing of captured image data by said image processor, is operable to determine a movement of a hand of the driver of the equipped vehicle;

wherein, responsive to determination of movement of the driver'"'"'s hand, said control is operable to determine if the movement is indicative of a gesture for controlling an accessory of the equipped vehicle;

wherein said control learns habitual movements of the driver'"'"'s hand and is operable to distinguish intended gestures from learned habitual movements;

wherein said control learns habitual movements of the driver'"'"'s hand responsive to image processing of multiple frames of captured image data by said image processor;

wherein said control, after learning habitual movements of the driver'"'"'s hand, is operable to discern an intended gesture by the driver to control an accessory from a habitual movement of the driver'"'"'s hand; and

wherein, responsive to (i) determination that a detected gesture is not indicative of a learned habitual movement and (ii) determination that the detected gesture is indicative of an intended gesture for controlling an accessory of the equipped vehicle, said control generates an output for controlling the accessory.

18. The vision system of claim 17, wherein said control learns habitual movements of multiple drivers and wherein said control, responsive to identification of the particular driver operating the equipped vehicle, is operable to determine whether a detected gesture of the particular driver is indicative of a learned habitual movement of that particular driver, and wherein identification of the particular driver is made via one of (i) a user input and (ii) image processing of captured image data by said image processor.

19. The vision system of claim 17, comprising a display screen disposed in the interior cabin of the equipped vehicle and viewable by the driver of the equipped vehicle, and wherein said display screen is operable to display at least one virtual user input and wherein said control is operable to determine actuation of the virtual user input responsive to movement of the driver'"'"'s hands relative to a virtual location and type of user input of the displayed virtual user input.

20. The vision system of claim 17, wherein the learned habitual movement comprises at least one of (i) a repeating scratching movement, (ii) a repeating reach towards a region in the vehicle, (iii) a repeating movement corresponding to music playing in the vehicle.