SCENE AWARENESS SYSTEM FOR A VEHICLE

• US 20160039285A1
• Filed: 10/22/2015
• Published: 02/11/2016
• Est. Priority Date: 04/25/2013
• Status: Abandoned Application

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Mathieu, Roy J., Szczerba, Joseph F., Jones, Micah R., Seder, Thomas A.
• Application Number:
  US14/920,420
• Publication Number:
  US 20160039285A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: B60C 9/00 Reinforcements or ply arrangement of pneumatic tyres inserts having reinforcing means B60C5/08; bead structure, e.g. turnup or overlap construction, B60C15/00
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 2360/197 Blocking or enabling of input functions
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/25 using filters
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60R 2300/8006 : for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8093 : for obstacle warning
• CPC Class Codes: B60W 2050/146 : Display means
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60Y 2302/03 : Actuating a signal or alarm device
• CPC Class Codes: G02B 2027/0112 : comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0118 : comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0141 : characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0194 : with combiner of laminated type, for optical or mechanical aspects
• CPC Class Codes: G02B 27/0101 : characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0927 : Systems for changing the beam intensity distribution, e.g. Gaussian to top-hat
• CPC Class Codes: G02B 27/0961 : Lens arrays lens arrays per se G02B3/0006
• CPC Class Codes: G02B 6/005 : provided by one optical element, or plurality thereof, placed on the light output side of the light guide
• CPC Class Codes: G05D 3/00 : Control of position or direction G05D1/00 takes precedence; numerical control to execute positioning G05B19/18
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G07C 5/02 : Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/08 : Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G08C 17/02 : using a radio link

Abstract

A method of alerting a user of a vehicle as to a scene, external to the vehicle, includes capturing data pertaining to the scene, external to the vehicle. The captured data, pertaining to the scene external to the vehicle, is transmitted to a processor. The determination is made, in the processor that a characteristic is in the scene that requires the user'"'"'s visual attention. Data pertaining to a gaze of the user is captured. Captured data, pertaining to the gaze of the user, is transmitted to the processor. A determination is made, in the processor that the drive is gazing toward a user device. A signal is transmitted to the user device such that the user device alerts the user as to the characteristic in the scene requiring the user'"'"'s visual attention.

Claims

1. A method of alerting a user of a vehicle as to a scene, external to the vehicle, the method comprising:

capturing data pertaining to the scene, external to the vehicle;

transmitting the captured data pertaining to the scene, external to the vehicle, to a processor;

determining, in the processor, a characteristic of the scene requiring the user'"'"'s visual attention;

capturing data pertaining to a gaze of the user;

transmitting the captured data pertaining to the gaze of the user to the processor;

determining, in the processor, the driver is gazing toward a user device; and

transmitting a signal to the user device such that the user device alerts the user as to the characteristic of the scene requiring the user'"'"'s visual attention.

2. A method, as set forth in claim 1, wherein transmitting is further defined as transmitting a signal to the user device such that the user device alerts the user as to the characteristic of the scene requiring the user'"'"'s visual attention by clearing the screen of the user device.

3. A method, as set forth in claim 2, wherein transmitting is further defined as transmitting a signal to the user device such that the user device alerts the user as to the characteristic of the scene requiring the user'"'"'s visual attention by displaying a visual depiction of the characteristic of the scene requiring the user'"'"'s visual attention on the display screen of the user device.

4. A method, as set forth in claim 1, further comprising:

receiving data pertaining to operation of the vehicle; and

determining the vehicle is being operated by the user.

5. A method, as set forth in claim 4, wherein capturing data pertaining to a gaze of the user is further defined as capturing data pertaining to a gaze of the user when a determination is made that the vehicle is being operated by the user.

6. A method, as set forth in claim 4, wherein receiving data is further defined as receiving geographic data regarding a vehicle position.

7. A method, as set forth in claim 1, wherein capturing data pertaining to the scene, external to the vehicle, is further defined as capturing data pertaining to the scene, external to the vehicle, with at least one external camera.

8. A method, as set forth in claim 1, wherein capturing data pertaining to a gaze of the user is further defined as capturing data pertaining to a gaze of the user when a determination is made that a characteristic of the scene requiring the user'"'"'s attention is detected in the scene.

9. A method, as set forth in claim 1, wherein capturing data pertaining to a gaze of the user is further defined as capturing data pertaining to a gaze of the user with at least one internal camera.

10. A method, as set forth in claim 9, wherein capturing data pertaining to a gaze of the user is further defined as capturing data pertaining to an orientation of at least one of the head and the eyes of the user.

11. A method, as set forth in claim 1, wherein capturing data pertaining to a gaze of the user is further defined as capturing data pertaining to a gaze of the user after determining, in the processor, a characteristic of the scene requiring the user'"'"'s visual attention.

12. A scene awareness system for a vehicle, the scene awareness system comprising:

at least one exterior camera configured to capture data pertaining to a scene, external to the vehicle;

at least one interior camera configured to capture data pertaining to an orientation of a gaze of a user of the vehicle;

a vehicle controller in communication with each of the at least one exterior camera and the at least one interior camera;

wherein the vehicle controller is configured to determine whether at least one characteristic in the scene, exterior to the vehicle, requires the user'"'"'s visual attention;

wherein the vehicle controller is configured to determine whether the user is gazing at a user device; and

wherein the vehicle controller is configured to transmit a signal to the user device such that the user device alerts the user as to the characteristic in the scene requiring the user'"'"'s visual attention when the controller determines there is at least one characteristic in the scene requiring the user'"'"'s attention simultaneous with the controller determining the user is gazing at the user device.

13. A scene awareness system, as set forth in claim 12, further comprising at least one sensor configured for transmitting data to the vehicle controller;

wherein the data pertains to operation of the vehicle.

14. A vehicle comprising:

a body;

a scene awareness system including;

at least one exterior camera operatively attached to the body;

wherein the at least one exterior camera is configured to capture data pertaining to a scene, external to the vehicle;

at least one interior camera operatively attached to the body;

wherein the at least one interior camera is configured to capture data pertaining to an orientation of a gaze of a user of the vehicle;

a vehicle controller in communication with each of the at least one exterior camera and the at least one interior camera, the vehicle controller operable for;

receiving data pertaining to the scene, external to the vehicle, from the at least one exterior camera;

determining, in the processor, a characteristic in the scene requiring the user'"'"'s visual attention;

receiving data pertaining to a gaze of the user from the at least one interior camera;

determining the driver is gazing toward a user device; and

transmitting a signal to a user device such that the user device alerts the user as to the characteristic in the scene requiring the user'"'"'s visual attention.

15. A vehicle, as set forth in claim 14, wherein the body extends between a forward end and a rearward end;

and

wherein the at least one exterior camera is operatively attached to the body, proximate the forward end.

16. A vehicle, as set forth in claim 15, wherein the body defines an interior configured for receiving the user of the vehicle therein;

wherein the at least one interior camera is operatively disposed in the interior of the body.

17. A vehicle, as set forth in claim 16, further comprising at least one sensor configured to be in operative communication with the vehicle controller;

wherein the vehicle controller is further operable for receiving data, pertaining to operation of the vehicle, from the at least one sensor.

18. A vehicle, as set forth in claim 14, wherein the vehicle controller is further operable for:

receiving data pertaining to operation of the vehicle; and

determining the vehicle is being operated by the user.