Vehicle occupant viewing systems and methods

• US 10,017,117 B2
• Filed: 05/18/2016
• Issued: 07/10/2018
• Est. Priority Date: 05/18/2016
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Lewis, Allan K., Chaney, Jr., Michael T., Naserian, Mohammad
• Primary Examiner(s): Nguyen, Huy Thanh
• Application Number:
  US15/158,186
• Publication Number:
  US 20170334357A1
• Time in Patent Office: 783 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/008 allowing the driver to see passengers, e.g. for busses combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 1/29 for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicle occupant viewing system and a corresponding method for displaying a vehicle occupant on an image display unit. The system may comprise one or more vehicle interior cameras for obtaining occupant images of one or more occupants in rear seats of the vehicle. The system may include two vehicle interior cameras, one in a forward facing orientation and another in a rear facing orientation, which can capture occupant images of occupants in forward facing positions and occupants in rear facing positions such as infants in car seats. The system may also include one or more modular vehicle interior cameras that are moveable between various orientations in order to capture occupant images of either forward or rear facing passengers. Adjustments can be made to display the occupant images to the driver in a more intuitive arrangement.

Claims

1. A vehicle occupant viewing system, comprising:

an image display unit;

a vehicle interior camera having a field of view at least partially covering an interior cabin of the vehicle; and

a video processing unit coupled to the image display unit and the vehicle interior camera, the video processing unit being configured to receive input from the vehicle interior camera, determine whether an occupant is present in the field of view of the vehicle interior camera, and when an occupant is present in the field of view, overlay an occupant image of the occupant onto another image or video being shown in the image display unit, a location of the overlaid occupant image being dependent on the vehicle interior camera'"'"'s orientation and the occupant'"'"'s position in the vehicle, wherein the video processing unit is further configured to display two or more occupant images from occupants in one row of seating on opposite sides of the image display unit with at least part of the another image or video being shown in the image display unit at least partially between the two or more occupant images.

2. The vehicle occupant viewing system of claim 1, wherein the video processing unit is configured to extract video data from an occupant region of interest to form the occupant image and overlay the occupant image in the form of a video stream onto the other image or video being shown in the image display unit.

3. The vehicle occupant viewing system of claim 1, wherein the image display unit is an electronic display mirror and the other image or video of the image display unit includes an image or video of an area behind the vehicle.

4. The vehicle occupant viewing system of claim 1, wherein the image display unit is a touch screen and a user can X-out one or more occupant images being displayed on the image display unit.

5. The vehicle occupant viewing system of claim 1, wherein the vehicle interior camera is a modular vehicle interior camera movable between the forward facing orientation and the rearward facing orientation.

6. The vehicle occupant viewing system of claim 5, further comprising a plurality of stationary vehicle interior camera docking stations coupled to a vehicle communication network, wherein the modular vehicle interior camera has a unique identifier and the video processing unit can determine the vehicle interior camera'"'"'s orientation based on the unique identifier of the modular vehicle interior camera or based on the coupling of the modular vehicle interior camera with at least one of the stationary vehicle interior camera docking stations.

7. The vehicle occupant viewing system of claim 1, wherein the video processing unit is configured to determine whether the vehicle interior camera is in a forward facing orientation that is oriented toward a front end of the vehicle or a rearward facing orientation that is oriented toward a rear end of the vehicle, and display an occupant image on a side of the image display unit depending on whether the vehicle interior camera is in the forward facing orientation or the rearward facing orientation.

8. The vehicle occupant viewing system of claim 7, wherein the field of view of the vehicle interior camera includes a left side perspective and a right side perspective.

9. The vehicle occupant viewing system of claim 8, wherein when the vehicle interior camera is in the forward facing orientation, the occupant image is displayed on the left side of the image display unit when an occupant is present in the left side perspective of the field of view of the vehicle interior camera and the occupant image is displayed on the right side of the image display unit when an occupant is present in the right side perspective of the field of view of the vehicle interior camera.

10. The vehicle occupant viewing system of claim 8, wherein when the vehicle interior camera is in the rearward facing orientation, the occupant image is displayed on the left side of the image display unit when an occupant is present in the right side perspective of the field of view of the vehicle interior camera and the occupant image is displayed on the right side of the image display unit when an occupant is present in the left side perspective of the field of view of the vehicle interior camera.

11. The vehicle occupant viewing system of claim 1, wherein the vehicle interior cabin has three rows of seating.

12. The vehicle occupant viewing system of claim 1, wherein the video processing unit is configured to preference occupants in a rear facing position over occupants in a forward facing position and occupants seated in child safety seats over occupants not seated in child safety seats.

13. A vehicle occupant viewing system, comprising:

an image display unit;

a first vehicle interior camera having a first field of view;

a second vehicle interior camera having a second field of view, wherein the first field of view of the first vehicle interior camera and the second field of view of the second vehicle interior camera at least partially overlap; and

a video processing unit coupled to the image display unit and the first and second vehicle interior cameras, the video processing unit being configured to receive input from the first and second vehicle interior cameras, determine whether an occupant is present in the first field of view of the first vehicle interior camera, the second field of view of the second vehicle interior camera, or both the first field of view of the first vehicle interior camera and the second field of view of the second vehicle interior camera, and display an occupant image on the image display unit in accordance with the occupant'"'"'s position in the vehicle, wherein the video processing unit is further configured to determine if the occupant is sitting in a forward facing position or a rearward facing position so a rearward facing occupant can be viewed with the first vehicle interior camera or the second vehicle interior camera depending on the orientation of the first vehicle interior camera or the second vehicle interior camera.

14. The vehicle occupant viewing system of claim 13, wherein the first vehicle interior camera is oriented in a rear facing orientation toward a rear end of the vehicle and when the occupant is sitting in a forward facing position, the occupant image is obtained from the input of the first vehicle interior camera and displayed on a side of the image display unit that coincides with a side of the first field of view.

15. The vehicle occupant viewing system of claim 13, wherein the second vehicle interior camera is oriented in a forward facing orientation toward a front end of the vehicle and when the occupant is sitting in a rearward facing position, the occupant image is obtained from the input of the second vehicle interior camera and displayed on a side of the image display unit that is opposite from a side of the second field of view.

16. The vehicle occupant viewing system of claim 13, wherein the video processing unit is configured to extract video data from an occupant region of interest to form the occupant image.

17. The vehicle occupant viewing system of claim 16, wherein the occupant image is a video stream and the video processing unit is configured to overlay the occupant image onto another image or video being shown in the image display unit.

18. The vehicle occupant viewing system of claim 13, wherein the image display unit is a touch screen and a user can X-out one or more occupant images being displayed on the image display unit.

19. A vehicle occupant viewing system, comprising:

an image display unit;

a plurality of stationary vehicle interior camera docking stations located in an interior cabin of the vehicle;

a modular vehicle interior camera configured to mount in one of the stationary vehicle interior camera docking stations and configured to be movable between the plurality of stationary vehicle interior docking stations such that the modular vehicle interior camera is configured to be mounted to any one of the plurality of stationary vehicle interior docking stations; and

a video processing unit coupled to the image display unit and the plurality of stationary vehicle interior camera docking stations, the video processing unit being configured to receive input from the modular vehicle interior camera, determine whether an occupant is present in the field of view of the vehicle interior camera, and when an occupant is present in the field of view, display an occupant image on the image display unit in accordance with the occupant'"'"'s position in the vehicle or the modular vehicle interior camera'"'"'s mounting orientation.

20. The vehicle occupant viewing system of claim 19, wherein the modular vehicle interior camera has a unique identifier and the modular vehicle interior camera is configured to identify itself to the video processing unit wirelessly via the unique identifier.

21. A method of displaying a vehicle occupant on an image display unit, comprising the steps of:

receiving input from a vehicle interior camera, the vehicle interior camera having a field of view at least partially covering an interior cabin of the vehicle;

determining whether an occupant is present in the vehicle interior camera field of view; and

when an occupant is present in the vehicle interior camera field of view, displaying an occupant image on the image display unit in accordance with the vehicle interior camera'"'"'s orientation and the occupant'"'"'s position in the vehicle and inverting a display orientation of the occupant image on the image display unit when the occupant image is of an occupant in a rear facing position and the vehicle interior camera is in a forward facing orientation.

22. The method of claim 21, further comprising the step of extracting video data from an occupant region of interest to form the occupant image.

23. The method of claim 21, further comprising the step of dewarping the input from the vehicle interior camera to correct for a lens distortion.