SYSTEM AND METHOD FOR MULTIMEDIA CAPTURE

• US 20170251163A1
• Filed: 02/29/2016
• Published: 08/31/2017
• Est. Priority Date: 02/29/2016
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Ochiai, Yuichi, Nagata, Katsumi, Sasaki, Akira
• Granted Patent: US 9,918,038 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253
• CPC Class Codes: H04N 9/8211 the additional signal being a sound signal H04N9/835 takes precedence
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal

Abstract

Methods, systems, and apparatus for multimedia capture. The system includes one or more storage devices connected through a network to a vehicle. The vehicle includes a memory for buffering video. The vehicle includes multiple cameras coupled to one or more video processors. The one or more video processors are configured to capture a first video of an interior view of the vehicle for a period of time, capture a second video of an exterior view of surroundings of the vehicle for the period of time, and synchronize the first video and the second video. The vehicle includes one or more microphones coupled to one or more audio processors for capturing audio and an electronic control unit. The electronic control unit includes a multimedia processor. The electronic control unit is configured to buffer the synchronized video, detect a triggering event, and store the synchronized video.

Claims

1. A multimedia capture system for a vehicle, comprising:

one or more storage devices connected through a network to the vehicle, the vehicle including;

a memory for buffering video and coupled to one or more video processors;

a plurality of cameras including a first camera and a second camera and coupled to one or more video processors, the one or more video processors configured to;

capture a first video of an interior view of the vehicle for a period of time;

capture a second video of an exterior view of surroundings of the vehicle for the period of time;

synchronize the first video and the second video;

one or more microphones coupled to one or more audio processors for capturing audio within the vehicle;

an electronic control unit including a multimedia processor, the multimedia processor including the one or more video processors and the one or more audio processors, the electronic control unit configured to;

buffer, in the memory, the synchronized video;

detect a triggering event; and

store the synchronized video based on the triggering event.

2. The multimedia capture system of claim 1, wherein the one or more audio processors is configured to:

capture audio from the interior of the vehicle, wherein the audio includes one or more audio expressions; and

measure a frequency level of the one or more audio expressions.

3. The multimedia capture system of claim 2, wherein to detect the triggering event, the electronic control unit is configured to:

determine that the triggering event occurred if the frequency level of the one or more audio expressions is within a particular range, wherein the particular range is user configurable or pre-determined.

4. The multimedia capture system of claim 3, wherein the multimedia processor is configured to synchronize the audio with the synchronized video.

5. The multimedia capture system of claim 1, wherein the electronic control unit is further configured to store the synchronized video between a start time and a finish time, the start time being at or before the triggering event and the finish time being a threshold amount of time after the triggering event.

6. The multimedia capture system of claim 5, wherein at least one of the start time, the finish time, or the threshold amount of time after the triggering event is a user configurable setting.

7. The multimedia capture system of claim 1, wherein the triggering event is a selection of a user interface element on a user interface.

8. The multimedia capture system of claim 1, the vehicle further comprising:

a navigation unit coupled to the electronic control unit for obtaining vehicle information including a current date and time and a current location of the vehicle;

wherein the electronic control unit is further configured to;

obtain, from the navigation unit, the vehicle information;

identify, one or more landmarks being captured in the second video based on the vehicle information; and

tag, by the electronic control unit, the one or more landmarks in the synchronized video.

9. The multimedia capture system of claim 1, further comprising:

one or more external databases connected to the vehicle through the network, wherein the one or more external databases includes landmark information,

wherein the electronic control unit is further configured to;

obtain, from the one or more external databases, the landmark information;

identifying, one or more landmarks being captured in the second video based on the landmark information; and

tag, by the electronic control unit, the one or more landmarks in the synchronized video.

10. The multimedia capture system of claim 1, wherein to store the synchronized video comprises:

accessing the one or more storage devices connected to the vehicle through the network; and

transmitting synchronized video to the one or more storage devices connected to the vehicle through the network.

11. A computer-implemented method for a road trip album for a vehicle, the method comprising:

capturing, in one or more video processors and using a first camera, a first video of an interior view of the vehicle for a period of time;

capturing, in the one or more video processors and using a second camera, a second video of an exterior view of surrounding of the vehicle for the period of time;

synchronizing, using the one or more video processors, the first video and the second video;

buffering, in a memory, the synchronized video;

detecting a triggering event; and

storing the synchronized video.

12. The computer-implemented method of claim 11, further comprising:

obtaining vehicle information including a current date, location, and elevation of the vehicle;

identifying, by an electronic control unit and using the navigation unit or the second camera, one or more landmarks being captured in the second video; and

tagging, by the electronic control unit, the one or more landmarks in the synchronized video.

13. The computer-implemented method of claim 12, wherein storing the synchronized video includes the tagged one or more landmarks, the vehicle information and one or more identities of occupants of the vehicle.

14. The computer-implemented method of claim 11, wherein the triggering event is user input that activates storing of the combined video.

15. The computer-implemented method of claim 11, wherein detecting the triggering event comprises:

determining, using one or more sensors, an audio expression for an occupant of the vehicle; and

determining that the audio expression has a frequency that is within a particular range.

16. The computer-implemented method of claim 11, further comprising:

capturing, in one or more audio processors and using a first microphone, an audio recording of an interior of the vehicle for the period of time;

synchronizing, in a multimedia processor, the captured audio recording and the synchronized video; and

storing the captured audio with the synchronized video.

17. The computer-implemented method of claim 11, further comprising:

obtaining, from a user interface, user input, wherein the user input is configuration settings for the triggering event.

18. The computer-implemented method of claim 17, wherein detecting the triggering event comprises:

capturing, using the first camera, one or more facial features of an occupant;

determining one or more facial expressions of an occupant of the vehicle based on the one or more facial features; and

determining that the triggering event occurred based on the one or more determined facial expressions and the configuration settings for the triggering event.

19. A multimedia capture system for an autonomous vehicle, comprising:

one or more storage devices connected through a network to the autonomous vehicle, the autonomous vehicle including;

a memory for buffering video and coupled to one or more video processors;

a plurality of cameras including a first camera and a second camera and coupled to one or more video processors, the one or more video processors configured to;

capture a first video of an interior view of the autonomous vehicle for a period of time;

capture a second video of an exterior view surroundings of the vehicle for the period of time;

synchronize the first video and the second video;

one or more microphones coupled to one or more audio processors for capturing audio within the autonomous vehicle;

an electronic control unit including a multimedia processor, the multimedia processor including the video processors and the one or more audio processors, the electronic control unit configured to;

buffer, in a memory, the synchronized video;

detect a triggering event; and

store the synchronized video based on the triggering event.

20. The multimedia capture system of claim 19, wherein the autonomous vehicle operates using one or more sensors or the navigation unit and without human input.