Vehicle monitoring device and method of monitoring vehicle

• US 9,842,261 B2
• Filed: 04/17/2012
• Issued: 12/12/2017
• Est. Priority Date: 05/24/2011
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Takano, Teruhisa, Yasuhara, Shinji
• Primary Examiner(s): Beasley, Deirdre
• Application Number:
  US14/119,324
• Publication Number:
  US 20140085477A1
• Time in Patent Office: 2,065 Days
• Field of Search: 348148
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A vehicle monitoring system is provided which determines the mode of display and the compressibility of a moving image, a streaming video and a still image based on a monitoring level determined by vehicle conditions detected. The system generates a monitoring image using the determined display mode or compressed at the determined compressibility and transmits the monitoring image to an external terminal device.

Claims

1. A vehicle monitoring device comprising:

a monitoring level determining mechanism that determines a monitoring level of a vehicle,

based in part on a state of the vehicle as determined from vehicle state detection sensors, on receipt of a request for a monitoring image, on a presence of a monitoring target object in a captured still image, and on a state of the monitoring target object in the captured still image as compared to a location, a movement, and an activity of the monitoring target object in previously captured still images, wherein the captured still image and the previously captured still images are received from a camera directed toward a periphery of the vehicle;

a monitoring image generating mechanism that determines a display mode based on the monitoring level to generate the monitoring image in the display mode using the captured still image and the previously captured still images,

wherein the display mode is one of a moving image, a streaming image, and a still image, and

wherein the monitoring image generating mechanism generates a monitoring image that is a moving image or a streaming image by successive sequencing of the captured still image and the previously captured still images; and

a transmission mechanism that transmits information containing the monitoring image to an external terminal device via a communication line, wherein

when receiving the request for the monitoring image from the external terminal device, the monitoring level determining mechanism determines the monitoring level to be a standard level, and

when the monitoring level is determined to be the standard level, the monitoring image generating mechanism generates the monitoring image in the display mode of the still image.

2. The vehicle monitoring device as claimed in claim 1, wherein

the monitoring level determining mechanism includes protocol to assess at least one of the following:

orientation of the monitoring target object or a time the monitoring target object stays in a vicinity of the vehicle, wherein the monitoring level determining mechanism determines at least one of the following;

a monitoring target object approaching the vehicle detected from the captured still image, a monitoring target object remaining in the vicinity of the vehicle for a predetermined time or more, and

the monitoring image generating mechanism generates, in accordance with an enhanced level, the monitoring image in the display mode of either the moving image or the streaming image with a number of frames per unit time at a predetermined value or more.

3. The vehicle monitoring device as claimed in claim 1, wherein

when receiving an abnormality occurrence signal from the vehicle, the monitoring level determining mechanism determines the monitoring level to be an enhanced level, and

the monitoring image generating mechanism generates, in accordance with the enhanced level, the monitoring image in the display mode of either the moving image or the streaming image with a number of frames per unit time at a predetermined value or more.

4. The vehicle monitoring device as claimed in claim 1, wherein the monitoring image generating mechanism generates the monitoring image at a corrected frame rate corrected to increase a number of frames per unit time in accordance with the monitoring level when the monitoring level is detected by the monitoring level determining mechanism after sunset.

5. The vehicle monitoring device as claimed in claim 1, wherein, based on a moving speed of a monitoring target object of the captured still image captured by the camera, the monitoring image generating mechanism generates the monitoring image at a corrected frame rate correcting a number of frames per unit time in accordance with the monitoring level.

6. The vehicle monitoring device in claim 1, wherein

when the monitoring level is in an enhanced level, the monitoring image generating mechanism sets a frequency at which the monitoring image is generated high, and

the transmission mechanism transmits the monitoring image to the external terminal device in accordance with the frequency.

23. The vehicle monitoring device as claimed in claim 1, wherein, when a detected monitoring target object is separating from the vehicle or is a stationary object, the monitoring level determining mechanism determines the vehicle monitoring level to be the standard level and the monitoring image generating mechanism generates the monitoring image in the display mode of the still image.

25. The vehicle monitoring device of claim 1 wherein the monitoring level determining mechanism is operatively connected to a plurality of motion sensors positioned on one or more exterior locations of the vehicle and monitoring an exterior periphery surrounding the vehicle.

26. The vehicle monitoring device of claim 1, wherein when the presence of a monitoring target object is detected from the monitoring image captured by the camera, the monitoring level determining mechanism determines that the monitoring level should be elevated to be an enhanced level, and the monitoring image generating mechanism generates the monitoring image in the display mode of either the moving image or the streaming image with a number of frames per unit time at a predetermined value or greater in accordance with the enhanced level.

7. A vehicle monitoring device comprising:

means for determining a vehicle monitoring level based on a state of the vehicle or a periphery of the vehicle, wherein the monitoring level is determined from a plurality of monitoring levels;

means for generating a monitoring image based on a captured still image captured by a camera mounted on the vehicle at a frame rate in accordance with the monitoring level, wherein the camera is oriented outward toward the periphery, wherein the means for generating a monitoring image generates the monitoring image by successive sequencing of one or more of captured still images; and

means for transmitting information containing the monitoring image via a communication line to an external terminal device, wherein

when receiving a request for the monitoring image from the external terminal device, the monitoring level determination means determines the vehicle monitoring level to be a standard level, and

when the monitoring level is determined to be the standard level, the monitoring image generating mechanism generates the monitoring image in a display mode of a still image and wherein means for monitoring generating generates the monitoring image at a rate greater than a still image when a target object is identified in the still image.

8. A vehicle monitoring method comprising:

receiving a captured still image from a camera mounted on the vehicle and directed to a periphery of the vehicle;

determining a monitoring level of the vehicle based in part on a state of the vehicle as determined based on vehicle state detection sensors, on presence of a monitoring target object in the captured still image, and on a state of the monitoring target object in the captured still image as compared to the state of the monitoring target object in previously captured still images received from the camera;

generating a monitoring image in a display mode of one of a moving image, a streaming video, and a still image based on the captured still image in accordance with the monitoring level;

transmitting generated information including the monitoring image to an external terminal device using a communication line; and

when a request is received for the monitoring image from the external terminal device and the monitoring level is determined to be a standard level, generating the monitoring image in the display mode of the still image.

9. A vehicle monitoring device comprising:

a monitoring level determining mechanism that determines a monitoring level of a vehicle based on a detected vehicle condition by detecting a state of the vehicle and of a surroundings of the vehicle, wherein the monitoring level determining mechanism determines the monitoring level from a plurality of monitoring levels;

a monitoring image generating mechanism that calculates a compressibility in accordance with the monitoring level and generates a monitoring image compressing a captured still image captured by a camera mounted on the vehicle and directed to the surroundings of the vehicle at the compressibility calculated, wherein the monitoring image generating mechanism generates a monitoring image by successive sequencing of the captured still image and previously captured still images; and

a transmission mechanism that transmits information including generated monitoring image to an external terminal device, wherein

when receiving a request for the monitoring image from the external terminal device, the monitoring level determining mechanism determines the monitoring level to be a standard level, and

the monitoring image generating mechanism generates, in accordance with the standard level, the monitoring image by compressing the captured still image captured by the camera at a compressibility higher than a previously set standard compressibility;

wherein the monitoring level determining mechanism calculates a lower compressibility as the monitoring level increases; and

wherein the monitoring level determining mechanism determines the monitoring level of the vehicle as an enhanced level when a presence of a monitoring target object is detected from the captured still image captured by the camera; and

the monitoring image generating mechanism generates the monitoring image by compressing the captured still image captured by the camera at a compressibility lower than the previously set standard compressibility.

10. The vehicle monitoring device as claimed in claim 9, wherein

the monitoring level determining mechanism determines the monitoring level of the vehicle to be an enhanced level when a monitoring target object is detected that approaches the vehicle based on the captured still image captured by the camera;

and

the monitoring image generating mechanism generates the monitoring image by compressing the captured still image captured by the camera at a compressibility lower than the previously set standard compressibility.

11. The vehicle monitoring device as claimed in claim 9, wherein

when a monitoring target object that stays in a vicinity of the vehicle for a predetermined time or more is detected, the monitoring level determining mechanism determines the vehicle monitoring level to be an enhanced level;

and

the monitoring image generating mechanism generates the monitoring image by compressing the captured still image captured by the camera in accordance with the enhanced level at a compressibility lower than the previously set standard compressibility.

12. The vehicle monitoring device as claimed in claim 9, wherein

if an abnormality signal from the vehicle is obtained, the monitoring level determining mechanism determines the vehicle monitoring level to be an enhanced level;

and

the monitoring image generating mechanism generates the monitoring image by compressing the captured still image captured by the camera in accordance with the enhanced level at a compressibility lower than the previously set standard compressibility.

13. The vehicle monitoring device as claimed in claim 9, wherein the monitoring image generating mechanism generates the monitoring image at a corrected compressibility that is set low in accordance with the monitoring level when the monitoring level is detected by the monitoring level determining mechanism after sunset.

14. The vehicle monitoring device as claimed in claim 9, wherein, when a monitoring target object is included in the captured still image captured by the camera, the monitoring image generating mechanism generates the monitoring image compressing the captured still image containing the monitoring target object at a corrected compressibility that is set lower in accordance with the monitoring level.

15. The vehicle monitoring device as claimed in claim 9, wherein

the monitoring image generating mechanism increases a frequency of generating the monitoring image when the monitoring level is determined to be an enhanced level;

and

the transmission mechanism transmits the monitoring image to the external terminal device in accordance with the frequency.

24. The vehicle monitoring device as claimed in claim 9, wherein, when a detected monitoring target object is separating from the vehicle, the monitoring level determining mechanism determines the monitoring level to be a standard level, and

in accordance with the standard level, the monitoring image generating mechanism generates the monitoring image by compressing the captured still image captured by the camera at the previously set standard compressibility or higher.

16. A vehicle monitoring method comprising:

determining a monitoring level of a vehicle based on a vehicle condition by detecting a state of the vehicle or of a surroundings of the vehicle, wherein the state of the vehicle or of the surroundings of the vehicle is one of request for monitoring image, still object, monitoring object separating, monitoring object approaching without motion, monitoring object approaching with motion, monitoring object staying, and abnormality detected by vehicle state detection sensors;

calculating a compressibility based on the monitoring level;

generating a monitoring image compressing a captured still image captured by a camera mounted on the vehicle by the compressibility calculated, wherein the monitoring image is generated by successive sequencing of the captured still image and previously captured still images;

transmitting information including the monitoring image generated to an external terminal device; and

when the monitoring level is determined to be a standard level, generating the monitoring image by compressing the captured still image captured by the camera at a compressibility higher than a previously set standard compressibility.

17. A vehicle monitoring device comprising:

a plurality cameras mounted on a vehicle exterior for capturing still images of a periphery of the vehicle;

a monitoring image generating mechanism that, when presence of a monitoring target object is detected in the periphery of the vehicle, generates a moving image indicating a state change in the periphery of the vehicle as a monitoring image based on a captured still image including the image corresponding to the monitoring target object, wherein the moving image is generated by successive sequencing of the captured still image and previously captured still images; and

a transmission mechanism that transmits the monitoring image to an external terminal device using a communication line, wherein, when the monitoring target object is not detected in the periphery of the vehicle, monitoring image generating mechanism generates a still image showing a temporary state of a vicinity of the vehicle based on the captured still image captured by the camera.

18. The vehicle monitoring device as claimed in claim 17, wherein

the monitoring image generating mechanism compares a reference monitoring image acquired at a reference monitor time representing a reference when monitoring the surroundings of the vehicle and a comparison image captured by the camera in a single or plurality of monitoring timings after the reference monitoring time to generate the monitoring image based on a difference image in response to a change in image data between the reference monitoring image and the comparison image;

and

the transmission mechanism transmits the monitoring image to the external device via the communication line after having sent the reference monitoring image to the external device via the communication line.

19. The vehicle monitoring device as claimed in claim 17, wherein the monitoring image generating mechanism:

compares a K-th comparison image (wherein K is a natural number) captured at a K-th monitoring timing after a reference monitoring timing and the reference monitoring image to extract a K-th difference image of a predetermined region containing the difference image of the two images;

compares the K-th comparison image captured at the K-th monitoring timing after the reference monitoring timing and a (K−

1)-th comparison image captured at a (K−

1)-th timing immediately before the K-th monitoring timing;

attaches to the image data of invariable region having no difference over the reference monitoring image of the K-th difference image a first identifier;

attaches to a duplicate change region included in the (K−

1) comparison image as well of the K-th difference image a second identifier;

attaches to a most receipt change region included only in the (K−

1)-th difference image of the K-th difference image a third identifier; and

replaces the image data of the image region of the K-th difference image to which the third identifier is attached with the image data of the K-th comparison image corresponding to the image region; and

generates the information including the image data of the K-th comparison image replaced, the first identifier and the second identifier as the monitoring image.

20. The vehicle monitoring device as claimed in claim 17, wherein, when a moving speed of the monitoring target object calculated based on the captured still image is less than a predetermined value, the monitoring image is generated by decreasing a frame rate.

21. A vehicle monitoring system using the vehicle monitoring device as set forth in claim 17, further comprising:

a terminal device capable of information transfer to and from the vehicle monitoring device, wherein the terminal device comprises;

a communication mechanism to communicate with the vehicle monitoring device;

a display image generating mechanism to generate an image for display based on the monitored image acquired through a communication means; and

a display mechanism for displaying the image for display generated by the display image generating mechanism.

22. A vehicle monitoring method comprising:

acquiring a plurality of captured still images of a surroundings of a vehicle, the images each associated with a discrete region of the surroundings of the vehicle;

when an image corresponding to a monitoring target object has been detected in one of the captured still images, generating a moving image as a monitoring image for the associated region that indicates a state of change in the surroundings of the vehicle based on the captured still image including a monitoring target object, while images associated with discrete regions of the surroundings lacking a monitoring target object are generated as still images, wherein the moving image is generated by successive sequencing of the captured still image and the plurality of captured still images;

when the monitoring target object has not been detected in at least one of the captured still image, generating a still image showing a temporary state of the surroundings of the vehicle based on the captured still image; and

transmitting the monitoring images to an external terminal device, such that the plurality of captured still images depicts the surroundings of the vehicle.