SYSTEM AND METHOD OF VEHICLE SENSOR MANAGEMENT

• US 20160325680A1
• Filed: 05/04/2016
• Published: 11/10/2016
• Est. Priority Date: 05/04/2015
• Status: Abandoned Application

• Current Assignee: Pearl Automation Inc.
• Original Assignee: Kamama, Inc.
• Inventors: Curtis, Robert, Vora, Saket, Sander, Brian, Fisher, Joseph, Gardner, Bryson, Mincey, Tyler, Du Bois, Ryan, Kocalar, Erturk, Shoemaker, David, Fino, Jorge
• Application Number:
  US15/146,705
• Publication Number:
  US 20160325680A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/50 characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller
• CPC Class Codes: G06T 2210/22 Cropping
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

A method for vehicle sensor management including: acquiring sensor measurements at a sensor module; transmitting the sensor measurements from the sensor module; processing the sensor measurements; and transmitting the processed sensor measurements to a client associated with the vehicle, wherein the processed sensor measurements are rendered by the client on the user device.

Claims

1. A method of operating a vehicle sensor management system, the system including:

an imaging system configured to removably mount to a vehicle exterior, a processing system configured to removably connect to a data bus of the vehicle, and a client configured to run on a user device, the method comprising;

acquiring a raw video stream at the imaging system;

processing the raw video stream into a user stream and an analysis stream at the imaging system;

transmitting the analysis stream and the user stream from the imaging system to the processing system;

transmitting the user stream from the processing system to the client;

determining an ambient environment parameter for the vehicle, based on the analysis stream, at the processing system;

generating a notification based on the ambient environment parameter at the processing system;

transmitting the notification from the processing system to the client;

generating a composite video stream by overlaying graphics associated with the notification over the user stream; and

displaying the composite video stream at the client.

2. The method of claim 1, further comprising:

powering the imaging system using power from a first secondary battery, wherein the imaging system comprises the first secondary battery;

powering the processing system with power from the vehicle; and

powering the user device with power from a second secondary battery, wherein the user device comprises the second secondary battery.

3. The method of claim 1, wherein the analysis stream, user stream, and notification are transmitted between the imaging system, processing system, and client through a high bandwidth wireless communication network created by the processing system.

4. The method of claim 3, further comprising transmitting control instructions between the client, processing system, and the imaging system using a low bandwidth wireless communication protocol.

5. The method of claim 1, further comprising:

operating the imaging system in a low-power mode;

generating, at the processing system, a streaming control signal in response to occurrence of a streaming event;

transmitting the streaming control signal from the processing system to the imaging system;

operating the imaging system in a high-power mode in response to receipt of the streaming control signal, prior to acquiring the raw video stream;

generating, at the processing system, a termination control signal in response to occurrence a termination event;

transmitting the termination control signal from the processing system to the imaging system; and

operating the imaging system in the low-power mode in response to receipt of the termination control signal.

6. The method of claim 5, wherein the initialization event comprises wireless connection of the user device to the processing system, wherein the termination event comprises receiving data indicative of vehicle parking gear engagement at the processing system from the data bus.

7. The method of claim 1, wherein identifying the ambient environment parameter comprises identifying an object from the analysis stream, wherein the notification is determined based on a position of the object within a video frame.

8. The method of claim 1, wherein processing the raw video stream further comprises generating a cropped video stream, the cropped video stream comprising a retained section of each video frame of the video stream, the retained section defined by a set of cropping dimensions and a first set of orientation pixel coordinates, wherein the user stream comprises the cropped video stream.

9. The method of claim 8, further comprising:

receiving a user input at an input region defined by the client, the user input indicative of moving a camera field of view;

generating new cropping instructions based on the user input at the client, the new cropping instructions comprising a second set of orientation pixel coordinates different from the first set of orientation pixel coordinates; and

transmitting the new cropping instructions to the imaging system.

10. The method of claim 9, further comprising, at the client:

generating and displaying a second user stream based on the cropped video stream and the analysis stream until occurrence of a transition event; and

in response to occurrence of the transition event, displaying a second cropped video stream, generated by the imaging system and received from the processing system.

11. The method of claim 1, further comprising:

determining a software update at a remote server system;

transmitting a data packet, based on the software update, to the client from the remote server system;

in response to client connection with the processing system, transmitting the data packet to the processing system; and

updating the processing system based on the data packet.

12. The method of claim 11, wherein the data packet comprises the software update.

13. The method of claim 1, wherein the notification is generated from a first subset of video frames of the analysis stream, wherein the graphics are composited with video frames of the user stream generated from a second subset of video frames of the analysis stream, the second subset of video frames different from the first subset of video frames.

14. A vehicular guidance method using a guidance system including:

an imaging system configured to removably mount to a vehicle exterior, the method comprising;

at the imaging system, concurrently recording a first and second raw video stream;

at the imaging system, processing the first raw video stream into a user stream;

transmitting the user stream to a client running on a user device;

determining an ambient environment parameter based on the first and second raw video streams;

generating a notification based on the ambient environment parameter;

transmitting the notification to the client;

at the client, generating a composite video stream by overlaying graphics associated with the notification over the user stream; and

presenting the composite video stream with the client on a display region of the user device.

15. The method of claim 14, wherein the notification is generated based on a first subset of video frames of the first raw video stream, wherein the graphics are composited with video frames of the user stream generated based on a second subset of video frames of the first raw video stream, the second subset of video frames different from the first subset of video frames.

16. The method of claim 14, wherein the user stream is transmitted over a high-bandwidth wireless communication network, wherein the client and imaging system are connected to the high-bandwidth wireless communication network.

17. The method of claim 14, further comprising processing the first and second raw video streams into a first and second analysis stream, respectively;

and transmitting the first and second analysis stream to a processing system, wherein the processing system determines the ambient environment parameter based on the first and second raw video streams, generates the notification, and transmits the notification to the client.

18. The method of claim 17, wherein the processing system is housed in a separate housing from the imaging system and user device, the processing system configured to removably mount to a vehicle interior.

19. The method of claim 14, further comprising:

transmitting a first analysis stream, generated from the first raw video stream, to the user device;

at the user device;

generating a second composite video stream by aligning the user stream with the first analysis stream;

receiving a user input at an input region on the user device, the input region overlaying the display region, the user input comprising an input direction;

in response to receipt of the user input, determining an adjusted user video stream from the composite stream, the adjusted user video stream comprising a section of the second composite stream, shifted relative to the user video stream, along a direction opposing the input direction;

in response to receipt of the user input, sending user stream instructions, determined based on the user input, to the imaging system;

receiving and storing the user stream instructions at the imaging system;

concurrently recording a third and fourth raw video stream at the imaging system, the third and fourth raw video stream recorded after first and second raw video stream recordation;

processing the third raw video stream into a second user stream according to the user stream instructions at the imaging system; and

transmitting the second user stream to the client, wherein the client displays the second user stream at the display region.

20. The method of claim 14, further comprising:

recording imaging system operation parameters at the imaging system;

transmitting imaging system operation parameters to the client; and

transmitting the imaging system operation parameters to a remote server system from the client.