VEHICLE NAVIGATION METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS

• US 20190023413A1
• Filed: 09/11/2018
• Published: 01/24/2019
• Est. Priority Date: 03/12/2015
• Status: Active Grant

• Current Assignee: Vu Systems, LLC
• Original Assignee: Vu Systems, LLC
• Inventors: CONNOR, Sidney A., STEVENS, J. Stedman
• Granted Patent: US 10,315,776 B2
• Time in Patent Office: Days
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

Vehicle navigation methods, systems and computer program products are provided that create a composite video image of a scene from video images of the scene generated by a plurality of video image sensors. The video image from each image sensor includes a respective array of pixels. The composite video image is created by selecting for a composite image pixel at a given position in an array of composite image pixels a pixel at the given position from one of the respective pixel arrays having the highest signal level or highest signal-to-noise ratio. The selecting is repeatedly performed for a plurality of given positions in the array of composite image pixels. The composite video image can be displayed via a display, such as a head-up display utilized in a vehicle.

Claims

1. A vehicle navigation method, comprising:

receiving from a plurality of video image sensors associated with the vehicle a respective plurality of video images of a scene, wherein each video image comprises a respective array of pixels;

creating a composite video image of the scene from the plurality of video images, wherein the composite video image includes an array of composite image pixels, and wherein each composite image pixel in the array of composite image pixels is selected from a respective pixel in one of the respective array of pixels that has a highest signal level or highest signal-to-noise ratio; and

displaying on a display the composite video image in real time so that the vehicle can be navigated in real time using the displayed composite video image.

2. The vehicle navigation method of claim 1, wherein at least one of the plurality of video image sensors is mounted on the vehicle.

3. The vehicle navigation method of claim 1, wherein the plurality of video image sensors are selected from visible light image sensors, non-visible light image sensors, radio-frequency (RF) image sensors, sound navigation and ranging (SONAR) image sensors, light detection and ranging (LIDAR) image sensors, and Doppler image sensors.

4. The vehicle navigation method of claim 1, wherein the plurality of video image sensors are of respective different technology types.

5. The vehicle navigation method of claim 1, further comprising changing one or more composite image pixels in the displayed composite video image responsive to user input and/or responsive to user preference.

6. A method of creating a composite video image that includes an array of composite image pixels, the method comprising:

receiving from a plurality of video image sensors a respective plurality of video images of a scene, wherein each video image comprises a respective array of pixels;

selecting for a composite image pixel at a given position in the array of composite image pixels a pixel at the given position in one of the respective array of pixels that has a highest signal level or highest signal-to-noise ratio.

7. The method of claim 6, wherein the selecting is repeatedly performed for a plurality of given positions in the array of composite image pixels.

8. The method of claim 6, comprising displaying the composite video image.

9. The method of claim 6, wherein at least one of the plurality of video image sensors is mounted on a vehicle.

10. The method of claim 8, comprising displaying the composite video image via a head-up display.

11. The method of claim 6, wherein the composite video image has a higher object resolution than at least one of the plurality of video images.

12. The method of claim 6, wherein the plurality of video image sensors are selected from visible light image sensors, non-visible light image sensors, radio-frequency (RF) image sensors, sound navigation and ranging (SONAR) image sensors, light detection and ranging (LIDAR) image sensors, and Doppler image sensors.

13. The method of claim 6, wherein the plurality of video image sensors are of respective different technology types.

14. The method of claim 6, further comprising changing one or more composite image pixels responsive to user input and/or responsive to user preference.

15. An imaging system, comprising:

first and second video image sensors configured to generate respective first and second video images of a scene, wherein the first and second video images comprise respective first and second arrays of pixels;

an image processor configured to receive the first and second video images and create a composite video image that includes an array of composite image pixels,

wherein, for a given pixel position in the composite pixel array, the image processor selects a pixel at the given position from one of the first and second arrays of pixels that has a highest signal level or highest signal-to-noise ratio.

16. The imaging system of claim 15, comprising a display configured to display the composite video image.

17. The imaging system of claim 16, further comprising a user control associated with the image processor that allows a user to change composite image pixels displayed within the display.

18. The imaging system of claim 16, wherein the image processor is configured to change composite image pixels responsive to a stored user preference.

19. The imaging system of claim 16, wherein the display comprises a head-up display.

20. The imaging system of claim 15, wherein at least one of the plurality of video image sensors is mounted on a vehicle.

21. The imaging system of claim 15, wherein the composite video image has a higher object resolution than at least one of the first and second video images.

22. The imaging system of claim 15, wherein the first and second video image sensors are selected from visible light image sensors, non-visible light image sensors, radio-frequency (RF) image sensors, sound navigation and ranging (SONAR) image sensors, light detection and ranging (LIDAR) image sensors, and Doppler image sensors.

23. The imaging system of claim 15, wherein the first and second video image sensors are of different technologies.

24. A computer program product for creating a composite video image that includes an array of composite image pixels, the computer program product comprising a non-transitory computer readable storage medium having encoded thereon instructions that, when executed on a processor, cause the processor to perform operations comprising:

receiving from a plurality of video image sensors a respective plurality of video images of a scene, wherein each video image comprises a respective array of pixels; and

selecting for a composite image pixel at a given position in the array of composite image pixels a pixel at the given position in one of the respective array of pixels that has a highest signal level or highest signal-to-noise ratio.

25. The computer program product of claim 24, wherein the computer readable storage medium has encoded thereon instructions that, when executed on the processor, cause the processor to repeatedly perform the selecting for a plurality of given positions in the array of composite image pixels.

26. The computer program product of claim 24, wherein the computer readable storage medium has encoded thereon instructions that, when executed on the processor, cause the processor to display the composite image.

27. A method for creating a video image, comprising:

receiving a video image from each of a plurality of video image sensors, wherein at least one of the plurality of video image sensors is mounted to a vehicle;

receiving ground data that is at a resolution that is higher than the resolution of at least one of the video image sensors;

using the ground data to determine expected image features; and

combining the plurality of video images based on at least one of a maximum information content, adherence to the expected image features, and image quality metrics.