Systems and methods for image stream processing

• US 8,713,215 B2
• Filed: 09/10/2012
• Issued: 04/29/2014
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Z Microsystems Incorporated
• Original Assignee: Z Microsystems Incorporated
• Inventors: Wade, Jack, Siewert, Charles, Brown, Joel
• Primary Examiner(s): SUN, SCOTT C
• Application Number:
  US13/609,133
• Publication Number:
  US 20130073775A1
• Time in Patent Office: 596 Days
• Field of Search: None
• US Class Current: 710/30
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Various embodiments relate to systems and methods for simultaneously switching input image streams to output devices, while providing optional image processing functions on the image streams. Certain embodiments may provide vision systems and methods suitable for use in vehicles, particularly windowless vehicles, such as armored ground vehicles, submerged watercraft, and spacecraft. Some embodiments may enable sharing of image streams (e.g., with one or more other vehicles), generation of panoramic views (e.g., from various camera feeds), intelligent encoding of image streams, and implementation of security features based on image streams.

Claims

1. A system for image stream processing, comprising:

an image stream input interface;

an image stream output interface;

a first image processing module configured to accept a plurality of image streams, stitch at least two image streams from the plurality of image streams into a contiguous image stream, and output the contiguous image stream, wherein the plurality of image streams comprises an image stream from the image stream input interface or from another image processing module;

a second image processing module; and

a switching matrix in communication with the image stream input interface, the image stream output interface, the first image processing module, and the second image processing module, wherein the switching matrix is configured to;

selectively map the image stream from the image stream input interface or from the second image processing module, to the first image processing module, and

selectively map the contiguous image stream from the first image processing module to the image stream output interface or to the second image processing module.

2. The system of claim 1, further comprising a plurality of image stream input interfaces, the plurality of image stream input interfaces being in communication with the switching matrix and including the image stream input interface, wherein at least two of the plurality of image streams are based on a second plurality of image streams received through the plurality of image stream input interfaces.

3. The system of claim 1, wherein the plurality of image streams are based on a second plurality of image streams from a plurality of adjacent cameras, and the contiguous image stream is a panoramic view from a vantage of the adjacent cameras.

4. The system of claim 3, further comprising a plurality of image stream input interfaces, the plurality of image stream input interfaces being in communication with the switching matrix and including the image stream input interface, wherein the second plurality of image streams are received through the plurality of image streams interfaces.

5. The system of claim 3, wherein at least one of the second plurality of image streams has been processed by the second image processing module before being accepted by the first image processing module.

6. The system of claim 1, wherein the switching matrix is further configured to adjust the selective mapping such that in real time, a particular image stream is added to or removed from the plurality of image streams being accepted by the first image processing module.

7. The system of claim 1, wherein the first image processing module performs stitching at frame-level.

8. The system of claim 1, wherein the second image processing module is configured to accept a second image stream from the image stream input interface or from another image processing module, apply an image processing function to the second image stream, and output a processed image stream.

9. The system of claim 8, wherein the image processing function comprises an image stream mix function, an image stream scale function, an image stream blend function, an image stream encoding function, an image stream enhancement function, or image stream cropping function.

10. The system of claim 9, wherein the image stream encoding function comprises a video or audio compression algorithm.

11. The system of claim 9, wherein the image stream enhancement function comprises a de-haze function, a de-blur function, a shadow function, a dawn-dusk function, a fusion function, a stabilization function, a thermal turbulence function, an equalization function, an edge detection function, a rain and fog function, or a light optimizing function.

12. The system of claim 1, wherein the image stream input interface is configured to receive the image stream from an image capture device, an image stream playback device, a computer system, a sensor device, or a medical device.

13. The system of claim 1, wherein the image stream output interface is configured to output to a display, a computer system, or recording device.

14. The system of claim 1, wherein the system is configured to output an image stream through a virtual display.

15. The system of claim 1, further comprising a control interface through which a user can operate the system, configure the system, or check status of the system.

16. A method for image stream processing, comprising:

receiving a plurality of image streams at least one of which is based on an image stream from an image stream input interface;

generating a contiguous image stream by stitching at least two image streams of the plurality of image streams into the contiguous image stream;

selectively mapping at least one image stream of the plurality of image streams to;

an image processing module, wherein the image processing module is configured to receive the at least one image stream from the switching matrix, apply an image processing function to the at least one image stream, and output a first processed image stream, or

an image stream output interface; and

selectively mapping the contiguous image stream to;

the image processing module, wherein the image processing module is further configured to receive the contiguous image stream from the switching matrix apply the image processing function to the contiguous image stream, and output a second processed image stream, or

the image stream output interface.

17. The method of claim 16, wherein the plurality of image streams originate from a plurality of adjacent cameras, and the contiguous image stream is a panoramic view from a composite perspective of the plurality of adjacent cameras.

18. The method of claim 16, further comprising adjusting the selective mapping in real time such that a particular image stream is added to or removed from the plurality of image streams being received.

19. The method of claim 16, wherein the stitching is performed at a frame-level.

20. The method of claim 16, wherein at least one of the image stream input interfaces is configured to receive at least one of the image streams from an image capture device, an image stream playback device, a computer system, a sensor device, or a medical device.

21. The method of claim 16, wherein the image stream output interface is configured to output to a display, a computer system, or recording device.

22. A computer readable storage medium having instructions embedded thereon configured to cause a processor to perform the operations of:

receiving a plurality of image streams at least one of which is based on an image stream from an image stream input interface;

generating a contiguous image stream by stitching the at least two image streams of the plurality of image streams into the contiguous image stream;

selectively mapping the at least one image stream of the plurality of image streams to;

an image processing module, wherein the image processing module is configured to receive the least one image stream from the switching matrix, apply an image processing function to the least one image stream, and output a first processed image stream, or

an image stream output interface; and

selectively mapping the contiguous image stream to;

the image processing module, wherein the image processing module is further configured to receive the contiguous image stream from the switching matrix, apply the image processing function to the contiguous image stream, and output a second processed image stream, or

the image stream output interface.

23. A vehicle vision system, comprising:

a plurality of image capture devices coupled to a vehicle;

a plurality of image stream input interfaces coupled to the plurality of image capture devices;

an image stream output interface;

a first image processing module configured to accept a plurality of image streams, stitch at least two image streams from the plurality of image streams into a contiguous image stream, and output the contiguous image stream, wherein the plurality of image streams comprises an image stream from at least one of the plurality of image stream input interfaces or from another image processing module;

a second image processing module; and

a switching matrix in communication with the plurality of image stream input interfaces, the image stream output interface, the first image processing module, and the second image processing module, wherein the switching matrix is configured to;

selectively map the image stream from the at least one of the plurality of image stream input interfaces or from the second image processing module, to the first image processing module, and

selectively map the contiguous image stream from the first image processing module to the image stream output interface or to the second image processing module.

24. The vehicle vision system of claim 23, wherein the contiguous image stream is a panoramic view from a vantage of the plurality of image capture devices.

25. The vehicle vision system of claim 23, wherein the switching matrix is configured to adjust the selective mapping in real time such that, a particular image stream is added to or removed from the plurality of image streams being accepted by the first image processing module.

26. The vehicle vision system of claim 23, wherein the vehicle is a ground vehicle, aircraft, watercraft, or a spacecraft.

27. The vehicle vision system of claim 23, wherein the image stream output interface is coupled to a display device disposed in a passenger cabin of the vehicle.

28. The vehicle vision system of claim 23, wherein at least one of the image capture devices comprises a video camera, an infrared camera, or an x-ray camera.

29. The vehicle vision system of claim 23, wherein the first image processing module performs stitching at frame-level.

30. The vehicle vision system of claim 23, wherein the vehicle vision system is coupled to another vehicle vision system and configured to transmit a shared image stream from the vehicle vision system to the other vehicle system, wherein the shared image stream is an image stream from at least one of the plurality of image stream input interfaces, the contiguous image stream, or a processed image stream outputted by the second image processing module.

31. The vehicle vision system of claim 30, wherein the other vehicle vision system is disposed at another vehicle, the vehicle vision system is coupled to the other vehicle vision system through a network connection between the vehicle and the other vehicle, and the shared image stream is transmitted from the vehicle to the other vehicle over a network connection.

32. The vehicle vision system of claim 23, wherein the second image processing module is configured to accept a second image stream from the image stream input interface or from another image processing module, apply an image processing function to the second image stream, and output a processed image stream.

33. The vehicle vision system of claim 32, wherein the image processing function comprises an image stream mix function, an image stream scale function, an image stream blend function, an image stream encoding function, an image stream enhancement function, or image stream cropping function.

34. The vehicle vision system of claim 33, wherein the image stream encoding function comprises a video or audio compression algorithm.

35. The vehicle vision system of claim 33, wherein the image stream enhancement function comprises a de-haze function, a de-blur function, a shadow function, a dawn-dusk function, a fusion function, a stabilization function, a thermal turbulence function, an equalization function, an edge detection function, a rain and fog function, or a light optimizing function.