Situational awareness by compressed display of panoramic views

• US 9,973,692 B2
• Filed: 01/16/2015
• Issued: 05/15/2018
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Szabo, Matthew Joseph, Vickery, Phillip, O'Dell, Brian D., Teich, Andrew C., Frank, Jeffrey D.
• Primary Examiner(s): Huang, Frank
• Application Number:
  US14/599,352
• Publication Number:
  US 20150172545A1
• Time in Patent Office: 1,215 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/663 for controlling interchangeable camera parts based on electronic image sensor signals
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

In one embodiment, a method for displaying a panoramic view image includes transmitting video data from a plurality of sensors to a data processor and using the processor to stitch the video data from respective ones of the sensors into a single panoramic image. A focus view of the image is defined and the panoramic image is scrolled such that the focus view is centered in the display. A high resolution camera is aimed along a line corresponding to a center of the focus view of the image and an image produced by the camera is stitched into the panoramic image. A mapping function is applied to the image data to compress the data and thereby reduce at least the horizontal resolution of the image in regions adjacent to the side edges thereof.

Claims

1. A method for displaying a panoramic view image, the method comprising:

obtaining a panoramic image, wherein the panoramic image comprises a central portion and peripheral portions;

applying a mapping function to the peripheral portions of the panoramic image to spatially compress each peripheral portion of the panoramic image based on a continuously increasing compression gradient that increases with distance from the central portion of the panoramic image to obtain, for each peripheral portion, a corresponding spatially compressed peripheral portion; and

displaying, on a display device, the spatially compressed peripheral portions together with the central portion of the panoramic image, wherein, for each spatially compressed peripheral portion, one or more dimensions of the spatially compressed peripheral portion are smaller than a corresponding one or more dimensions of the corresponding peripheral portion to which the mapping function is applied.

2. The method of claim 1, wherein the applying comprises applying the mapping function to spatially compress the peripheral portions of the panoramic image on at least two opposing sides of the central portion of the panoramic image.

3. The method of claim 1, wherein the panoramic image comprises a 360 degree field of view image and wherein the continuously increasing compression gradient comprises a linear compression gradient or a compression gradient specific to a particular viewer.

4. The method of claim 1, wherein the obtaining of the panoramic image comprises receiving a stitched image or a virtual image from an external system.

5. The method of claim 1, wherein the obtaining of the panoramic image comprises:

transmitting video data from a plurality of sensors disposed on a wearable structure to a data processor, wherein the plurality of sensors comprises a plurality of imaging modules, and wherein the wearable structure comprises a helmet; and

stitching the video data from respective ones of the plurality of sensors to form the panoramic image with the data processor.

6. The method of claim 1, wherein:

the peripheral portions comprise first peripheral portions;

the applying comprises defining a focus view, the first peripheral portions, and second peripheral portions of the panoramic image;

the focus view is a central focus view disposed in a center region of the panoramic image;

the first peripheral portions are located between the central focus view and side edges of the panoramic image;

the applying comprises applying the mapping function to reduce at least a horizontal resolution in the first peripheral portions;

the panoramic image comprises upper and lower edges disposed above and below the center region and running between the side edges;

the second peripheral portions are located between the central focus view and the upper and lower edges; and

the applying further comprises applying the mapping function to reduce a vertical resolution of the panoramic image in the second peripheral portions.

13. The method of claim 1, wherein the central portion is associated with a forward looking direction, wherein a size of each spatially compressed peripheral portion comprises a horizontal dimension and a vertical dimension arranged on the display device, and wherein at least one of the horizontal dimension or the vertical dimension of each spatially compressed peripheral portion is based on an angular distance of the associated peripheral portion from the forward looking direction.

14. The method of claim 1, wherein the displaying comprises displaying the spatially compressed peripheral portions and the central portion together as part of a substantially rectilinear image, wherein the central portion is associated with a forward looking direction, and wherein each of the spatially compressed peripheral portions is arranged on one of at least two opposing sides of the central portion based on an angular distance of the associated peripheral portion from the forward looking direction.

16. The method of claim 1, wherein the applying comprises, for each of the peripheral portions, spatially compressing the peripheral portion by a continuously increasing amount from one side of the peripheral portion to an opposing side of the peripheral portion.

19. The method of claim 1, wherein at least one of the spatially compressed peripheral portions has the same vertical dimension and a different horizontal dimension from at least one other of the spatially compressed peripheral images.

7. A system, comprising:

memory configured to store a panoramic image, wherein the panoramic image comprises a central portion and peripheral portions;

a processor configured to apply a mapping function to the peripheral portions of the panoramic image to spatially compress each peripheral portion of the panoramic image based on a continuously increasing compression gradient that increases with distance from the central portion of the panoramic image to obtain, for each peripheral portion, a corresponding spatially compressed peripheral portion; and

a display configured to display the spatially compressed peripheral portions together with the central portion of the panoramic image, wherein, for each spatially compressed peripheral portion, one or more dimensions of the spatially compressed peripheral portion are smaller than a corresponding one or more dimensions of the corresponding peripheral portion to which the mapping function is applied.

8. The system of claim 7, wherein the processor is configured to apply the mapping function to spatially compress the peripheral portions of the panoramic image on at least two opposing sides of the central portion of the panoramic image.

9. The system of claim 7, wherein the panoramic image comprises a 360 degree field of view image and wherein the continuously increasing compression gradient comprises a linear compression gradient or a compression gradient specific to a particular viewer.

10. The system of claim 7, further comprising:

a wearable structure; and

a plurality of sensors disposed on the wearable structure, wherein the plurality of sensors is configured to transmit video data to the processor, and wherein the processor is configured to stitch the video data from respective ones of the sensors to form the panoramic image.

11. The system of claim 10, wherein the plurality of sensors comprises a plurality of imaging modules disposed on the wearable structure, and wherein the wearable structure comprises a helmet.

12. The system of claim 7, wherein:

the peripheral portions comprise first peripheral portions;

the processor is configured to define a focus view, the first peripheral portions, and second peripheral portions of the panoramic image;

the focus view is a central focus view disposed in a center region of the panoramic image;

the first peripheral portions are located between the central focus view and side edges of the panoramic image;

the processor is configured to apply the mapping function to reduce at least a horizontal resolution in the first peripheral portions;

the panoramic image comprises upper and lower edges running between the side edges;

the second peripheral portions are located between the central focus view and the upper and lower edges; and

the processor is further configured to apply the mapping function to reduce a vertical resolution of the panoramic image in the second peripheral portions.

18. The system of claim 7, further comprising:

a wearable structure comprising;

a projector configured to provide, on the display, the spatially compressed peripheral portions and the central portion of the panoramic image, wherein the display is coupled to the wearable structure.

15. A method for displaying a panoramic view image, the method comprising:

obtaining a panoramic image, wherein the panoramic image comprises a central portion and peripheral portions, and wherein the central portion is associated with a forward looking direction;

applying a mapping function to the peripheral portions of the panoramic image to spatially compress the peripheral portions of the panoramic image to obtain spatially compressed peripheral portions; and

displaying, on a display device, the spatially compressed peripheral portions together with the central portion of the panoramic image, wherein the displaying comprises displaying the central portion together with a substantially rectilinear image formed of the spatially compressed peripheral portions, wherein a position of each spatially compressed peripheral portion on the display device is based on an angular distance between the associated peripheral portion and the forwarding looking direction, wherein the central portion and the rectilinear image together form a 360 degree view about the forward looking direction, and wherein, for each spatially compressed peripheral portion, one or more dimensions of the spatially compressed peripheral portion are smaller than a corresponding one or more dimensions of the associated peripheral portion to which the mapping function is applied.

17. The method of claim 15, wherein a peripheral portion at a larger angular distance from the forward looking direction has an associated spatially compressed peripheral portion of a smaller size than an associated spatially compressed peripheral portion of a peripheral portion at a smaller angular distance from the forward looking direction.

20. The method of claim 15, wherein the applying comprises applying the mapping function to the peripheral portions to spatially compress the peripheral portions based on a continuously increasing compression gradient that increases with distance from the central portion of the panoramic image.