Providing a real-time via a wireless communication channel associated with a panoramic video capture device

• US 9,883,101 B1
• Filed: 07/23/2014
• Issued: 01/30/2018
• Est. Priority Date: 07/23/2014
• Status: Expired due to Fees

• Current Assignee: Hoyos Integrity Corporation
• Original Assignee: Hoyos Integrity Corporation
• Inventors: Aloumanis, Peter, De La Cruz, Moises, Mor, Tal, Padin, Guillermo, Verma, Amit
• Primary Examiner(s): Villecco, John
• Application Number:
  US14/339,083
• Time in Patent Office: 1,287 Days
• Field of Search: 348 36, 348 39, 382285, 382293, 382295
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A 360 degree panorama can capture a view of a real world environment. The view can be a 360 panorama which can conform to a stereographic projection. The capturing can be performed by a one shot panoramic video capture device lacking a display. A 360 degree arc frame can be analyzed to determine a one-to-one mapping to a target projection. The target projection can be a cylindrical or an equirectangular projection. The stereographic projection can be mapped to the target projection in real-time without performing a digital stitching operation within the device. Distortion in the target projection resulting from the mapping can be corrected. The target projection can be conveyed in real-time wirelessly to a display of a communicatively linked computing device.

Claims

1. A method for presenting a 360 degree field of view comprising:

capturing a 360 degree panorama view of a real world environment as a stereographic projection, wherein the view is a 360 degree panorama, wherein the capturing is performed by a one shot panoramic video capture device in a video feed comprised of a plurality of frames, each frame generated from the stereographic projection;

transforming each frame by;

analyzing the stereographic projection to determine a one-to-one mapping to a target projection, wherein the target projection is at least one of a cylindrical projection or an equirectangular projection;

mapping the stereographic projection to the target projection within the panoramic video capture device;

correcting a distortion in the target projection resulting from the mapping to produce a corrected target projection showing a perspective view of the 360 degree panorama view, wherein the distortion results from the captured stereographic projection being mapped to the target projection;

wherein the transforming for each respective frame is performed within a resource threshold of less than forty milliseconds of processing time; and

conveying the target projection for each frame in real-time wirelessly to a communicatively linked computing device.

2. The method of claim 1, wherein the panoramic video capture device lacks a display.

3. The method of claim 1, wherein the transforming further comprises:

applying a smoothing function to average out a pixel density of the target projection.

4. The method of claim 1, wherein the mapping further comprises:

determining a radial line of the stereographic projection starting at a center of the stereographic projection and ending at a boundary edge of of the stereographic projection; and

deforming the stereographic projection resulting in the radial line becoming a vertical boundary edge.

5. A system for presenting a 360 degree field of view comprising:

a panoramic video capture device including a processor configured to execute a transform engine to transform each frame of a video feed to a target projection resulting in a transformed frame being generated for each corresponding frame of the video feed, wherein each frame of the video feed is captured as a stereographic projection of a 360 degree panorama view, wherein the target projection is at least one of a cylindrical projection or a equirectangular projection, wherein the transform occurs in real-time, wherein the video capture device conveys each transformed frame wirelessly to a communicatively linked computing device which displays the transformed frames as video;

a data store configured to persist at least one of the video feed, a mapping, and the transformed frame;

wherein the transform is performed within a resource threshold of less than forty milliseconds of processing time.

6. The system of claim 5, wherein the video capture device lacks a display.

7. The system of claim 5, wherein the processor is further configured to:

responsive to the transform, apply a smoothing function to average out the pixel density of the target projection.

8. The system of claim 5, wherein the processor is further configured to:

determine a radial line of the stereographic projection starting at a center of the stereographic projection and ending at a boundary edge of the stereographic projection; and

deform the stereographic projection resulting in the radial line becoming a vertical boundary edge.

9. The system of claim 5, wherein each transformed frame is conveyed via a Real-Time Transport protocol to a proximate or remote computing device.

10. The system of claim 5, wherein the processor is further configured to:

prior to the transformed framed being conveyed, compress a set of transformed frames using a High Efficiency Video Coding (HEVC) or a VP9 coding for the conveyance.

11. A panoramic video capture device comprising:

a panoramic video capture device configured to capture a continuous 360 degree panorama view of a real world environment in a video feed comprised of a plurality of frames, wherein each frame is initially captured as a stereographic projection, wherein the continuous 360 degree panorama view comprises a vertical range of negative fifteen degrees to positive forty five degrees and a horizontal range of at least two hundred and seventy degrees, wherein the video capture devices lacks a display

the video capture device further comprising;

a wireless transceiver able to wirelessly convey the view to a communicatively linked computing device;

an optical lens stack,

a quadric reflector having an aperture over the optical lens stack, and through which light passes into the optical lens stack,

an image sensor onto which light is projected through the optical lens stack,

an overhang enclosure made of a polymer, and in which is disposed a reflecting mirror, wherein the overhang enclosure blocks secondary reflections from entering the aperature, and

a processor configured to transform each stereographic projection for each frame of the 360 degree panoramic view to a continuous equirectangular view of the real world environment in real-time, wherein, to perform the transform, the processor is configured to map each stereographic projection to a target projection, and correct a distortion in the target projection that results upon being mapped to produce a corrected target projection as the equirectangular view of the 360 degree panorama view, wherein the distortion results from the captured stereographic projection being mapped to the target projection, wherein the transform for each frame occurs within a resource threshold time of less than forty milliseconds.

12. The video capture device of claim 11, wherein the processor is configured to compress at least one frame of the continuous equirectangular view using a High Efficiency Video Coding (HEVC) or a VP9 coding.

13. The video capture device of claim 11, wherein the continuous equirectangular view is conveyed via a Real-Time Transport protocol to a proximate or remote computing device.

14. The video capture device of claim 11, wherein the processor is configured to apply a smoothing function to average out the pixel density of the continuous equirectangular view.