Apparatus and method to record a 360 degree image

• US 9,906,721 B2
• Filed: 05/20/2016
• Issued: 02/27/2018
• Est. Priority Date: 10/30/2015
• Status: Expired due to Fees

• Current Assignee: Essential Products, Inc.
• Original Assignee: Essential Products, Inc.
• Inventors: Evans, V, David John, Jiang, Xinrui, Rubin, Andrew E., Hershenson, Matthew, Miao, Xiaoyu
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US15/161,153
• Publication Number:
  US 20170126971A1
• Time in Patent Office: 648 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The technology disclosed herein combines wide angle lenses with normal lenses to create an undistorted 360° view of the surrounding environment. The normal lenses record an image of the surrounding environment in the areas where the wide angle lenses provide a distorted view. The wide angle lenses and the normal lenses can be part of a stand-alone imaging device, can be accessories to a mobile device, or can be integrated into a mobile device. Various ways to integrate the wide lenses and/or the normal lenses into the camera are disclosed herein.

Claims

1. An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:

a first plurality of cameras associated with a mobile device, the first plurality of cameras comprising a plurality of fisheye lenses distributed evenly around the first plurality of cameras, wherein the first plurality of cameras is configured to record a first plurality of images distorted at a plurality of image edges;

a second plurality of cameras integrated with the mobile device, the second plurality of cameras comprising a plurality of normal lenses evenly distributed around the second plurality of cameras, wherein each normal lens in the plurality of normal lenses is oriented in a direction substantially equally angularly distant from directions of at least two fisheye lenses in the plurality of fisheye lenses, wherein the second plurality of cameras is configured to record a second plurality of images associated with the surrounding environment, wherein each image in the second plurality of images comprises a second image center undistorted by a camera in the second plurality of cameras, wherein the second plurality of image centers associated with the second plurality of images overlaps the plurality of image edges associated with the first plurality of images,

wherein the first plurality of cameras and the second plurality of cameras are configured to record the first plurality of images and the second plurality of images synchronously; and

a processor configured to combine the first plurality of images and the second plurality of images into an image undistorted by the plurality of fisheye lenses.

2. The imaging device of claim 1, the first plurality of cameras comprising a mobile device accessory configured to attach to the mobile device.

3. The imaging device of claim 1, comprising a swivel configured to dispose the first plurality of cameras to assume a first position, wherein the first position enables the first plurality of cameras to record a third plurality of images.

4. The imaging device of claim 1, the first plurality of cameras integrated into the mobile device, the first plurality of cameras movably disposed inside a camera port associated with the mobile device,

the first plurality of cameras configured,

when the first plurality of cameras is inactive, to retract inside the camera port, and to align with an outside casing associated with the mobile device, and

the first plurality of cameras configured,

when the first plurality of cameras is active, to protrude from the outside casing associated with the mobile device, and to position the plurality of fisheye lenses to receive light unobstructed by the mobile device.

5. The imaging device of claim 1, the second plurality of cameras comprising:

a front lens proximate to a display associated with the mobile device;

a back lens disposed in a direction opposite of the front lens;

a plurality of photo sensors; and

an array of light deflectors associated with the front lens, the back lens, and the plurality of photo sensors, the array of light deflectors changing a direction of a light beam, the array of light deflectors operable to assume a first position, and a second position, the first position deflecting a light beam associated with the front lens to the plurality of photo sensors associated with the camera, and the second position deflecting a light beam associated with the back lens to the plurality of photo sensors associated with the camera.

6. The imaging device of claim 1, the second plurality of cameras comprising a front camera proximate to a display associated with the mobile device, and a back facing camera disposed in a direction opposite of the front camera.

7. The imaging device of claim 1, the second plurality of cameras occupying a portion of a display screen associated with the mobile device, the portion of the display screen occupied by the camera appearing to be a camera icon, the camera icon operable to activate the camera when the camera icon is selected.

8. The mobile device of claim 7, the camera icon operable to launch a camera application on the mobile device when the camera icon is selected.

9. An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:

a first plurality of cameras comprising a plurality of wide angle lenses, wherein the first plurality of cameras is configured to record a first plurality of images distorted at a plurality of image edges;

a second plurality of cameras comprising a plurality of normal lenses, wherein the second plurality of cameras is configured to record a second plurality of images associated with the surrounding environment, wherein each normal lens in the plurality of normal lenses is oriented in a direction substantially equally angularly distant from directions of at least two fisheye lenses in the plurality of fisheye lenses, wherein each image in the second plurality of images comprises a second image center undistorted by a camera in the second plurality of cameras, wherein the second plurality of image centers associated with the second plurality of images overlaps the plurality of image edges associated with the first plurality of images.

10. The imaging device of claim 9, wherein the plurality of wide angle lenses are distributed evenly around the first plurality of cameras, and wherein the plurality of normal lenses are distributed evenly around the second plurality of cameras.

11. The imaging device of claim 9, wherein the first plurality of cameras and the second plurality of cameras are configured to record the first plurality of images and the second plurality of images synchronously.

12. The imaging device of claim 9, wherein a wide angle lens in the plurality of wide-angle lenses comprises a fisheye lens.

13. The imaging device of claim 9, wherein the first plurality of cameras comprises a mobile device accessory, and wherein the second plurality of cameras is integrated into a mobile device.

14. The imaging device of claim 9, wherein the imaging device comprises an elongated stand, each elongated stand attached to the imaging device at a juncture disposed between two normal lenses in the plurality of normal lenses, wherein the elongated stand is configured to support the imaging device, when the elongated stand is in contact with a support surface.

15. The imaging device of claim 9, wherein the first plurality of cameras comprises a first mobile device accessory, and wherein the second plurality of cameras comprises a second mobile device accessory.

16. The imaging device of claim 9, comprising a swivel configured to dispose the first plurality of cameras and the second plurality of cameras to assume a first position, wherein the first plurality of cameras is configured to record a third plurality of images, and the second plurality of cameras is configured to record a forth plurality of images, when the first plurality of cameras and the second plurality of cameras assume the first position.

17. The imaging device of claim 16, the swivel configured to automatically rotate the first plurality of cameras and the second plurality of cameras after the first plurality of cameras and the second plurality of cameras record the first plurality of images and the second plurality of images, respectively.

18. The imaging device of claim 9, comprising a processor configured to combine the first plurality of images and the second plurality of images into an image undistorted by the plurality of wide angle lenses.

19. The imaging device of claim 9, the first plurality of cameras integrated into a mobile device, the first plurality of cameras movably disposed inside a camera port associated with the mobile device, the first plurality of cameras configured,

when the first plurality of cameras is inactive, to retract inside the camera port, and to align with an outside casing associated with the mobile device, and

the first plurality of cameras configured,

when the first plurality of cameras is active, to protrude from the outside casing associated with the mobile device, and to position the plurality of wide angle lenses to receive light unobstructed by the mobile device.

20. The mobile device of claim 19, the first plurality of cameras comprising:

a first wide angle lens;

a second wide angle lens;

a plurality of photo sensors; and

an array of light deflectors associated with the first wide angle lens, the second wide angle lens, and the plurality of photo sensors, the array of light deflectors changing a direction of a light beam, the array of light deflectors operable to assume a first position, and a second position, the first position deflecting a light beam associated with the first wide angle lens to the plurality of photo sensors associated with the first plurality of cameras, and the second position deflecting a light beam associated with the second wide angle lens to the plurality of photo sensors associated with the first plurality of cameras.

21. The mobile device of claim 20, the camera comprising an array of micro-electromechanical systems devices corresponding to the array of light deflectors, the array of micro-electromechanical systems devices operable to individually actuate each light deflector in the array of light deflectors.

22. The mobile device of claim 20, each light deflector in the array of light deflectors configured to axially rotate and dispose the array of light deflectors in a curved surface.

23. The imaging device of claim 9, the first plurality of cameras integrated into a mobile device, a camera in the first plurality of cameras occupying a portion of a display screen associated with the mobile device, the portion of the display screen occupied by the camera appearing to be a camera icon, the camera icon operable to activate the camera when the camera icon is selected.

24. The mobile device of claim 23, the camera icon disposed in a portion of the display screen reserved for a plurality of icons corresponding to a plurality of mobile devices operating system functions.

25. An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:

a first plurality of cameras associated with a mobile device, the first plurality of cameras comprising a plurality of fisheye lenses oriented in a plurality of directions;

a second plurality of cameras integrated with the mobile device, the second plurality of cameras comprising a plurality of normal lenses, wherein each normal lens in the plurality of normal lenses is oriented in a direction substantially equally angularly distant from directions of at least two fisheye lenses in the plurality of fisheye lenses,

wherein the first plurality of cameras and the second plurality of cameras are configured to record the first plurality of images and the second plurality of images synchronously; and

a processor configured to combine the first plurality of images and the second plurality of images into an image undistorted by the plurality of fisheye lenses.

26. An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:

a first plurality of cameras associated with a mobile device, the first plurality of cameras comprising a plurality of fisheye lenses oriented in a plurality of directions substantially equally angularly distant from each other; and

a second plurality of cameras comprising a plurality of normal lenses, wherein each normal lens in the plurality of normal lenses is oriented in a direction substantially equally angularly distant from directions of at least two fisheye lenses in the plurality of fisheye lenses.