OMNI-DIRECTIONAL CAMERA AND RELATED VIEWING SOFTWARE

• US 20120206565A1
• Filed: 02/10/2012
• Published: 08/16/2012
• Est. Priority Date: 02/10/2011
• Status: Active Grant

• Current Assignee: Villmer LLC
• Original Assignee: Villmer LLC
• Inventors: Villmer, Jason
• Granted Patent: US 9,930,225 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 1/00 Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3254 Orientation, e.g. landscape or portrait; Location or order of the image data, e.g. in memory
• CPC Class Codes: H04N 2201/3264 of sound signals
• CPC Class Codes: H04N 2201/3267 of motion picture signals, e.g. video clip
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Creating spherical video and corresponding audio utilizing a plurality of cameras by acquiring images via the cameras, processing the images, and mathematically converting and/or transforming the images into a combined spherical image file. A camera device creates the spherical image file and transfers it to devices running a viewer application, whereby the transfer occurs by saving the file to a storage file, streaming it to the devices via wired/wireless communication, or both.

Claims

1. A camera device for creating spherical video and corresponding audio, said camera device comprising:

a camera device housing;

a computer processor for operating the camera device, said computer processor being located inside the housing;

a plurality of cameras, each of said cameras having a lens attached thereto located outside the housing and oriented in different directions for acquiring image data;

at least one microphone for acquiring audio data corresponding to the acquired image data;

a system memory having stored thereon computer-processor executable instructions for operating the camera device, said computer-processor executable instructions comprising instructions for;

initiating image acquisition;

acquiring at least first and second image data synchronously from the plurality of cameras;

acquiring audio data from the at least one microphone synchronously with the image data acquisition;

processing the acquired first image data and the acquired second image data to create a single image including at least a portion of the acquired first image data and at least a portion of the acquired second image;

encoding the single image and the acquired audio data into a spherical video file; and

saving the spherical video file to a system memory.

2. The camera device of claim 1, wherein processing the acquired first image data and the acquired second image data comprises:

cropping the acquired first image data and the acquired second image data;

scaling the cropped first image data and the cropped second image data;

rotating the scaled first image data and the scaled second image data;

adjusting one or more visual properties of the first image data and the rotated second image data, said visual properties comprising one or more of color, brightness, and contrast;

correcting for vignette features of the adjusted first image data and the adjusted second image data;

converting the corrected first image data and the corrected second image data by applying one or more mathematical transformations to the corrected first image data and the corrected second image data; and

creating the spherical image file by fusing the converted first image data and the converted second image data.

3. The camera device of claim 2, wherein said fusing comprises:

merging the converted first image data and the converted second image data into an image container;

masking the converted first image data and the converted second image data into the image container; and

blending the converted first image data and the converted second image data into the image container, to create the single image.

4. The camera device of claim 2, wherein said fusing comprises:

creating a single, spherical image by placing the converted first image date and the converted second image data together edge-to-edge.

5. The camera device of claim 1, wherein the housing has at least a first side and a second side, and wherein the cameras include a first camera oriented on the first side of the housing and a second camera oriented on the second side of the housing.

6. The camera device of claim 1, said computing processor being located outside of the housing.

7. The camera device of claim 1, said camera device further comprising an externally oriented input/output port.

8. The camera device of claim 1, said camera device further comprising a card slot for accepting a card-based storage.

9. The camera device of claim 8, wherein the card-based storage includes one of a card-based wireless communication module and a card-based storage that includes a wireless communication module.

10. The camera device of claim 1, further comprising an integrated wireless communication device.

11. The camera device of claim 1, wherein said computer-processor executable instructions further comprise instructions for transmitting the spherical video file to a remote computing device via at least one of a wired connection via the camera device input/output port, an integrated Wi-Fi communication module, a card-based storage card with an integrated wireless communication module, a built-in cellular communication module, and an external cellular communication module.

12. The camera device of claim 1, further comprising a touch screen display for accepting user input and displaying information regarding the state of the camera device.

13. The camera device of claim 12, wherein the display permits the user to enter camera device preferences and commands via the display.

14. The camera device of claim 1, further comprising an accelerometer and a gyroscope for determining rotational changes of the camera device during camera device operation, said rotational changes being stored as rotational metadata.

15. The camera device of claim 1, further comprising a Global Positioning System (“

GPS”

) device for determining positional changes of the camera device during camera device operation, said positional changes being stored as positional metadata.

16. A computerized method for viewing a spherical video file including directional sound created by a camera in a spherical video viewer, said viewer being executed by a computing device, said method comprising:

receiving, at the computing device, the spherical video file;

generating, at the computing device, a three-dimensional virtual environment;

creating, at the computing device, a render sphere within the virtual environment;

applying, at the computing device, the spherical video file as a texture to the interior of the render sphere;

positioning, at the computing device, a virtual render camera at the center of the render sphere, said render camera defining a user view of the spherical video file playing on the interior of the render sphere, said view determined by one or more properties of the render camera, said properties including rotational position of the camera about the camera'"'"'s x-axis, y-axis, and z-axis, and a field-of-view value defining a camera zoom;

displaying, on a display device of the computing device, the user view to the user, said user view including directional audio that varies with the position of rotational position of the camera;

receiving, at the computing device, user input to change the view;

changing, at the computing device, one or more properties of the virtual render camera about one or more of the camera axes in response to the received user input to change the view, wherein changing the properties includes at least one of changing one or more of the rotational properties of the camera and changing a field-of-view value; and

updating, at the computing device, the user view including the directional audio based in response to changing the one or more camera properties.

17. The method of claim 16, wherein said receiving comprises receiving orientation metadata describing one or more orientation properties of the spherical camera during the creation of the video.

18. The method of claim 17, wherein the spherical video viewer rotates the render sphere about one or more axis of the render sphere based on the orientation metadata associated with the spherical video being played on the interior of the render sphere, said render sphere rotation compensating for any rotation of the spherical camera during creation of the spherical video.

19. The method of claim 16, wherein said receiving comprises receiving positional metadata describing one or more positions of the spherical camera during the creation of the video, and wherein the video viewer overlays a map over a portion of the user viewer, said map visually indicating the received positional metadata.

20. A system for creating omni-directional video with directional audio for viewing by a video viewer, said system comprising:

a video capture device comprising at least a first camera with a first lens attached thereto for acquiring a first video image and a second camera with a second lens attached thereto for acquiring a second video image, said cameras being oriented on or about the video capture device such that the first video image and the second video image include video images of substantially 360 degrees about a central point;

at least a first microphone oriented for acquiring audio corresponding to the first video image and at least a second microphone oriented for acquiring audio corresponding to the second video image;

a positional information acquisition device for acquiring position information for the video capture device;

an orientation information acquisition device for acquiring orientation information for the video capture device indicating rotation of the video capture device about one or more axis;

a computer for operating the camera device; and

one or more computer-readable storage media having stored thereon computer-processor executable instructions for operating the camera device, said computer-processor executable instructions comprising instructions for;

acquiring first image data and second image data synchronously from the first camera and the second camera respectively;

acquiring audio data from the first microphone and second microphone synchronously with the image data acquisition;

adjusting one or more visual properties of the acquired first image data and the second image data;

converting the adjusted first image data and the adjusted second image data by applying one or more mathematical transformations to the corrected first image data and the corrected second image data; and

creating a spherical image from the converted first image data and the converted second image data;

encoding the spherical image and the acquired audio data into a spherical video file; and

transferring the spherical video file, the acquired positional information, and the acquired orientation information to at least one of a user computing device, a web server, and a storage device of the camera device.

21. The system of claim 20, wherein the mathematical transformation comprises at least one or more circular-to-rectilinear mathematical transformations.