Systems and methods for greater than 360 degree capture for virtual reality

• US 9,773,350 B1
• Filed: 09/15/2015
• Issued: 09/26/2017
• Est. Priority Date: 09/16/2014
• Status: Active Grant

• Current Assignee: Silverthread Inc.
• Original Assignee: Silverthread Inc.
• Inventors: Crosby, Tai
• Primary Examiner(s): Ren, Zhubing
• Application Number:
  US14/855,180
• Time in Patent Office: 742 Days
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

Systems and methods for capturing video information in the real world use a set of cameras that are supported, carried, and/or worn by a subject. Virtual reality content is generated based on the captured video and audio information, which provides a true point-of-view (POV) experience. Virtual reality content is played back for other users to have an experience based on the subject'"'"'s experience.

Claims

1. A system configured to capture video information in the real world near a first subject as the first subject experiences a real-world experience and to generate and subsequently store and/or transmit virtual reality content based on the captured video information, wherein the virtual reality content facilitates an artificial environment which is experienced through sensory stimuli including sights and sounds, wherein the sensory stimuli are provided by a computer, wherein a second subject'"'"'s actions partially determine what happens in the artificial environment, the system comprising:

electronic storage configured to store information electronically;

a set of cameras configured to capture real-world video information, wherein one or more individual cameras in the set are arranged to capture the real-world video information from a point-of-view of the first subject, wherein the real-world video information includes images captured in multiple viewing directions simultaneously, wherein the multiple viewing directions include a downward viewing direction;

a support structure configured to support the set of multiple cameras and furthermore configured to be carried and/or worn by the first subject such that the one or more individual cameras are positioned at least at eye-level of the first subject while the support structure is being carried and/or worn by the first subject; and

one or more physical processors configured via computer-readable instructions to;

control the set of cameras to capture, at a first moment in time, the real-world video information, wherein the one or more individual cameras capture the real-world video information that includes images of a body of the first subject as the first subject experiences the real-world experience;

generate virtual reality content that facilitates the artificial environment which is experienced, at a second moment in time after the first moment in time, through sensory stimuli including sights and sounds,

wherein the second subject'"'"'s actions partially determine what happens in the artificial environment, wherein the virtual reality content is based on the real-world video information captured as the first subject experienced the real-world experience, wherein actions by the second subject partially determine which images of the body of the first subject are experienced in the artificial environment by the second subject,

wherein, responsive to the second subject adjusting a viewing direction while experiencing the artificial environment, the artificial environment is facilitated by the virtual reality content that is based on the real-world video information that includes the images captured in an individual viewing direction selected from the multiple viewing directions,

wherein the individual viewing direction corresponds to the adjusted viewing direction of the second subject; and

store the virtual reality content in the electronic storage.

2. The system of claim 1, wherein the support structure includes a helmet configured to be worn on the first subject'"'"'s head.

3. The system of claim 1, wherein the one or more individual cameras are further positioned less than 3 inches above eye-level of the first subject.

4. The system of claim 1, wherein the one or more individual cameras in the set are further arranged to capture real-world video information that includes images captured both in front of and behind the first subject'"'"'s feet, and wherein the virtual reality content further includes imagery based on the images in front of and behind the first subject'"'"'s feet.

5. The system of claim 1, wherein the one or more physical processors are configured to generate the virtual reality content to be a photo-realistic representation of the real-world video information captured by the set of cameras.

6. The system of claim 1, wherein the set of cameras is configured and arranged to capture real-world video information in a vertical plane traversing more than 360 degrees, wherein the images captured in a downward direction overlap by at least 30 degrees.

7. The system of claim 6, wherein the one or more physical processors are configured to store the virtual reality content in a storage structure within the electronic storage by compressing the virtual reality content that is based on the real-world video information corresponding to a view of 180 degrees into a portion of the storage structure corresponding to a view of less than 180 degrees.

8. A system configured to play back virtual reality content that is generated based on video information captured near a first subject as the first subject experiences a real-world experience, wherein the virtual reality content facilitates an artificial environment which is experienced through sensory stimuli including sights and sounds, wherein the sensory stimuli are provided by a computer, wherein a second subject'"'"'s actions partially determine what happens in the artificial environment, wherein the virtual reality content is played back for a second subject, the system comprising:

an electronic display configured to present information to the second subject;

electronic storage configured to store information electronically, wherein the information includes virtual reality content generated based on real-world video information captured by a set of cameras, from a point-of-view of the first subject, as the first subject experienced the real-word experience at a first moment in time,

wherein the virtual reality content facilitates the artificial environment which is experienced, at a second moment in time after the first moment in time, by the second subject through sensory stimuli including sights and sounds,

wherein the second subject'"'"'s actions partially determine what happens in the artificial environment, wherein actions by the second subject partially determine which images of a body of the first subject are experienced in the artificial environment by the second subject,

wherein, responsive to the second subject adjusting a viewing direction while experiencing the artificial environment, the artificial environment is facilitated by the virtual reality content that is based on the real-world video information that includes the images captured in an individual viewing direction selected from multiple viewing directions,

wherein the individual viewing direction corresponds to the adjusted viewing direction of the second subject, wherein the first subject carried the set of cameras during capture of the real-world video information, wherein the real-world video information includes images captured in the multiple viewing directions simultaneously, wherein the multiple viewing directions include a downward viewing direction;

one or more physical processors configured via computer-readable instructions to;

communicate with the electronic storage to access the virtual reality content stored thereon; and

play back, on the electronic display, the virtual reality content for the second subject such that, responsive to actions by the second subject that effectuate the downward viewing direction, the artificial environment experienced by the second subject includes the images captured in the downward viewing direction.

9. The system of claim 8, wherein the system further comprises a display support structure configured to support the electronic display and furthermore configured to be carried and/or worn by the second subject.

10. The system of claim 9, wherein the system further comprises one or more sensors configured to generate output signals conveying information related to one or more positional parameters of at least one of the second subject and the electronic display, and wherein the one or more physical processors are configured to play back the virtual reality content based on the one or more positional parameters.

11. The system of claim 8, wherein the first subject is a different person than the second subject.