Systems and methods for identifying viewing directions for video content

• US 10,687,039 B1
• Filed: 05/16/2018
• Issued: 06/16/2020
• Est. Priority Date: 05/16/2018
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Vaden, Mark, Cotoros, Ingrid A., Novell, Scott, Dottel, Valerian, Tilloux, Guillaume, Bailliard, Loic, Lanoir, Loic
• Primary Examiner(s): Nirjhar, Nasim N
• Application Number:
  US15/981,667
• Time in Patent Office: 762 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Video information and sensor information may be obtained. The video information may define spherical video content having a progress length. The spherical video content may define visual content viewable from a point a view as a function of progress through the progress length of the spherical video content. The spherical video content may be captured by one or more image capture devices. The sensor information may characterize capture of the spherical video content. A viewing direction for the spherical video content may be determined based on the sensor information and/or other information. The viewing direction may define a direction of view for the spherical video content from the point of view as the function of progress through the progress length of the spherical video content. The spherical video content may be presented on a display based on the viewing direction.

Claims

1. A system that identifies viewing directions for video content, the system comprising:

one or more physical processors configured by machine-readable instructions to;

obtain video information defining spherical video content, the spherical video content having a progress length, the spherical video content defining visual content viewable from a point of view as a function of progress through the progress length of the spherical video content, the spherical video content captured by one or more image sensors of an image capture device, the image capture device carried during capture of the spherical video content by a person riding a vehicle, forward motion of the vehicle causing forward motion of the image capture device, the image capture device rotated by the person during the capture of the spherical video content such that a front direction of the spherical video content is pointed away from the forward motion of the image capture device during the capture of the spherical video content, the image capture device rotated by the person independently of the forward motion of the vehicle;

obtain sensor information generated by a motion sensor or a location sensor separate from the one or more image sensors of the image capture device, the sensor information characterizing the forward motion of the image capture device during the capture of the spherical video content;

determine a viewing direction for the spherical video content based on the forward motion of the image capture device and the rotation of the image capture device that causes the front direction of the spherical video content to be pointed away from the forward motion of the image capture device, the viewing direction determined to be pointed in the forward motion of the image capture device during the capture of the spherical video content and to compensate for the rotation of the image capture device by the person during the capture of the spherical video content such that a punch-out view of the spherical video content follows and stays centered in the forward motion of the image capture device during the capture of the spherical video content, the viewing direction defining a direction of view for the spherical video content from the point of view as the function of progress through the progress length of the spherical video content; and

present the spherical video content on a display based on the viewing direction such that the presentation of the spherical video content includes the punch-out view of the spherical video content that follows and stays centered in the forward motion of the image capture device during the capture of the spherical video content.

2. The system of claim 1, wherein the motion sensor or the location sensor is carried by the vehicle.

3. The system of claim 1, wherein the motion sensor or the location sensor is carried by the image capture device.

4. The system of claim 1, wherein the sensor information further characterizes a tilt of the vehicle, and the viewing direction is determined based on the tilt of the vehicle such that the viewing direction is directed towards back of the vehicle rather than the forward motion of the image capture device based on the tilt of the vehicle exceeding a threshold angle.

5. The system of claim 1, wherein:

the image capture device is carried by the person based on the image capture device being carried on the person'"'"'s head; and

the image capture device rotates independently of the forward motion of the vehicle based on rotation of the person'"'"'s head.

6. The system of claim 5, wherein the image capture device being carried on the person'"'"'s head includes the image capture device being mounted on a helmet worn by the person.

7. The system of claim 6, wherein the determination of the viewing direction to be pointed in the forward motion of the image capture device during the capture of the spherical video content and to compensate for the rotation of the image capture device by the person during the capture of the spherical video content includes the viewing direction being determined to compensate for the rotation of the person'"'"'s head during the capture of the spherical video content.

8. The system of claim 1, wherein the one or more physical processors are further configured by the machine-readable instructions to stabilize the spherical video content based on the viewing direction.

9. A method for identifying viewing directions for video content, the method performed by a computing system including one or more processors, the method comprising:

obtaining, by the computing system, video information defining spherical video content, the spherical video content having a progress length, the spherical video content defining visual content viewable from a point of view as a function of progress through the progress length of the spherical video content, the spherical video content captured by one or more image sensors of an image capture device, the image capture device carried during capture of the spherical video content by a person riding a vehicle, forward motion of the vehicle causing forward motion of the image capture device, the image capture device rotated by the person during the capture of the spherical video content such that a front direction of the spherical video content is pointed away from the forward motion of the image capture device during the capture of the spherical video content, the image capture device rotated by the person independently of the forward motion of the vehicle;

obtaining, by the computing system, sensor information generated by a motion sensor or a location sensor separate from the one or more image sensors of the image capture device, the sensor information characterizing the forward motion of the image capture device during the capture of the spherical video content;

determining, by the computing system, a viewing direction for the spherical video content based on the forward motion of the image capture device and the rotation of the image capture device that causes the front direction of the spherical video content to be pointed away from the forward motion of the image capture device, the viewing direction determined to be pointed in the forward motion of the image capture device during the capture of the spherical video content and to compensate for the rotation of the image capture device by the person during the capture of the spherical video content such that a punch-out view of the spherical video content follows and stays centered in the forward motion of the image capture device during the capture of the spherical video content, the viewing direction defining a direction of view for the spherical video content from the point of view as the function of progress through the progress length of the spherical video content; and

presenting, by the computing system, the spherical video content on a display based on the viewing direction such that the presentation of the spherical video content includes a punch-out view of the spherical video content that follows and stays centered in the forward motion of the image capture device during the capture of the spherical video content.

10. The method of claim 9, wherein the motion sensor or the location sensor is carried by the vehicle.

11. The method of claim 9, wherein the sensor information further characterizes a tilt of the vehicle, and the viewing direction is determined based on the tilt of the vehicle such that the viewing direction is directed towards back of the vehicle rather than the forward motion of the image capture device based on the tilt of the vehicle exceeding a threshold angle.

12. The method of claim 9, wherein:

the image capture device is carried by the person based on the image capture device being carried on the person'"'"'s head; and

the image capture device rotates independently of the forward motion of the vehicle based on rotation of the person'"'"'s head.

13. The method of claim 12, wherein the image capture device being carried on the person'"'"'s head includes the image capture device being mounted on a helmet worn by the person.

14. The method of claim 13, wherein determining the viewing direction to be pointed in the forward motion of the image capture device during the capture of the spherical video content and to compensate for the rotation of the image capture device by the person during the capture of the spherical video content includes the viewing direction being determined to compensate for the rotation of the person'"'"'s head during the capture of the spherical video content.