Panoramic video distribution and viewing

• US 10,015,527 B1
• Filed: 12/16/2014
• Issued: 07/03/2018
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Banta, William Christopher, Selmanovic, Demir
• Primary Examiner(s): Rahman, Mohammad J
• Application Number:
  US14/571,731
• Time in Patent Office: 1,295 Days
• Field of Search: None
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

Various examples are directed to the generating and playback of panoramic videos comprising a plurality of tiles. Each tile may be a video having a tile field-of-view that is a portion of a full field-of-view of the panoramic video. An active field-of-view may be displayed from the first tile, where the active field-of-view is selected from a first tile field-of-view. A translate command may indicate a translate direction for the active field of view. A second tile may be requested. The second tile may have a second tile field-of-view that is adjacent to the first file field-of-view in the translate direction. The active field of view may be translated in the translate direction. When the active field-of-view reaches an edge of the first tile field-of-view, the active field-of-view may be displayed, at least in part, from the second tile.

Claims

1. A computer-implemented method for receiving and displaying a panoramic video, the method comprising:

receiving from a content provider at least a portion of a first tile selected from a plurality of tiles of the panoramic video, wherein each tile of the plurality of tiles is a video having a tile field-of-view that is a portion of a full field-of-view of the panoramic video, and wherein each tile field-of-view overlaps a tile field-of-view of at least one adjacent tile;

displaying an active field-of-view from the first tile, wherein the active field-of-view is selected from a first tile field-of-view of the first tile, wherein the active field-of-view is a subset of the tile field-of-view;

receiving from a user a pan command indicating a pan direction in which the active field-of-view is to be translated;

determining a buffer navigation position in a second tile selected from the plurality of tiles, wherein a second tile field-of-view of the second tile is adjacent to the first tile field-of-view of the first tile in the pan direction, and wherein the buffer navigation position indicates a first frame in the second tile that can be displayed after the second tile is buffered;

sending to the content provider a request for the second tile, wherein the request comprises an indication of the buffer navigation position;

receiving from the content provider a portion of the second tile beginning from the buffer navigation position;

buffering the second tile, wherein buffering the second tile comprises downloading a threshold number of frames of the second tile ahead of the buffer navigation position;

determining a pan rate limit, wherein the pan rate limit is a first rate at which the active field-of-view can be translated in the pan direction without reaching an edge of the first tile field-of-view before buffering of the second tile is complete;

translating the active field-of-view in the pan direction at a second rate less than or equal to the pan rate limit, wherein the active field-of-view reaches the edge of the first tile field-of-view; and

after the active field-of-view reaches the edge of the first tile field-of-view, displaying the active field-of-view from the second tile.

2. The method of claim 1, wherein determining the buffer navigation position comprises:

determining a buffer time for the second tile, wherein the buffer time indicates an amount of time to download the threshold number of frames;

determining a current navigation position indicating a currently displayed frame of the first tile; and

determining a frame of the first tile that would be displayed at a first time equal to a current time plus the buffer time if the active field-of-view were to continue being displayed from the first tile; and

identifying the first frame in the second tile as equivalent to the frame of the first tile.

3. The method of claim 1, wherein displaying the active field-of-view from the first tile comprises displaying a first portion of the first tile field-of-view, and wherein translating the active field-of-view in the pan direction before the active field-of-view reaches the edge of the first tile field-of-view comprises displaying a second portion of the first tile field-of-view different than the first portion of the first tile field-of-view.

4. The method of claim 1, wherein determining the pan rate limit comprises:

determining a first number of pixels between a current position of the active field-of-view and the edge of the first tile field-of-view;

determining a time to complete the buffering of the second tile; and

determining a second number of pixels per unit time at which the active field-of-view can be translated in the pan direction without reaching the edge of the first tile field-of-view before buffering of the second tile is complete.

5. A computer-implemented method for receiving and displaying a panoramic video, the method comprising:

receiving from a content provider at least a portion of a first tile selected from a plurality of tiles of the panoramic video, wherein each tile of the plurality of tiles is a video having a tile field-of-view that is a portion of a full field-of-view of the panoramic video;

displaying an active field-of-view from the first tile, wherein the active field-of-view is selected from a first tile field-of-view of the first tile, wherein the active field-of-view comprises a subset of the first tile field-of-view;

receiving from a user a translate command indicating a translate direction for the active field-of-view;

sending to the content provider a request for a second tile selected from the plurality of tiles, wherein a second tile field-of-view of the second tile is adjacent to the first tile field-of-view in the translate direction;

receiving a portion of the second tile;

translating the active field-of-view in the translate direction, wherein a leading edge of the active field-of-view along the translate direction reaches an edge of the first tile field-of-view; and

displaying the active field-of-view at least in part from the second tile.

6. The method of claim 5, wherein displaying the active field-of-view at least in part from the second tile comprises:

displaying a first portion of the active field-of-view from the first tile; and

displaying a second portion of the active field-of-view from the second tile.

7. The method of claim 5, further comprising determining a buffer navigation position in the second tile, wherein the portion of the second tile begins from the buffer navigation position.

8. The method of claim 7, wherein determining the buffer navigation position in the second tile further comprises:

determining a time to download a threshold amount of the portion of the second tile; and

adding the time to download the threshold amount of the portion of the second tile to a current navigation position of the first tile.

9. The method of claim 5, further comprising determining a rate at which the active field-of-view can be translated in the translate direction without reaching the edge of the first tile field-of-view before a threshold amount of the portion of the second tile is downloaded.

10. The method of claim 5, further comprising:

displaying a thumbnail version of the panoramic video, wherein the thumbnail version has a thumbnail field-of-view greater than the active field-of-view; and

playing an audio track synchronized to the thumbnail version of the panoramic video.

11. The method of claim 5, further comprising playing an audio track synchronized to a frame counter indicating a current navigation position of the first tile.

12. The method of claim 5, wherein an overlap of the first tile field-of-view and the second tile field-of-view is at least a width of the active field-of-view.

13. The method of claim 5, further comprising receiving an indication of an initial active field-of-view.

14. A computer-implemented method for providing a panoramic video for display, the method comprising:

receiving a request from a client device for the panoramic video, wherein the panoramic video comprises a plurality of tiles, and wherein each tile of the plurality of tiles is a video having a tile field-of-view that is a first portion of a full field-of-view of the panoramic video;

sending to the client device at least a second portion of a first tile selected from the plurality of tiles;

receiving from the client device a translate command indicating a translate direction for an active field-of-view, wherein the active field-of-view indicates a third portion of a first tile field-of-view displayed from the first tile at the client device;

determining a buffer navigation position in a second tile selected from the plurality of tiles, wherein a second tile field-of-view of the second tile is adjacent to the first tile field-of-view of the first tile in the translate direction;

sending to the client device a fourth portion of the second tile beginning at the buffer navigation position;

determining a rate at which the active field-of-view can be translated in the translate direction without reaching an edge of the first tile field-of-view before the active field-of-view can be displayed from the second tile; and

sending the rate to the client device.

15. The method of claim 14, further comprising:

receiving translate commands from a plurality of client devices including the client device;

determining an aggregated field-of-view based on the translate commands; and

transmitting the aggregated field-of-view to the client device.

16. The method of claim 14, further comprising transmitting to the client device at least a fifth portion of a thumbnail version of the panoramic video, wherein the thumbnail version of the panoramic video has a thumbnail field-of-view greater than the first tile field-of-view and wherein a first resolution of the thumbnail version of the panoramic video is less than a second resolution of the panoramic video.

17. The method of claim 16, wherein a distribution of color saturation levels of the thumbnail version indicates a distribution of fields-of-view selectable from the full field-of-view.

18. A computer-implemented method for providing a panoramic video for display, the method comprising:

transmitting to a client device at least a first portion of a first tile of the panoramic video, wherein the panoramic video comprises a plurality of tiles, and wherein each tile of the plurality of tiles is a video having a tile field-of-view that is a second portion of a full field-of-view of the panoramic video;

determining a translate command indicating a translate direction for an active field-of-view, wherein the active field-of-view indicates a third portion of a first tile field-of-view displayed from the first tile at the client device, wherein the translate command indicates the translate direction and a translate rate for the active field of view to translate to match a predetermined active field-of-view;

sending the translate command to the client device; and

sending the client device a fourth portion of a second tile selected from the plurality of tiles, wherein a second tile field-of-view is adjacent to the first tile field-of-view in the translate direction.

19. The method of claim 18, further comprising selecting a buffer navigation position for the second tile, wherein the fourth portion of the second tile sent to the client device begins at the buffer navigation position.

20. The method of claim 18, further comprising:

receiving from a second client device a first indication of a position of an active field-of-view displayed at the second client device;

receiving from a third client device a second indication of an active field-of-view displayed at the third client device; and

aggregating at least the active field-of-view at the second client device and the active field-of-view at the third client device to generate the predetermined active field-of-view.

21. The method of claim 20, wherein the indication of the active field-of-view at the second client device comprises at least one translate command received by the second client device.