Apparatus and method for panoramic video hosting

• US 9,843,840 B1
• Filed: 12/01/2016
• Issued: 12/12/2017
• Est. Priority Date: 12/02/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Banta, Bill, Donaldson, Geoff
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Mesa, Jose
• Application Number:
  US15/366,878
• Time in Patent Office: 376 Days
• Field of Search: 348 36
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4668 for recommending content, e.g. movies
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A server includes an input node to receive video streams forming a panoramic video. A module forms a suggested field of view in the panoramic video. An output node sends the suggested field of view to a client device.

Claims

1. A method comprising:

receiving, by at least one server, video data from a plurality of image sensors, including first video data captured by a first image sensor and second video data captured by a second image sensor, wherein the first video data is stitched together with the second video data in a panoramic video;

receiving, by the at least one server, data representing a plurality of fields of view from the panoramic video, and wherein a first field of view of the plurality of fields of view and a second field of view of the plurality of fields of view have been requested for display by at least a first client device;

determining that the first field of view was requested for display more times than the second field of view was requested for display;

selecting the first field of view as a suggested field of view, wherein the suggested field of view includes a viewing angle that is less than a full field of view angle of the panoramic video, and wherein the suggested field of view comprises at least a portion of the first video data and at least a portion of the second video data; and

sending a segment of the panoramic video from a viewpoint of the suggested field of view to the first client device.

2. The method of claim 1 wherein receiving the data representing the plurality of fields of view comprises receiving a field of view selection from the first client device.

3. The method of claim 1 wherein receiving the data representing the plurality of fields of view from the panoramic video comprises receiving a panning sequence from a second client device.

4. The method of claim 1 wherein selecting the first field of view as the suggested field of view comprises selecting as the suggested field of view a most popular field of view from the plurality of fields of view.

5. The method of claim 1, further comprising sending a buffer positioned around at least a portion of the segment of the panoramic video from the viewpoint of the suggested field of view to the first client device.

6. The method of claim 5, further comprising:

receiving, by the at least one server, an indication of a client device panning speed requested by the first client device; and

setting a size of the buffer.

7. The method of claim 1 wherein receiving the data representing the plurality of fields of view from the panoramic video comprises receiving from a second client device at least one current user center point within the panoramic video.

8. The method of claim 1 wherein receiving the data representing the plurality of fields of view from the panoramic video comprises receiving from a second client device at least one current user field of view within the panoramic video.

9. The method of claim 1, wherein the segment is a first segment and the viewpoint is a first viewpoint, the method further comprising:

sending a modified panoramic video to at least the first client device, wherein the modified panoramic video includes a second segment of the panoramic video from a second viewpoint outside the suggested field of view, and wherein the first segment of the panoramic video has a higher resolution in the suggested field of view than the resolution of the second segment from the second viewpoint outside the suggested field of view.

10. A computing device, comprising:

at least one processor;

a non-transitory computer-readable memory storing computer-executable instructions which when executed cause the at least one processor to perform a method comprising;

selecting a suggested field of view from a panoramic video, the panoramic video comprising first video data captured by a first sensor stitched together with second video data captured by a second sensor, wherein selecting the suggested field of view comprises;

receiving data representing a plurality of fields of view from the panoramic video, wherein a first field of view of the plurality of fields of view and a second field of view of the plurality of fields of view have been requested for display by at least a first client device;

determining that the first field of view was requested for display more times than the second field of view was requested for display; and

selecting the first field of view as the suggested field of view, wherein the suggested field of view includes a viewing angle that is less than a full field of view angle of the panoramic video, and wherein the suggested field of view comprises a first portion of the first video data and a second portion of the second video data; and

an output node effective to send to the first client device a segment of the panoramic video from a viewpoint of the suggested field of view.

11. The computing device of claim 10 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising receiving the data representing the plurality of fields of view comprises receiving a field of view selection from the first client device.

12. The computing device of claim 10 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising receiving the data representing the plurality of fields of view from the panoramic video comprises receiving a panning sequence from a second client device.

13. The computing device of claim 10 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising selecting the suggested field of view comprises selecting a most popular field of view from the plurality of fields of view.

14. The computing device of claim 10, wherein the output node is effective to send the segment to the first client device by sending to the client device the segment and a buffer positioned around at least a portion of the segment.

15. The computing device of claim 14 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising:

receiving an indication of a client device panning speed requested by the first client device; and

setting a size of the buffer based at least in part on the client device panning speed.

16. The computing device of claim 10 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising receiving from a second client device a current user center point within the panoramic video.

17. The computing device of claim 10 wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising receiving from a second client device a current user field of view within the panoramic video.

18. The computing device of claim 10, wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising modulating video delivered to the first client device based upon a dynamic assessment of available network bandwidth.

19. The computing device of claim 10, wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising sending a modified panoramic video to the client device, wherein the modified panoramic video has a higher resolution in the suggested field of view than outside the suggested field of view.

20. The computing device of claim 10, wherein the computer-executable instructions, when executed by the at least one processor are effective to perform the method further comprising:

determining that a third field of view of the plurality of fields of view was requested for display less times than at least the first field of view; and

reducing a data rate of a second segment of the panoramic video from a second viewpoint of the third field of view;

wherein the output node is further effective to send the second segment of the panoramic video from the second viewpoint to a second client device requesting the third field of view.