Techniques for video data encoding

• US 8,908,775 B1
• Filed: 03/30/2011
• Issued: 12/09/2014
• Est. Priority Date: 03/30/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Higa, Anthony Norihisa
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Cattungal, Rowina
• Application Number:
  US13/076,363
• Time in Patent Office: 1,350 Days
• Field of Search: 382/154, 382/284, 715/719, 345/420, 345/428
• US Class Current: 375/240.26
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Abstract

The techniques for encoding video content are disclosed. In an online game environment, the techniques include obtaining information for a first and second successive frames of video content and information for a position of a virtual camera associated with each frame, determining virtual camera translation information based on the positions of the virtual camera, determining a projected movement between the frames of an object included in each frame, determining the portion of the first frame to be excluded from the second frame and a new portion of the second frame, and providing the determined encoded information for a reconstruction of the second frame based on the provided information.

Claims

1. A computer-implemented method for encoding video content in a video game environment, comprising:

obtaining, by one or more computer systems, a first data set defining a first frame of a plurality of frames of video content, the first data set including information associated with a first position of a virtual camera and defining a first boundary of the first frame, the first data set further including a subset defining at least one object within the first frame;

obtaining a second data set defining a second frame of video content immediately succeeding the first frame in the plurality of frames, the second set of data including information associated with a second position of the virtual camera and defining a second boundary of the second frame;

determining a first portion of the first frame at least partially overlapping the second frame based at least in part upon virtual camera translation information, the virtual camera translation information reflecting movement of the virtual camera between the first and second positions;

determining a second portion of the first frame that does not overlap with the second frame and a third portion of the second frame that does not overlap with the first frame based on a difference between the first and second boundaries;

determining object translation information based at least in part upon a projected movement of the at least one object between the first frame and the second frame and upon the virtual camera translation information;

encoding, by the one or more computer systems, a video file with information defining the first portion of the first frame, the second portion of the first frame, and the third portion of the second frame based upon the virtual camera translation information and the object translation information

determining a frame degradation based on at least camera movement information;

determining whether the frame degradation is above a threshold;

when the frame degradation is above the threshold, transmitting the first frame as the second frame; and

when the frame degradation is below the threshold, generating data for new frame encoding based at least in part on the camera or object movement information and reconstructing the plurality of frames of video content based at least in part on the generated data for new frame encoding.

2. The computer-implemented method of claim 1, further comprising:

determining a change in shape of the at least one object between the first and second frames and including the determined change in the at least one object translation information.

3. The computer-implemented method of claim 1, further comprising:

determining a change in color of the at least one object between the first and second frames and including the determined change in the at least one object translation information.

4. The computer-implemented method of claim 1, wherein a decoder is associated with a client computing device configured to render reconstructed frames to a user.

5. The computer-implemented method of claim 1, wherein the virtual camera translation information reflects the coordinates of the virtual camera in a 3D space defined by each frame and a direction of the camera in the 3D space defined by each frame.

6. The computer-implemented method of claim 1, further comprising:

determining pixel velocity values based on the first and second positions of the virtual camera and assigning the determined pixel velocity values to the first portion of the first frame.

7. A computer-implemented method for encoding video content comprising a plurality of successive frames, the method comprising:

obtaining, by one or more computer systems, a first data set defining a first frame of video content, and a second data set defining a second frame of video content immediately succeeding the first frame;

determining virtual camera translation information based on a first position of a virtual camera associated with the first frame and a second position of the virtual camera associated with the second frame;

determining information defining a first and second subsets of the first data set, the first subset defining a first portion of the first frame that overlaps with the second frame, the second subset defining a second portion of the first frame that does not overlap with the second frame, the information defining the first subset being determined based on the virtual camera translation information;

determining information defining a subset of the second data set, the subset defining a first portion of the second frame that does not overlap with the first frame;

encoding, by the one or more computer systems, a video file with the determined information,

determining a frame degradation based on at least the virtual camera translation information;

determining whether the frame degradation is above a threshold;

when the frame degradation is above the threshold, transmitting the first frame as the second frame; and

when the frame degradation is below the threshold, generating data for new frame encoding based at least in part on camera or object movement information and reconstructing the plurality of frames of video content based at least in part on the generated data for new frame encoding.

8. The computer-implemented method of claim 7, wherein determining information defining the first subset of the first data set includes assigning pixel velocity values to the first subset, the pixel velocity values being derived from the virtual camera translation information.

9. The computer-implemented method of claim 7, wherein the first position of the virtual camera comprises data defining a first viewpoint of the virtual camera that corresponds to the first frame.

10. The computer-implemented method of claim 9, wherein the data defining the first viewpoint comprises coordinates of the virtual camera in a 3D space defined by the first frame and a direction of the camera in the 3D space.

11. The computer-implemented method of claim 7, wherein the second position of the virtual camera comprises data defining a second viewpoint of the virtual camera that corresponds to the second frame.

12. The computer-implemented method of claim 11, wherein the data defining the second viewpoint comprises coordinates of the virtual camera in a 3D space defined by the second frame and a direction of the camera in the 3D space.

13. The computer-implemented method of claim 7, further comprising:

determining that the first frame is the same as the second frame when the overlap between the first and second frames is determined to be above a first predetermined threshold.

14. The computer-implemented method of claim 7, further comprising:

transmitting entire second frame of video content to a decoder when the overlap between the first and second boundaries is determined to be below a second predetermined threshold.

15. A computer system for encoding video content comprising a plurality of frames in a video game environment, the system comprising a processor and a memory having computer-executable instructions that, when executed on the processor, cause the processor to:

obtain a first data set defining a first frame of video content, the first data set including a subset defining at least one object within the first frame;

determine a projected movement of the at least one object between the first frame and a second frame that is subsequent to the first frame based at least in part on a movement of the at least one object between at least two frames preceding the second frame;

determine object translation information based on the determined projected movement, the at least one object translation information defining a location of the at least one object in the second frame;

encoding a video file with the determined object translation information,

determining a frame degradation based on at least the virtual camera translation information;

determining whether the frame degradation is above a threshold;

when the frame degradation is above the threshold, transmitting the first frame as the second frame; and

when the frame degradation is below the threshold, generating data for new frame encoding based at least in part on camera or object movement information and reconstructing the plurality of frames of video content based at least in part on the generated data for new frame encoding.

16. The computer system of claim 15, wherein the computer-executable instructions that cause the processor to determine object translation information further cause the processor to derive velocity values from the projected movement and values defining the change of shape of the at least one object and to assign the derived values to the subset defining the at least one object.

17. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to determine a change in color of the at least one object between the first and second frames and to include the determined change in the object translation information.

18. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to determine a change in size of the at least one object between the first and second frames and to include the determined change in the object translation information.

19. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to determine a difference between the first and second frames based on the determined projected movement and change of shape of the object and to provide information for the determined difference to the decoder for reconstruction of the second frame.

20. The computer system of claim 19, wherein the computer-executable instructions that cause the processor to determine a difference between the first and second frames based on the determined projected movement and change of shape of the object further cause the computer to determine a space occupied by the object in the first frame and vacated by the object in the second frame and to determine data needed to fill the vacated space in the second frame.

21. The computer system of claim 15, wherein the computer-executable instructions further cause the processor to determine a projected change of shape of the at least one object between the first frame and the second frame based at least in part on the movement of the at least one object between the at least two frames preceding the second frame and include the projected change of shape of the at least one object in the object translation information.

22. A non-transitory computer-readable storage medium having computer-executable instructions for encoding video content comprising a plurality of frames recorded by a camera in real time stored thereon that, when executed by a computer, cause the computer to:

obtain information for a first frame of video content and a second frame of video content immediately succeeding the first frame;

obtain camera translation information based on a movement of a camera between the first and second frames;

determine information indicating a difference between the first and second frames based on the camera translation information;

encode a video file with the determined information,

determine a frame degradation based on at least the virtual camera translation information;

determine whether the frame degradation is above a threshold;

when the frame degradation is above the threshold, transmit the first frame as the second frame; and

when the frame degradation is below the threshold, generating data for new frame encoding based at least in part on camera or object movement information and reconstructing the plurality of frames of video content based at least in part on the generated data for new frame encoding.

23. The non-transitory computer-readable storage medium of claim 22, wherein the instructions that cause the computer to obtain camera translation information further cause the computer to determine a difference between the coordinates of the camera associated with the first frame and the coordinates of the camera associated with the second frame and a difference between the camera direction associated with the first frame and the camera direction associated with the second frame.

24. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates and direction are associated with information for each frame.

25. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates for each frame are derived from the camera'"'"'s physical location in a scene based on a position of one or more markers inserted in the scene.

26. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates and direction for each frame are captured by a computer component associated with the camera.

27. The non-transitory computer-readable storage medium of claim 23, wherein the camera direction for each frame is supplied by an angle-measuring device associated with the camera.

28. The non-transitory computer-readable storage medium of claim 27, wherein the angle-measuring device is a gyroscope.