Techniques for video data encoding

• US 9,497,487 B1
• Filed: 12/05/2014
• Issued: 11/15/2016
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Higa, Anthony Norihisa
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Cattungal, Rowina
• Application Number:
  US14/562,476
• Time in Patent Office: 711 Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Abstract

The techniques for encoding video content are disclosed. In an online game environment, the techniques include obtaining information for a first and second successive frames of video content and information for a position of a virtual camera associated with each frame, determining virtual camera translation information based on the positions of the virtual camera, determining a projected movement between the frames of an object included in each frame, determining the portion of the first frame to be excluded from the second frame and a new portion of the second frame, and providing the determined encoded information for a reconstruction of the second frame based on the provided information.

Claims

1. A computer-implemented method for encoding video content in a video game environment, comprising:

obtaining, by one or more computer systems, a first frame of a plurality of frames of video content, the first frame including information associated with a first position of a virtual camera and at least one object within the first frame;

obtaining a second frame of the plurality of frames of video content immediately succeeding the first frame in the plurality of frames, the second frame including information associated with a second position of the virtual camera;

determining one or more portions of partial overlap between the first frame and the second frame based at least in part on virtual camera translation information, the virtual camera translation information reflecting movement of the virtual camera between the first and second positions;

encoding video with the one or more portions of partial overlap from the first frame and the second frame based upon the virtual camera translation information and object translation information, the object translation information based at least in part upon a projected movement of the at least one object between the first frame and the second frame and upon the virtual camera translation information;

determining a frame quality value based on at least the virtual camera translation information;

determining whether the frame quality value is above a threshold;

when the frame quality value is above the threshold, transmitting the first frame as the second frame; and

when the frame quality value is below the threshold, generating data for frame encoding based at least in part on the projected movement of the at least one object.

2. The computer-implemented method of claim 1, further comprising:

reconstructing the plurality of frames of video content based at least in part on the generated data for new frame encoding.

3. The computer-implemented method of claim 1, further comprising:

determining a change in color or a change in shape of the at least one object between the first and second frames and including the determined change in the at least one object translation information.

4. The computer-implemented method of claim 1, wherein the decoder is associated with a client computing device configured to render reconstructed frames to a user.

5. The computer-implemented method of claim 1, wherein the virtual camera translation information reflects the coordinates of the virtual camera in a 3D space defined by each frame and a direction of the camera in the 3D space defined by each frame.

6. The computer-implemented method of claim 1, further comprising:

determining pixel velocity values based on the first and second positions of the virtual camera and assigning the determined pixel velocity values to the first portion of the first frame.

7. A computer-implemented method for encoding video content comprising a plurality of successive frames, the method comprising:

obtaining, by one or more computer systems, a first data set defining a first frame of video content, and a second data set defining a second frame of video content immediately succeeding the first frame;

determining virtual camera translation information based on a first position of a virtual camera associated with the first frame and a second position of the virtual camera associated with the second frame;

determining information defining a first portion of the first frame that overlaps with the second frame, a second portion of the first frame that does not overlap with the second frame, and a first portion of the second frame that does not overlap with the first frame, the information being determined based at least in part on the virtual camera translation information;

encoding, by the one or more computer systems, video with the determined information;

determining a frame quality value based on at least the virtual camera translation information;

determining whether the frame quality value is above a threshold;

when the frame quality value is above the threshold, transmitting the first frame as the second frame; and

when the frame quality value is below the threshold, generating data for frame encoding based at least in part on camera or object movement information.

8. The computer-implemented method of claim 7, wherein determining information defining the first subset of the first data set includes assigning pixel velocity values to the first subset, the pixel velocity values being derived from the virtual camera translation information.

9. The computer-implemented method of claim 7, wherein the first position of the virtual camera comprises data defining a first viewpoint of the virtual camera that corresponds to the first frame.

10. The computer-implemented method of claim 7, wherein the second position of the virtual camera comprises data defining a second viewpoint of the virtual camera that corresponds to the second frame.

11. The computer-implemented method of claim 9, wherein the data defining the first viewpoint comprises coordinates of the virtual camera in a 3D space defined by the first frame and a direction of the camera in the 3D space.

12. The computer-implemented method of claim 10, wherein the data defining the second viewpoint comprises coordinates of the virtual camera in a 3D space defined by the second frame and a direction of the camera in the 3D space.

13. The computer-implemented method of claim 7, further comprising:

determining that the first frame is the same as the second frame when the overlap between the first and second frames is determined to be above a first predetermined threshold.

14. The computer-implemented method of claim 7, further comprising:

transmitting entire second frame of video content to the decoder when the overlap between the first and second boundaries is determined to be below a second predetermined threshold.

15. A computer system for encoding video content comprising a plurality of frames in a video game environment, the system comprising a processor and a memory having computer-executable instructions that, when executed on the processor, cause the processor to:

obtain a first data set defining a first frame of video content, the first data set including a subset defining at least one object within the first frame;

determine a movement of the at least one object between at least two frames preceding the second frame;

determine object translation information based on the determined projected movement, the at least one object translation information defining a location of the at least one object in the second frame; and

encode video with the determined object translation information,

determine a frame quality value based on at least the virtual camera translation information;

determine whether the frame quality value is above a threshold;

when the frame quality value is above the threshold, transmit the first frame as the second frame; and

when the frame quality value is below the threshold, generate data for frame encoding based at least in part on the movement of the at least one object.

16. The computer system of claim 15, wherein the computer-executable instructions that cause the processor to determine object translation information further cause the processor to derive velocity values from a projected of the at least one object between the first frame and a second frame that is subsequent to the first frame based at least in part on and values defining the change of shape of the at least one object and to assign the derived values to the subset defining the at least one object.

17. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to reconstruct the plurality of frames of video content based at least in part on the generated data for frame encoding.

18. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to determine a change in size or a change in color of the at least one object between the first and second frames and to include the determined change in the object translation information.

19. The computer system of claim 16, wherein the computer-executable instructions further cause the processor to determine a difference between the first and second frames based on the determined projected movement and change of shape of the object and to provide information for the determined difference to the decoder for reconstruction of the second frame.

20. The computer system of claim 19, wherein the computer-executable instructions that cause the processor to determine a difference between the first and second frames based on the determined projected movement and change of shape of the object further cause the computer to determine a space occupied by the object in the first frame and vacated by the object in the second frame and to determine data needed to fill the vacated space in the second frame.

21. The computer system of claim 15, wherein the computer-executable instructions further cause the processor to determine a projected change of shape of the at least one object between the first frame and the second frame based at least in part on the movement of the at least one object between the at least two frames preceding the second frame and include the projected change of shape of the at least one object in the object translation information.

22. A non-transitory computer-readable storage medium having computer-executable instructions for encoding video content comprising a plurality of frames recorded by a camera in real time stored thereon that, when executed by a computer, cause the computer to:

obtain information for a first frame of video content and a second frame of video content immediately succeeding the first frame;

obtain camera translation information based on a movement of a camera between the first and second frames;

determine information indicating a difference between the first and second frames based on the camera translation information; and

encode video with the determined information,

determine a frame quality value based on at least the camera translation information;

determine whether the frame quality value is above a threshold;

when the frame quality value is above the threshold, transmit the first frame as the second frame; and

when the frame quality value is below the threshold, generate data for frame encoding based at least in part on the movement of the camera.

23. The non-transitory computer-readable storage medium of claim 22, wherein the instructions that cause the computer to obtain camera translation information further cause the computer to determine a difference between the coordinates of the camera associated with the first frame and the coordinates of the camera associated with the second frame and a difference between the camera direction associated with the first frame and the camera direction associated with the second frame.

24. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates for each frame are derived from the camera'"'"'s physical location in a scene based on a position of one or more markers inserted in the scene.

25. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates and direction for each frame are captured by a computer component associated with the camera.

26. The non-transitory computer-readable storage medium of claim 23, wherein the camera direction for each frame is supplied by an angle-measuring device associated with the camera.

27. The non-transitory computer-readable storage medium of claim 26, wherein the angle-measuring device is a gyroscope.

28. The non-transitory computer-readable storage medium of claim 23, wherein the camera coordinates and direction are associated with information for each frame.