METHOD AND SYSTEM OF ROTATION OF VIDEO FRAMES FOR DISPLAYING A VIDEO

• US 20170187983A1
• Filed: 12/26/2015
• Published: 06/29/2017
• Est. Priority Date: 12/26/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: LAWRENCE, SEAN J.
• Granted Patent: US 9,961,297 B2
• Time in Patent Office: Days
• CPC Class Codes: G11B 27/36 Monitoring, i.e. supervising the progress of recording or reproducing for digital recording G11B20/00 and s.gr.; for monitoring, testing or measuring of TV recorders of the type covered by H04N5/76 and subgroups, see H04N17/06
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/92 Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A system, article, and method of rotation of video frames for displaying a video.

Claims

1. A computer-implemented method of rotation of video frames for displaying a video, comprising:

obtaining frames of a video sequence and a plurality of device orientation indicators each associated with a different frame of the video sequence and that indicate an orientation of the frame while the frame was recorded;

determining a view orientation of the video sequence indicating an orientation of a device using, or to be used, to display the video sequence; and

determining a frame of the video sequence is to be rotated when the device orientation of the frame is different than the view orientation, and to display both rotated and non-rotated frames of the video sequence according to the view orientation.

2. The method of claim 1 wherein the available orientations are landscape and portrait.

3. The method of claim 1 wherein the individual device orientation indicators are permitted to differ from frame to frame in the video sequence.

4. The method of claim 1 wherein the rotated frames provide an image that is smaller than an entire view area of a frame so that a margin is formed at the unoccupied portion(s) of the view area.

5. The method of claim 4 wherein the margin is provided with at least one of a background color and background luminance.

6. The method of claim 4 comprising providing a margin to non-rotated frames so that the rotated frames and non-rotated frames show images of substantially the same size.

7. The method of claim 4 comprising extending the scene of at least one rotated image onto the margins so that an image substantially fills the view area.

8. The method of claim 7 comprising using the image data of at least one other frame of the video sequence to determine the image data to be used to extend the rotated frame image onto the margin(s) of the frame.

9. The method of claim 8 comprising using interpolation from a plurality of other frames in the video sequence to determine the image data to extend the rotated frame image onto the margin(s) of the frame.

10. The method of claim 1 comprising obtaining the individual device orientation indicators embedded in data transmitted in a bitstream and data respectively associated with individual frames of the video sequence.

11. The method of claim 1 wherein the device orientation indicator is a single bit indicating whether the device orientation during recording of a frame is in a landscape orientation or portrait orientation.

12. The method of claim 1 comprising determining which part of a rotated frame to display in the view orientation when the entire rotated frame will not fit on a screen of a device playing the video sequence at the view orientation.

13. The method of claim 12 wherein the determining of which part of a rotated frame to display comprises determining an overlap of the rotated frame and at least one other frame in the video sequence wherein the overlap is to be displayed on the rotated frame.

14. The method of claim 12 comprising performing at least one of object detection and multi-frame tracking to determine objects on the rotated frame similar to objects on other frames of the video sequence and that should be kept in the part of the rotated frame to be displayed.

15. The method of claim 1 wherein the available orientations are landscape and portrait;

wherein the device orientation indicator is a single bit indicating whether the device orientation during recording of a frame is in a landscape orientation or portrait orientation;

wherein the individual device orientation indicators are permitted to differ from frame to frame in the video sequence

wherein the rotated frames provide an image that is smaller than an entire view area of a frame so that a margin is formed at the unoccupied portion(s) of the view area;

the method comprising at least one of;

(1) providing the margin with at least one of a background color and background luminance, and providing a margin to non-rotated frames so that the rotated frames and non-rotated frames show images of substantially the same size; and

(2) extending the scene of at least one rotated image onto the margins so that an image substantially fills the view area;

using the image data of at least one other frame of the video sequence to determine the image data to be used to extend the rotated frame image onto the margin(s) of the frame; and

using interpolation from a plurality of other frames in the video sequence to determine the image data to extend the rotated frame image onto the margin(s) of the frame;

the method comprising;

retrieving the device orientation indicator from the metadata of individual frames;

obtaining the individual device orientation indicators embedded in data transmitted in a bitstream and data respectively associated with individual frames of the video sequence;

determining which part of a rotated frame to display in the view orientation when the entire rotated frame will not fit on a screen of a device playing the video sequence at the view orientation, wherein the determining of which part of a rotated frame to display comprises determining an overlap of the rotated frame and at least one other frame in the video sequence wherein the overlap is to be displayed on the rotated frame; and

performing at least one of object detection and multi-frame tracking to determine objects on the rotated frame similar to objects on other frames of the video sequence and that should be kept in the part of the rotated frame to be displayed.

16. A computer-implemented system of rotation of video frames for displaying a video, comprising:

at least one display;

at least one memory;

at least one processor communicatively coupled to the display and the memory; and

a frame rotation unit operated by the at least one processor and to;

obtain frames of a video sequence and a plurality of device orientation indicators each associated with a different frame of the video sequence and that indicate an orientation of the frame while the frame was recorded;

determine a view orientation of the video sequence indicating an orientation of a device using, or to be used, to display the video sequence; and

determine a frame of the video sequence is to be rotated when the device orientation of the frame is different than the view orientation, and to display both rotated and non-rotated frames of the video sequence according to the view orientation.

17. The system of claim 16 wherein the frame rotation unit is arranged to obtain the individual device orientation indicators embedded in data transmitted in a bitstream and data respectively associated with individual frames of the video sequence wherein the individual device orientation indicators are permitted to differ from frame to frame in the video sequence.

18. The system of claim 16 wherein the rotated frames provide an image that is smaller than an entire view area of a frame so that at least one margin is formed at the unoccupied portion(s) of the view area.

19. The system of claim 18 wherein the margin is provided with at least one of a background color and background luminance.

20. The system of claim 18 comprising a pre-rendering processing unit to provide a margin to non-rotated frames so that the rotated frames and non-rotated frames show images of substantially the same size.

21. The system of claim 18 comprising a pre-rendering processing unit to extend the scene of at least one rotated image onto the margins so that an image substantially fills the view area, and uses the image data of at least one other frame of the video sequence to determine the image data to be used to extend the rotated frame image onto the margin(s) of the frame.

22. The system of claim 16 comprising a pre-rendering processing unit to determine which part of a rotated frame to display in the view orientation when the entire rotated frame will not fit on a screen of a device playing the video sequence at the view orientation, and

by determining an overlap of the rotated frame and at least one other frame in the video sequence wherein the overlap is to be displayed on the rotated frame.

23. The system of claim 16 wherein the available orientations are landscape and portrait;

wherein the device orientation indicator is a single bit indicating whether the device orientation during recording of a frame is in a landscape orientation or portrait orientation;

wherein the individual device orientation indicators are permitted to differ from frame to frame in the video sequence

wherein the rotated frames provide an image that is smaller than an entire view area of a frame so that a margin is formed at the unoccupied portion(s) of the view area;

the system comprising a pre-rendering processing unit being arranged to at least one of;

(1) provide the margin with at least one of a background color and background luminance, and provide a margin to non-rotated frames so that the rotated frames and non-rotated frames show images of substantially the same size; and

(2) extend the scene of at least one rotated image onto the margins so that an image substantially fills the view area;

use the image data of at least one other frame of the video sequence to determine the image data to be used to extend the rotated frame image onto the margin(s) of the frame; and

use interpolation from a plurality of other frames in the video sequence to determine the image data to extend the rotated frame image onto the margin(s) of the frame;

the frame rotation unit being arranged to;

retrieve the device orientation indicator from the metadata of individual frames;

obtain the individual device orientation indicators embedded in data transmitted in a bitstream and data respectively associated with individual frames of the video sequence; and

the pre-rendering processing unit being arranged to;

determine which part of a rotated frame to display in the view orientation when the entire rotated frame will not fit on a screen of a device playing the video sequence at the view orientation, wherein the determining of which part of a rotated frame to display comprises determining an overlap of the rotated frame and at least one other frame in the video sequence wherein the overlap is to be displayed on the rotated frame; and

perform at least one of object detection and multi-frame tracking to determine objects on the rotated frame similar to objects on other frames of the video sequence and that should be kept in the part of the rotated frame to be displayed.

24. A computer-readable medium having stored thereon instructions that when executed cause a computing device to:

obtain frames of a video sequence and a plurality of device orientation indicators each associated with a different frame of the video sequence and that indicate an orientation of the frame while the frame was recorded;

determine a view orientation of the video sequence indicating an orientation of a device using, or to be used, to display the video sequence; and

determine a frame of the video sequence is to be rotated when the device orientation of the frame is different than the view orientation, and to display both rotated and non-rotated frames of the video sequence according to the view orientation.

25. The computer-readable medium of claim 24 wherein the available orientations are landscape and portrait;

wherein the device orientation indicator is a single bit indicating whether the device orientation during recording of a frame is in a landscape orientation or portrait orientation;

wherein the individual device orientation indicators are permitted to differ from frame to frame in the video sequence

wherein the rotated frames provide an image that is smaller than an entire view area of a frame so that a margin is formed at the unoccupied portion(s) of the view area;

the instructions causing the computing device to at least one of;

(1) provide the margin with at least one of a background color and background luminance, and provide a margin to non-rotated frames so that the rotated frames and non-rotated frames show images of substantially the same size; and

(2) extend the scene of at least one rotated image onto the margins so that an image substantially fills the view area;

use the image data of at least one other frame of the video sequence to determine the image data to be used to extend the rotated frame image onto the margin(s) of the frame; and

use interpolation from a plurality of other frames in the video sequence to determine the image data to extend the rotated frame image onto the margin(s) of the frame;

the instructions causing the computing device to;

retrieve the device orientation indicator from the metadata of individual frames;

obtain the individual device orientation indicators embedded in data transmitted in a bitstream and data respectively associated with individual frames of the video sequence;

determine which part of a rotated frame to display in the view orientation when the entire rotated frame will not fit on a screen of a device playing the video sequence at the view orientation, wherein the determining of which part of a rotated frame to display comprises determining an overlap of the rotated frame and at least one other frame in the video sequence wherein the overlap is to be displayed on the rotated frame; and

perform at least one of object detection and multi-frame tracking to determine objects on the rotated frame similar to objects on other frames of the video sequence and that should be kept in the part of the rotated frame to be displayed.