System and method for adjusting orientation of captured video

• US 9,516,229 B2
• Filed: 11/27/2012
• Issued: 12/06/2016
• Est. Priority Date: 11/27/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Xu, Wentao, Qiu, Gang, Zhang, Xiaopeng, Zhou, Xiaoming, Qurashi, Farrukh
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Segura, Cynthia
• Application Number:
  US13/686,466
• Publication Number:
  US 20140146205A1
• Time in Patent Office: 1,470 Days
• Field of Search: 348/208.2, 348/239
• US Class Current: 1/1
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

Described herein is a system and method for adjusting the orientation of captured video utilizing the data received from a built-in inertial measurement unit such as an accelerometer. During video capture, the device may be held in a position that is not fully vertical or horizontal, and thus not match the true orientation of a scene. This can cause the captured video to appear rotated during playback at the same angle of rotation as the video capture device. The described system can adjust the playback orientation of the rotated video by utilizing sensor data captured during the same time as the video was captured.

Claims

1. A method of encoding a video captured on a video capture device at two orientations, comprising:

reading orientation data for a first orientation from the video capture, device as a video of a scene is being captured, wherein the first orientation has a first resolution;

determining that the video capture device is rotated to a second orientation as the video of the scene is being captured, wherein the second orientation has a second resolution;

reading orientation data from the video capture device as a video of the scene is being captured when the video capture device is in the second orientation;

generating a rotation matrix based on a relationship between the first orientation and the second orientation;

adjusting the orientation of frames of the video captured when the video capture device is in the second orientation to match the orientation of the video captured when the video capture device is in the first orientation;

cropping the adjusted frames of the video captured when the video capture device is in the second orientation, the cropped frames having the first resolution; and

encoding the cropped frames of the captured video.

2. The method of claim 1, wherein the orientation data is inertial measurement data from an accelerometer or a gyroscope.

3. The method of claim 1, wherein the video capture device comprises a mobile phone or tablet.

4. The method of claim 1, further comprising storing the adjusted captured video to a memory in the video capture device.

5. The method of claim 1, wherein the rotated captured video is transmitted to another device with video display capability.

6. The method of claim 1, wherein the first orientation is a true orientation in which the acceleration of gravity aligns with a vertical axis of the of the video capture device.

7. The method of claim 1, wherein the rotation matrix includes a rotation angle determined by projecting gravity vectors of successive frames onto an X-Y plane of the video capture device.

8. The method of claim 7, wherein projecting the gravity vectors does not include projecting a Z-component of the gravity vector.

9. The method of claim 1, wherein the rotation matrix comprises a 2×

2 matrix.

10. A system for encoding a video capture on a video capture device at two orientations, comprising:

an image sensor configured to capture a video of a scene;

an inertial measurement unit comprising at least one of an accelerometer or a gyroscope configured to provide orientation data from the video capture device as a video of a scene is being captured when the video capture device is in a first orientation and provide orientation from the video capture device as a video of a scene is being captured when the video capture device is rotated to a second orientation, wherein the first orientation has a first resolution and the second orientation has a second resolution;

a processor executing a rotation module configured to;

generate a rotation matrix based on a relationship between the first orientation and the second orientation,

adjust the orientation of frames of the video captured when the video capture device is in the second orientation to match the orientation of the video captured when the video capture device is in the first orientation, and

crop the adjusted frames of the video captured when the video capture device is in the second orientation, the cropped frames having the first resolution; and

an encoder configured to encode the cropped frames of the captured video.

11. The system of claim 10, wherein the video capture device comprises a mobile phone or tablet.

12. The system of claim 10, further comprising a storage configured to store the adjusted captured video.

13. The system of claim 10, further comprising a transmitter for transmitting the adjusted captured video to a second device with video display capability.

14. A system of encoding a video captured on a video capture device at two orientations, comprising:

means for reading orientation data from the video capture device as a video of a scene is being captured when the video capture device is in a first orientation, wherein the first orientation has a first resolution;

means for determining that the video capture device is rotated to a second orientation as the video of the scene is being captured, wherein the second orientation has a second resolution;

means for reading orientation data from the video capture device as a video of the scene is being captured when the video capture device is in the second orientation;

means for generating a rotation matrix based on a relationship between the first orientation and the second orientation;

means for adjusting the orientation of frames of the video captured when the video capture device is in the second orientation to match the orientation of the video captured when the video capture device is in the first orientation;

means for cropping the adjusted frames of the video captured when the video capture device is in the second orientation, the cropped frames having the first resolution; and

means for encoding the cropped frames of the captured video.

15. The system of claim 14, wherein the means for reading orientation data comprises an inertial measurement unit configured to provide orientation data.

16. The system of claim 14, wherein the means for generating a rotation matrix based on the inertial measurement unit data comprises a rotation module configured to be operated by a processor.

17. The system of claim 14, wherein the means for encoding the rotated captured video comprises a video encoder.

18. A non-transient computer readable medium configured to store instructions that when executed by a processor perform a method comprising:

reading orientation data for a first orientation from a video capture device as a video of a scene is being captured, wherein the first orientation has a first resolution;

determining that the video capture device is rotated to a second orientation as the video of the scene is being captured, wherein the second orientation has a second resolution;

reading orientation data from the video capture device as a video of the scene is being captured when the video capture device is in the second orientation;

generating a rotation matrix based on a relationship between the first orientation and the second orientation;

adjusting the orientation of frames of the video captured when the video capture device is in the second orientation to match the orientation of the video captured when the video capture device is in the first orientation such that the video remains encoded at the first orientation after the video capture device moves to the second orientation;

cropping the adjusted frames of the video captured when the video capture device is in the second orientation, the cropped frames having the first resolution; and

encoding the cropped frames of the captured video.

19. The computer readable medium of claim 18, wherein the orientation data is inertial measurement data from an accelerometer or a gyroscope.

20. The computer readable medium of claim 18, wherein the orientation data is accelerometer data.