Automatic adjustment of video orientation

• US 10,469,797 B2
• Filed: 09/11/2017
• Issued: 11/05/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Nowakowski, Maciek S., Szabo, Balazs
• Primary Examiner(s): Wang, Yi
• Application Number:
  US15/701,403
• Publication Number:
  US 20170374314A1
• Time in Patent Office: 785 Days
• Field of Search: None
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407

Abstract

Methods, systems and computer program products for automatic adjustment of video orientation are provided. A computer-implemented method may include receiving a video comprising a plurality of image frames, determining that shaking of a mobile device has occurred during recording of the video, determining a baseline alignment for the video, adjusting the video in view of the baseline alignment, and providing a user interface comprising the video player to present a preview of the adjusted video to a user on the mobile device.

Claims

1. A computer-implemented method comprising:

receiving, by a processor of a mobile device of a user, a video recorded by a video recording device of the mobile device of the user, the video comprising a plurality of image frames;

determining, by the processor of the mobile device, that shaking of the mobile device has occurred during recording of the video by the video recording device of the mobile device, wherein the shaking is determined for the video based on data stored by the video recording device of the mobile device for the plurality of image frames of the video recorded by the video recording device of the mobile device, the data comprising position information for a corresponding image frame of the video recorded by the video recording device of the mobile device, the position information being indicative of shaking;

notifying the user of the shaking of the mobile device that occurred during the recording of the video;

receiving, via a user interface, a user request to initiate an automatic correction of the video that was recorded by the video recording device of the mobile device that was shaken during the recording;

performing the automatic correction of the video, comprising;

determining, by the processor of the mobile device, a baseline alignment for the video recorded by the video recording device of the mobile device; and

adjusting, by the processor of the mobile device, the video recorded by the video recording device of the mobile device in view of the baseline alignment, wherein the adjusting comprises automatically adjusting one or more of the plurality of image frames in the video to the baseline alignment to correct shaking; and

presenting the user interface providing a playback of a preview of the adjusted video, and also providing, alongside the playback of the preview of the adjusted video, a playback of the video originally recorded by the video recording device of the mobile device.

2. The computer-implemented method of claim 1, wherein determining that the shaking of the mobile device has occurred during recording of the video comprises:

comparing a video recording device position associated with a first image frame in the video to a second video recording device position associated with a second image frame in the video.

3. The computer-implemented method of claim 1, further comprising:

determining that an orientation change of the mobile device has occurred during recording of the video, wherein the orientation change is determined for the video based on data stored by the video recording device in each of the plurality of image frames of the video, the data comprising orientation information of a corresponding image frame of the video.

4. The computer-implemented method of claim 3, further comprising:

determining a standard orientation for the video, the standard orientation being determined based on a most common orientation detected throughout the video, and one or more parameters of a video player to present the video.

5. The computer-implemented method of claim 4, further comprising:

adjusting the video to the standard orientation.

6. The computer-implemented method of claim 5, wherein adjusting the video to the standard orientation further comprises:

determining a degree of rotation required to adjust the image frame to the standard orientation.

7. The computer-implemented method of claim 5, wherein adjusting the video to the standard orientation further comprises:

providing an interface to allow the user to adjust the video to the standard orientation.

8. The computer-implemented method of claim 5, wherein adjusting the video to the standard orientation further comprises:

automatically resizing one or more of the plurality of image frames in the video based on the standard orientation; and

automatically rotating one or more of the plurality of image frames in the video to the standard orientation.

9. A non-transitory computer readable medium having instructions stored thereon, that when executed by a processor, cause the processor to perform operations comprising:

receiving, at a mobile device of a user, a video recorded by a video recording device of the mobile device of the user, the video comprising a plurality of image frames;

determining, at the mobile device, that shaking of the mobile device has occurred during recording of the video by the video recording device of the mobile device, wherein the shaking is determined for the video based on data stored by the video recording device of the mobile device for the plurality of image frames of the video recorded by the video recording device of the mobile device, the data comprising position information for a corresponding image frame of the video recorded by the video recording device of the mobile device, the position information being indicative of shaking;

notifying the user of the shaking of the mobile device that occurred during the recording of the video;

receiving, via a user interface, a user request to initiate an automatic correction of the video that was recorded by the video recording device of the mobile device that was shaken during the recording;

performing the automatic correction of the video, comprising;

determining, at the mobile device, a baseline alignment for the video recorded by the video recording device of the mobile device; and

adjusting, at the mobile device, the video recorded by the video recording device of the mobile device in view of the baseline alignment, wherein the adjusting comprises automatically adjusting one or more of the plurality of image frames in the video to the baseline alignment to correct shaking; and

presenting the user interface providing a playback of a preview of the adjusted video, and also providing, alongside the playback of the preview of the adjusted video, a playback of the video originally recorded by the video recording device of the mobile device.

10. The non-transitory computer readable medium of claim 9, wherein determining that the shaking of the mobile device has occurred during recording of the video comprises:

comparing a video recording device position associated with a first image frame in the video to a second video recording device position associated with a second image frame in the video.

11. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

determining that an orientation change of the mobile device has occurred during recording of the video, wherein the orientation change is determined for the video based on data stored by the video recording device in each of the plurality of image frames of the video, the data comprising orientation information of a corresponding image frame of the video.

12. The non-transitory computer readable medium of claim 11, wherein the operations further comprise:

determining, by the processor, a standard orientation for the video, the standard orientation being determined based on a most common orientation detected throughout the video, and one or more parameters of a video player to present the video.

13. The non-transitory computer readable medium of claim 12, wherein the operations further comprise:

adjusting the video to the standard orientation.

14. The non-transitory computer readable medium of claim 13, wherein adjusting the video to the standard orientation further comprises:

determining a degree of rotation required to adjust the image frame to the standard orientation.

15. The non-transitory computer readable medium of claim 13, wherein adjusting the video to the standard orientation further comprises:

providing an interface to allow the user to adjust the video to the standard orientation.

16. A system comprising:

a memory; and

a processing device, coupled to the memory, to;

receive, at a mobile device of a user, a video recorded by a video recording device of the mobile device of the user, the video comprising a plurality of image frames;

determine, at the mobile device, that shaking of the mobile device has occurred during recording of the video by the video recording device of the mobile device, wherein the shaking is determined for the video based on data stored by the video recording device of the mobile device for the plurality of image frames of the video recorded by the video recording device of the mobile device, the data comprising position information for a corresponding image frame of the video recorded by the video recording device of the mobile device, the position information being indicative of shaking;

notify the user of the shaking of the mobile device that occurred during the recording of the video;

receive, via a user interface, a user request to initiate an automatic correction of the video that was recorded by the video recording device of the mobile device that was shaken during the recording;

perform the automatic correction of the video, wherein to perform the automatic correction of the video, the processing device is to;

determine a baseline alignment for the video recorded by the video recording device of the mobile device; and

adjust the video recorded by the video recording device of the mobile device in view of the baseline alignment, wherein the adjusting comprises automatically adjusting one or more of the plurality of image frames in the video to the baseline alignment to correct shaking; and

present the user interface providing a playback of a preview of the adjusted video, and also providing, alongside the playback of the preview of the adjusted video, a playback of the video originally recorded by the video recording device of the mobile device.

17. The system of claim 16, wherein to determine that the shaking of the mobile device has occurred during recording of the video, the processing device is to compare a video recording device position associated with a first image frame in the video to a second video recording device position associated with a second image frame in the video.