METHOD FOR PROCESSING CAPTURED VIDEO DATA BASED ON CAPTURE DEVICE ORIENTATION

• US 20160365117A1
• Filed: 10/09/2015
• Published: 12/15/2016
• Est. Priority Date: 06/11/2015
• Status: Abandoned Application

• Current Assignee: Martin Paul Boliek, Yaron Galant
• Original Assignee: Martin Paul Boliek, Yaron Galant
• Inventors: Boliek, Martin Paul, Galant, Yaron
• Application Number:
  US14/879,792
• Publication Number:
  US 20160365117A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/031 : Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/06 : Cutting and rejoining; Notching, or perforating record carriers otherwise than by recording styli record carriers with provision for splicing G11B23/20
• CPC Class Codes: G11B 27/105 : of operating discs
• CPC Class Codes: G11B 27/11 : by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/34 : Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: G11B 27/36 : Monitoring, i.e. supervising the progress of recording or reproducing for digital recording G11B20/00 and s.gr.; for monitoring, testing or measuring of TV recorders of the type covered by H04N5/76 and subgroups, see H04N17/06
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/91 : Television signal processing therefor
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

Methods for processing captured video data based on capture device orientation are described. In one embodiment, the method comprises capturing video data with a video capture device, detecting orientation of the video capture device, mapping pixels of the video data captured to a landscape orientation if the video capture device is in a portrait orientation, and displaying the video data on a screen of the video capture device in landscape orientation regardless of the orientation of the video capture device.

Claims

1. A method comprising:

a) capturing video data with a video capture device;

b) detecting orientation of the video capture device;

c) mapping pixels of the video data captured to a landscape orientation if the video capture device is in a portrait orientation; and

d) displaying the video data on a screen of the video capture device in landscape orientation regardless of the orientation of the video capture device.

2. The method defined in claim 1 further comprising:

detecting a change in the orientation from portrait to landscape while capturing video data; and

continuously mapping pixels of captured video data to a landscape orientation while the video capture device is in a landscape orientation.

3. The method defined in claim 2 further comprising downsampling captured video data to reduce a number of pixels in frames of the captured video data when capturing in landscape to match a number of pixels in frames of video data captured while the video capture device is in portrait orientation.

4. The method defined in claim 2 further comprising creating a zoomed out effect for captured video data in response to detecting the orientation has been changed from portrait to landscape, the zoomed out effect being based on use of a smaller angle of view when capturing video data with the video capture device in the portrait orientation than the angle of view when capturing video data with the video capture device in the landscape orientation.

5. The method defined in claim 1 further comprising upsampling captured video data to increase a number of pixels in frames of the captured video data to match a number of pixels in frames of video data captured while the video capture device is in landscape orientation.

6. The method defined in claim 1 wherein detecting orientation of the video capture device occurs while capturing the video data.

7. The method defined in claim 1 wherein the video capture device comprises a smart phone.

8. The method defined in claim 1 wherein if the orientation is portrait, then rotating and trimming video frames of the video data captured by the video capture device.

9. The method defined in claim 1 wherein if the orientation is landscape, then

determining whether to trim video frames based on a mode of the video capture device, and

trimming the video data if the mode of the video capture device is a first mode.

10. The method defined in claim 9 wherein, if in the first mode, then trimming the video data if the mode of the video capture device by reducing image resolution of the captured video data from N×

M to M×

P via downsampling, where N, M, and P are integers and N is a width of the captured video data prior to trimming and M is the height of the captured video data prior to trimming and N is greater than M, and M is the width of the captured video data after trimming and P is the height of the captured video data after trimming and M is greater than P.

11. The method defined in claim 1 further comprising:

detecting a change in the orientation from landscape to portrait while capturing video data; and

repeating c) based on the change in orientation.

12. The method defined in claim 1 further comprising digitally upsampling captured video data to increase resolution from M×

P to N×

M in response to detecting a change in orientation to landscape, where N, M and P are integers, and

for M×

P, M is the width of the captured video data and P is the height of the captured video data prior to upsampling and M is greater than P, and

for N×

M, N is a width of the captured video data and M is the height of the captured video data after upsampling and N is greater than M.

13. The method defined in claim 1 wherein the captured video data after trimming has an aspect ratio equal to the aspect ratio of the captured video prior to trimming.

14. The method defined in claim 1 further comprising, while capturing video data with a video capture device in a portrait orientation, displaying a preview of the captured video in a landscape perspective, wherein the preview has a size equal to a size of a portrait orientation preview.

15. The method defined in claim 1 further comprising:

capturing a landscape aspect ratio of a camera sensor of the video capture device oriented in portrait mode; and

preserving the landscape aspect ratio when the orientation is changed between landscape and portrait.

16. The method defined in claim 1 further comprising:

capturing a portrait aspect ratio of a camera sensor of the video capture device oriented in landscape mode; and

preserving the portrait aspect ratio when the orientation is changed between landscape and portrait.

17. The method defined in claim 1 further comprising:

displaying at least portions of the captured video data in a first orientation.

18. The method defined in claim 17 wherein the first orientation is user selected, by default or learned.

19. The method defined in claim 1 further comprising creating a zoomed in effect for a display of at least portions of captured video data, the zoomed in effect being based on a change from a full viewing angle in which video data is being captured in one orientation and a limited viewing angle in which the video data is being captured after a change in orientation.

20. The method defined in claim 1 wherein the landscape orientation is either landscape left or landscape right and the portrait orientation is either portrait up or portrait down.

21. The method defined in claim 1 further comprising:

converting at least a portion of captured video data to a predetermined orientation format; and

performing one or more image processing operations on the captured video data based on the predetermined orientation.

22. The method defined in claim 21 wherein capturing the video data occurs in real-time, and further comprising:

saving the captured video data in the predetermined orientation format in real-time.

23. The method defined in claim 21 further comprising:

displaying a preview of at least a portion of the captured video data in the predetermined orientation.

24. The method defined in claim 21 further comprising collecting metadata indicative of one or more of rotation, crop, resolution enhancement, and resolution reduction operations to be performed at playback and clip extraction time for the captured video data.

25. The method defined in claim 21 further comprising displaying a cropped portion of the captured video data to appear as if captured with a panning effect.

26. An article of manufacture having one or more non-transitory storage media storing instruction thereon which, when executed by a system, cause the system to perform a method comprising:

a) capturing video data with a video capture device;

b) detecting orientation of the video capture device;

c) mapping pixels of the video data captured to a landscape orientation if the video capture device is in a portrait orientation; and

d) displaying the video data on a screen of the video capture device in landscape orientation regardless of the orientation of the video capture device.

27. The article of manufacture defined in claim 26 wherein the method further comprises:

detecting a change in the orientation from portrait to landscape while capturing video data; and

continuously mapping pixels of captured video data to a landscape orientation while the video capture device is in a landscape orientation.

28. The article of manufacture defined in claim 27 wherein the method further comprises downsampling captured video data to reduce a number of pixels in frames of the captured video data when capturing in landscape to match a number of pixels in frames of video data captured while the video capture device is in portrait orientation.

29. The article of manufacture defined in claim 27 wherein the method further comprises creating a zoomed out effect for captured video data in response to detecting the orientation has been changed from portrait to landscape, the zoomed out effect being based on use of a smaller angle of view when capturing video data with the video capture device in the portrait orientation than the angle of view when capturing video data with the video capture device in the landscape orientation.

30. A method comprising:

capturing video data with a camera of a mobile phone while the mobile phone is in a portrait orientation;

generating a landscape window on the screen of the mobile phone; and

displaying in the landscape window a cropped and rotated version of the captured video so that the landscape window shows a zoomed and correctly oriented region of video images at a resolution equal to resolution of the captured video data had the video data been captured originally with the mobile phone in a landscape orientation.