Handling video frames compromised by camera motion

• US 9,407,823 B2
• Filed: 12/09/2013
• Issued: 08/02/2016
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khoury, Issa
• Primary Examiner(s): GILES, NICHOLAS G
• Application Number:
  US14/100,189
• Publication Number:
  US 20150163407A1
• Time in Patent Office: 967 Days
• US Class Current: 1/1
• CPC Class Codes: G11B 27/02 Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

Example embodiments concern reducing the negative impacts of camera motion during video capture. Processing, transmitting, or displaying motion compromised video data yields inefficient use of bandwidth, battery power, memory, and other resources. Watching video acquired from a moving video camera may induce motion sickness. Unlike conventional systems that attempt to address motion by increasing frame capture rates to produce better individual frames, example embodiments identify frames that are compromised by motion data and do not provide or present them for viewing. Compromised frames may be logically or physically deleted from a video stream or memory. Motion may be detected by physical apparatus (e.g., accelerometer) or through image processing (e.g., motion artifacts, low correlation between frames, low signal-to-noise ratio). A display device may be controlled to operate in a video display mode or a still frame mode depending on motion detected in captured video frames.

Claims

1. An apparatus, comprising:

a memory configured to store a set of frames acquired by a video camera; and

an interface configured to connect a processor and the memory, the processor to;

produce motion data that describes how the video camera was moving while the video camera acquired members of the set of frames, where the motion data is a function of gyroscopic data produced by the video camera, accelerometer data produced by the video camera, single frame motion analysis, or frame-to-frame motion analysis;

control whether members of the set of frames are displayed as a continuous video display or as selected still frames based, at least in part, on the motion data, where members of a first series of frames acquired while the motion data does not satisfy a motion threshold are presented in a first discrete display mode and where a second series of frames acquired while the motion data does satisfy the motion threshold is presented in a second continuous display mode; and

control whether members of the set of frames are displayed or not displayed based on a determination of whether each member exhibits less than a threshold amount of damage associated with camera motion, wherein the damage is to be calculated based on a signal-to-noise ratio.

2. The apparatus of claim 1, wherein the processor is to control a frame capture parameter for the video camera based, at least in part, on the motion data, where the frame capture parameter is frame capture rate or shutter speed.

3. The apparatus of claim 2, where the motion data comprises a per frame motion data or a per clip motion data.

4. The apparatus of claim 3, where the processor is to selectively exclude individual members of the set of frames based on the per frame motion data, to selectively exclude individual members of the set of frames based on the per clip motion data, or to selectively exclude subsets of members of the set of frames based on the per clip motion data.

5. The apparatus of claim 3, wherein the per clip motion data identifies frames that contribute to blurring.

6. The apparatus of claim 3, wherein the per clip motion data identifies a series of frames that do not satisfy a continuity standard.

7. The apparatus of claim 6, wherein the processor is to exclude the series of frames that do not satisfy the continuity standard.

8. The apparatus of claim 1, wherein the single frame motion analysis comprises identification of motion artifacts, a signal-to-noise ratio, or both.

9. The apparatus of claim 1, wherein the frame-to-frame motion analysis comprises an examination of a registration between objects, a continuity between the objects, or both.

10. The apparatus of claim 1, wherein the motion data reports apparent movement.

11. The apparatus of claim 1, wherein the memory is configured to store metadata associated with one or more frames of the set of frames.

12. The apparatus of claim 11, wherein the metadata comprises sensor data, motion analysis data, an include value, an exclude value, or any combination thereof.