CONTENT AWARE SELECTIVE ADJUSTING OF MOTION ESTIMATION

• US 20140192133A1
• Filed: 03/28/2012
• Published: 07/10/2014
• Est. Priority Date: 03/28/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Cheung, Kin-Hang, Liu, Ping
• Granted Patent: US 9,019,340 B2
• Time in Patent Office: Days
• US Class Current: 348/14.1
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085

Abstract

Systems, apparatus, articles, and methods are described including operations for content aware selective adjusting of motion estimation.

Claims

1-25. -25. (canceled)

26. A computer-implemented method for content aware selective adjusting of motion estimation, comprising:

receiving visual data, wherein the visual data includes video;

determining a amount of motion based at least in part on the received visual data; and

selectively adjusting a motion estimation applied to the received visual data based at least in part on the determined amount of motion.

27. The method of claim 26, further comprising performing facial detection for at least one of one or more users based at least in part on the received visual data.

28. The method of claim 26, further comprising:

performing facial detection for at least one of one or more users based at least in part on the received visual data; and

identifying a target user and a background area based at least in part on the performed facial detection,

wherein the determination of the amount of motion comprises determining the amount of motion based on the target user and not the background area.

29. The method of claim 26, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, and

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling.

30. The method of claim 26, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation,

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling, and

wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation.

31. The method of claim 26, further comprising:

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is an external power source.

32. The method of claim 26, further comprising:

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source and an indication that the battery charge level is below a charge threshold value.

33. The method of claim 26, further comprising:

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value.

34. The method of claim 26, further comprising:

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value.

35. The method of claim 26, further comprising:

performing facial detection for at least one of one or more users based at least in part on the received visual data; and

identifying a target user based at least in part on the performed facial detection,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

36. The method of claim 26, further comprising:

performing facial detection for at least one of one or more users based at least in part on the received visual data;

identifying a target user and a background area based at least in part on the performed facial detection, wherein the determination of the amount of motion comprises determining the amount of motion based on the target user and not the background area; and

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source,

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling, and wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

37. A system for content aware selective adjusting of motion estimation on a computer, comprising:

an imaging device configured to capture visual data;

one or more processors communicatively coupled to the imaging device;

one or more memory stores communicatively coupled to the one or more processors;

a visual data logic module communicatively coupled to imaging device and configured to receive visual data, wherein the visual data includes video;

a motion determination logic module communicatively coupled to the visual data logic module and configured to determine a amount of motion based at least in part on the received visual data;

a motion estimation adjustment logic module communicatively coupled to the motion determination logic module and configured to selectively adjust a motion estimation applied to the received visual data based at least in part on the determined amount of motion; and

a video encoder communicatively coupled to the motion estimation adjustment logic module and configured to encode the received video data based at least in part on the selectively adjusted motion estimation.

38. The system of claim 37, further comprising a facial detection logic module communicatively coupled to the visual data logic module and configured to perform facial detection for at least one of one or more users based at least in part on the received visual data.

39. The system of claim 37, further comprising:

a facial detection logic module communicatively coupled to the visual data logic module and configured to perform facial detection for at least one of one or more users based at least in part on the received visual data and identify a target user and a background area based at least in part on the performed facial detection, and

wherein the determination of the amount of motion comprises determining the amount of motion based on the target user and not the background area.

40. The system of claim 37, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, and

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling.

41. The system of claim 37, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation,

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling, and

wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation.

42. The system of claim 37, further comprising:

a power supply data logic module communicatively coupled to the one or more processors and the one or more memory stores and configured to receive power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is an external power source.

43. The system of claim 37, further comprising:

a power supply data logic module communicatively coupled to the one or more processors and the one or more memory stores and configured to receive power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source and an indication that the battery charge level is below a charge threshold value.

44. The system of claim 37, further comprising:

a power supply data logic module communicatively coupled to the one or more processors and the one or more memory stores and configured to receive power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value.

45. The system of claim 37, further comprising:

a power supply data logic module communicatively coupled to the one or more processors and the one or more memory stores and configured to receive power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value.

46. The system of claim 37, wherein the video encoder comprises at least a portion of one or more of the logic modules including the visual data logic module, the motion determination logic module, and the motion estimation adjustment logic module.

47. The system of claim 37, further comprising:

a facial detection logic module communicatively coupled to the visual data logic module and configured to perform facial detection for at least one of one or more users based at least in part on the received visual data and identify a target user based at least in part on the performed facial detection, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

48. The system of claim 37, further comprising:

a facial detection logic module communicatively coupled to the visual data logic module and configured to perform facial detection for at least one of one or more users based at least in part on the received visual data and identify a target user and a background area based at least in part on the performed facial detection, and wherein the determination of the amount of motion comprises determining the amount of motion based on the target user and not the background area; and

a power supply data logic module communicatively coupled to the one or more processors and the one or more memory stores and configured to receive power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source,

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling, and wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value,

wherein the video encoder comprises at least a portion of one or more of the logic modules including the visual data logic module, the motion determination logic module, the motion estimation adjustment logic module, the facial detection logic module, and the power supply data logic module,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

49. At least one machine readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to code data by:

receiving visual data, wherein the visual data includes video;

determining a amount of motion based at least in part on the received visual data; and

selectively adjusting a motion estimation applied to the received visual data based at least in part on the determined amount of motion.

50. The machine readable medium of claim 49, further comprising instructions that in response to being executed on the computing device, cause the computing device to operate by:

performing facial detection for at least one of one or more users based at least in part on the received visual data;

identifying a target user and a background area based at least in part on the performed facial detection, wherein the determination of the amount of motion comprises determining the amount of motion based on the target user and not the background area; and

receiving power supply data, wherein the power supply data includes an indication of a current power source and/or an indication of battery charge level, wherein the indication of the current power source indicates whether the current power source is a battery-type power source or an external power source,

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data where the second level of downscaling is greater than the first level of downscaling, and wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source is a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.