Content aware selective adjusting of motion estimation

• US 9,019,340 B2
• Filed: 03/28/2012
• Issued: 04/28/2015
• Est. Priority Date: 03/28/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Cheung, Kin-Hang, Liu, Ping
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US13/977,446
• Publication Number:
  US 20140192133A1
• Time in Patent Office: 1,126 Days
• Field of Search: 348/14.1, 348/452, 348/497, 375/240.02, 375/240.16, 713/176
• US Class Current: 348/14.1
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085

Abstract

Systems, apparatus, articles, and methods are described including operations for content aware selective adjusting of motion estimation.

Claims

1. A computer-implemented method for content aware selective adjusting of motion estimation, comprising:

receiving visual data from an imaging device, the received visual data comprising visual data of one or more users and a background area;

performing, at one or more processors, facial detection for at least one of the one or more users based at least in part on the received visual data;

identifying, at the one or more processors, a target user from the one or more users and the background area;

determining, at the one or more processors, an amount of motion associated with the target user and not the background area; and

selectively adjusting, at the one or more processors, a motion estimation applied to the received visual data based at least in part on the determined amount of motion.

2. The method of claim 1, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, and

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling.

3. The method of claim 1, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation,

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling, and

wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation.

4. The method of claim 1, further comprising:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source.

5. The method of claim 1, further comprising:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value.

6. The method of claim 1, further comprising:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value.

7. The method of claim 1, further comprising:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value.

8. The method of claim 1, wherein

identifying the target user is based at least in part on the performed facial detection for the at least one of the one or more users, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

9. The method of claim 1, wherein

identifying the target user and the background area is based at least in part on the performed facial detection for the at least one of the one or more users,

the method further comprising:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, the base level of motion estimation being associated with a first level of downscaled visual data and the enhanced level of motion estimation being associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling, and the base level of motion estimation comprising hierarchical motion estimation and the enhanced level of motion estimation comprising super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

10. A system for content aware selective adjusting of motion estimation on a computer, comprising:

an imaging device configured to capture visual data, the captured visual data comprising visual data of one or more users and a background area;

a visual data logic module communicatively coupled to imaging device and configured to receive the captured visual data;

a facial detection logic module communicatively coupled to the visual data logic module and configured to perform facial detection for at least one of one or more users based at least in part on the received visual data, the facial detection logic module further configured to identify a target user from the one or more users and a background area based at least in part on the performed facial detection;

a motion determination logic module communicatively coupled to the visual data logic module and configured to determine ana amount of motion associated with the target user and not the background area based at least in part on the received visual data;

a motion estimation adjustment logic module communicatively coupled to the motion determination logic module and configured to selectively adjust a motion estimation applied to the received visual data based at least in part on the determined amount of motion; and

a video encoder communicatively coupled to the motion estimation adjustment logic module and configured to encode the received video data based at least in part on the selectively adjusted motion estimation.

11. The system of claim 10, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, and

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling.

12. The system of claim 10, wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation,

wherein the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling, and

wherein the base level of motion estimation comprises hierarchical motion estimation and the enhanced level of motion estimation comprises super hierarchical motion estimation.

13. The system of claim 10, further comprising:

a power supply data logic module communicatively coupled to the motion estimation adjustment logic module and configured to receive power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicates whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source.

14. The system of claim 10, further comprising:

a power supply data logic module communicatively coupled to the motion estimation adjustment logic module and configured to receive power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicates whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value.

15. The system of claim 10, further comprising:

a power supply data logic module communicatively coupled to the motion estimation adjustment logic module and configured to receive power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicates whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value.

16. The system of claim 10, further comprising:

a power supply data logic module communicatively coupled to the motion estimation adjustment logic module and configured to receive power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicates whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value.

17. The system of claim 10, wherein the video encoder comprises at least a portion of one or more of the logic modules including the visual data logic module, the motion determination logic module, and the motion estimation adjustment logic module.

18. The system of claim 10,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

19. The system of claim 10, further comprising:

a power supply data logic module communicatively coupled to the motion estimation adjustment logic module and configured to receive power supply data, wherein the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicates whether the current power source comprises a battery-type power source or an external power source,

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, the base level of motion estimation being associated with a first level of downscaled visual data and the enhanced level of motion estimation beings associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling, and the base level of motion estimation comprising hierarchical motion estimation and the enhanced level of motion estimation comprising super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

20. An article comprising a machine-readable medium comprising a plurality of instructions that in response to being executed on a computing device result in:

receiving visual data from an imaging device, the received visual data comprising visual data of one or more users and a background area

performing, at one or more processors, facial detection for at least one of the one or more users based at least in part on the received visual data;

identifying, at the one or more processors, a target user from the one or more users and the background area;

determining, at the one or more processors, an amount of motion associated with the target user and not the background area; and

selectively, at the one or more processors, adjusting a motion estimation applied to the received visual data based at least in part on the determined amount of motion.

21. The article of claim 20, wherein the machine-readable medium further comprises instructions that in response to being executed on the computing device result in:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source,

wherein the motion estimation is adjusted between a base level of motion estimation and an enhanced level of motion estimation, the base level of motion estimation is associated with a first level of downscaled visual data and the enhanced level of motion estimation is associated with a second level of visual data, the second level of downscaling being greater than the first level of downscaling, and the base level of motion estimation comprising hierarchical motion estimation and the enhanced level of motion estimation comprising super hierarchical motion estimation,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value,

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an identification of the target user as a child.

22. The article of claim 20, wherein the machine-readable medium further comprises instructions that in response to being executed on the computing device result in:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises an external power source.

23. The article of claim 20, wherein the machine-readable medium further comprises instructions that in response to being executed on the computing device result in:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source and an indication that the battery charge level is below a charge threshold value.

24. The article of claim 20, wherein the machine-readable medium further comprises instructions that in response to being executed on the computing device result in:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the enhanced level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is above a motion threshold value.

25. The article of claim 20, wherein the machine-readable medium further comprises instructions that in response to being executed on the computing device result in:

receiving, at the one or more processors, power supply data, the power supply data comprising an indication of a current power source and/or an indication of battery charge level, and the indication of the current power source indicating whether the current power source comprises a battery-type power source or an external power source, and

wherein the selectively adjusted motion estimation applied to the received visual data comprises selectively adjusting the motion estimation to the base level of motion estimation in response to an indication that the current power source comprises a battery-type power source, an indication that the battery charge level is above a given threshold value, and an indication that the determined amount of motion is below a motion threshold value.