Hierarchical scheme for blur detection in digital image using wavelet transform

• US 7,257,273 B2
• Filed: 08/22/2003
• Issued: 08/14/2007
• Est. Priority Date: 04/09/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Hao, Li, Mingjing, Zhang, Hong-Jiang
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US10/646,387
• Publication Number:
  US 20040066981A1
• Time in Patent Office: 1,453 Days
• Field of Search: 382/199, 382/240, 382/264, 382/274, 382/275, 382/286, 382/300, 382/302, 375/240.19, 375/240.27, 375/254
• US Class Current: 382/286
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 7/12 Edge-based segmentation

Abstract

Methods and apparatuses are provided for detecting blur within digital images using wavelet transform and/or Cepstrum analysis blur detection techniques that are able to detect motion blur and/or out-of-focus blur.

Claims

1. A method comprising:

accessing at least a portion of a digital image;

determining if at least said portion is blurred based on a wavelet transform blur detection process, wherein the wavelet transform blur detection process comprises;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level including a plurality of bands;

generating at least one edge map for each of said resolution levels; and

detecting blur in at least said portion of said digital image based on said resulting edge maps by;

normalizing a total edge amplitude of said edge map;

Emap_i(k,l)=Emap_i(k,l)/max(Emap_i);

partitioning said edge map into edge map blocks;

determining a maximal edge amplitude in each of said edge map blocks and using it to represent the respective edge map block; and

using Emax_i to denote a discretization result of Emap_i for each of said edge map blocks.

2. A method comprising:

accessing at least a portion of a digital image; and

determining if at least said portion is blurred based on a wavelet transform blur detection process, wherein the wavelet transform blur detection process includes;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level includinq a plurality of bands;

generating at least one edge map for each of said resolution levels, wherein I_iv,I_ih,I_id denote LH_i,HL_i,HH_i bands, respectively, and wherein generating said at least one edge map for each of said resolution levels includes constructing said edge map in scale i as follows;

${\mathrm{Emap}}_i<br><br>\left(k,l\right)=\sqrt{I_{\mathrm{iv}}^2<br><br>\left(k,l\right)+I_{\mathrm{ih}}^2<br><br>\left(k,l\right)+I_{\mathrm{id}}^2<br><br>\left(k,l\right)}$

where (k,l) is the coordinate of a pixel in scale i; and

detecting blur in at least said portion of said digital image based on said resulting edge maps.

3. The method as recited in claim 2, wherein detecting blur in at least said portion of said digital image based on said resulting edge maps further includes:

comparing amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels, and

wherein comparing said amplitude variations includes generating a difference map Dmap based on

$\mathrm{Dmap}<br><br>\left(i,j\right)=\sqrt{{\left(E<br><br><br><br>{\max }_3<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)\right)}^2+{\left(E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_1<br><br>\left(i,j\right)\right)}^2}.$

4. A method comprising:

accessing at least a portion of a digital image; and

determining if at least said portion is blurred based on a wavlet transform blur detection process, wherein the wavelet transform blur detection process includes;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level including a plurality of bands;

generating at least one edge map for each of said resolution levels; and

detecting blur in at least said portion of said digital image based on said resulting edge maps by comparing amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels.

5. The method as recited in claim 4, wherein comparing said amplitude variations includes generating a difference map.

6. The method as recited in claim 5, wherein detecting blur in at least said portion of said digital image based on said resulting edge maps further includes:

generating a binary difference map BDmap such that,

BDmap(i,j)=1 if Dmap(i,j)>

t1

BDmap(i,j)=0 otherwise

where t1 is a first threshold value; and

determining that at least one edge map block (i, j) is blurred if said corresponding BDmap(i,j)=1.

7. The method as recited in claim 6, further comprising:

determining that at least said portion of said digital image is blurred if an applicable percentage of edge map blocks are determined to be blurred exceeds a second threshold value.

8. The method as recited in claim 4, wherein in said difference map a position of a plurality of relatively large amplitude values corresponds to at least one blurred edge.

9. A computer-readable medium having computer-implementable instructions suitable for causing at least one processing unit to perform acts comprising:

determining if at least a portion of a digital image is motion blurred or out-of-focus blurred based on a wavelet transform blur detection process that includes;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level including a plurality of bands;

generating at least one edge map for each of said resolution levels; and

detecting blur in at least said portion of said digital image based on said resulting edge mars by normalizing each of said resulting edge maps, wherein normalizing each of said resulting edge maps further includes;

normalizing a total edge amplitude of said edge map;

Emap_i(k,l)=Emap_i(k,l)/max(Emap_i);

partitioning said edge map into edge map blocks;

determining a maximal edge amplitude in each of said edge map blocks and using it to represent the respective edge map block; and

using Emax_i to denote a discretization result of Emap_i for each of said edge map blocks.

10. A computer-readable medium having computer-implementable instructions suitable for causing at least one processing unit to perform acts comprising:

determining if at least a portion of a digital image is motion blurred or out-of-focus blurred based on a wavelet transform blur detection orocess that includes;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level including a plurality of bands;

generating at least one edge map for each of said resolution level, wherein I_iv,I_ih,I_id denote LH_i,HL_i,HH_i bands, respectively, and wherein generating said at least one edge map for each of said resolution levels further includes constructing said edge map in scale i as follows;

${\mathrm{Emap}}_i<br><br>\left(k,l\right)=\sqrt{I_{i<br><br><br><br>v}^2<br><br>\left(k,l\right)+I_{i<br><br><br><br>h}^2<br><br>\left(k,l\right)+I_{i<br><br><br><br>d}^2<br><br>\left(k,l\right)}$

where (k, l) is the coordinate of a pixel in scale i; and

detecting blur in at least said portion of said digital image based on said resulting edge maps.

11. The computer-readable medium as recited in claim 10, wherein detecting blur in at least said portion of said digital image based on said resulting edge maps further includes:

comparing amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels, and

wherein comparing said amplitude variations includes generating a difference map Dmap based on

$\mathrm{Dmap}<br><br>\left(i,j\right)=\sqrt{{\left(E<br><br><br><br>{\max }_3<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)\right)}^2+{\left(E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_1<br><br>\left(i,j\right)\right)}^2}.$

12. A computer-readable medium having computer-implementable instructions suitable for causing at least one processing unit to perform acts comprising:

determining if at least a portion of a digital image is motion blurred or out-of-focus blurred based on a wavelet transform blur detection process that includes;

wavelet transforming at least said portion of said digital image to produce a plurality of corresponding different resolution levels, each resolution level including a plurality of bands;

generating at least one edge map for each of said resolution levels; and

detecting blur in at least said portion of said digital image based on said resulting edge maps by comparing amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels.

13. The computer-readable medium as recited in claim 12, wherein comparing said amplitude variations includes generating a difference map.

14. The computer-readable medium as recited in claim 13, wherein detecting blur in at least said portion of said digital image based on said resulting edge maps further includes:

generating a binary difference map BDmap such that,

BDmap(i,j)=1 if Dmap(i,j)>

t1

BDmap(i,j)=0 otherwise

where t1 is a first threshold value; and

determining that at least one edge map block (i, j) is blurred if said corresponding BDmap(i,j)=1.

15. The computer-readable medium as recited in claim 14, further comprising:

determining that at least said portion of said digital image is blurred if an applicable percentage of edge map blocks are determined to be blurred exceeds a second threshold value.

16. The computer-readable medium as recited in claim 12, wherein in said difference map a position of a plurality of relatively large amplitude values corresponds to at least one blurred edge.

17. An apparatus comprising:

logic operatively configured to access digital image data and determine if at least a portion of said image is blurry using a wavelet transform blur detector operatively configured to;

wavelet transform at least said portion to produce a plurality of corresponding different resolution levels with each resolution level including a plurality of bands;

generate at least one edge map for each of said resolution levels;

normalize each of said resulting edge maps by normalizing a total edge amplitude of said edge map such that Emap_i(k,l)=Emap_i(k,l)/max(Emap_i), partitions said edge map into edge map blocks, and determines a maximal edge amplitude in each of said edge map blocks and uses it to represent the respective edge map block, and using Emax_i denotes a discretization result of Emap_i for each of said edge map blocks; and

detect blur in at least said portion of said digital image based on said resulting edge maps.

18. The apparatus as recited in claim 17, wherein said apparatus includes at least one device selected from a group of devices comprising a computer, a camera, a set top box, an optical disc player, an optical disc player recorder, a portable communication device, a display device, a television set, and a projector.

19. An apparatus comprising:

logic operatively configured to access digital image data and determine if at least a portion of said image is blurry using a wavelet transform blur detector that is operatively configured to;

wavelet transform at least said portion to produce a plurality of corresponding different resolution levels with each resolution level including a plurality of bands;

generate at least one edge map for each of said resolution levels, wherein I_iv,I_ih,I_id denote LH_i,HL_i,HH_i bands, respectively, by constructing said edge map in scale i as follows;

${\mathrm{Emap}}_i<br><br>\left(k,l\right)=\sqrt{I_{i<br><br><br><br>v}^2<br><br>\left(k,l\right)+I_{i<br><br><br><br>h}^2<br><br>\left(k,l\right)+I_{i<br><br><br><br>d}^2<br><br>\left(k,l\right)}$

where (k, l) is the coordinate of a pixel in scale i; and

detect blur in at least said portion of said digital image based on said resulting edge maps.

20. The apparatus as recited in claim 19, wherein said wavelet transform blur detector compares amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels, and generates a difference map Dmap based on

$\mathrm{Dmap}\; <li></li>\left(i,j\right)=\sqrt{{\left(E<br><br><br><br>{\max }_3<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)\right)}^2+{\left(E<br><br><br><br>{\max }_2<br><br>\left(i,j\right)-E<br><br><br><br>{\max }_1<br><br>\left(i,j\right)\right)}^2}.$

21. The apparatus as recited in claim 19, wherein said apparatus includes at least one device selected from a group of devices comprising a computer, a camera, a set top box, an optical disc player, an optical disc player recorder, a portable communication device, a display device, a television set, and a projector.

22. An apparatus comprising:

logic operatively configured to access digital image data and determine if at least a portion of said image is blurry using a wavelet transform blur detector that is operatively configured to;

wavelet transform at least said portion to nroduce a plurality of corresponding different resolution levels with each resolution level including a olurality of bands;

generate at least one edge map for each of said resolution levels;

compare amplitude variations of corresponding edge nodes in at least two different edge maps of at least two different levels; and

detect blur in at least said portion of said digital image based on said resulting edge maps.

23. The apparatus as recited in claim 22, wherein said wavelet transform blur detector generates a difference map.

24. The apparatus as recited in claim 23, wherein said wavelet transform blur detector generates a binary difference map BDmap such that,

BDmap(i,j)=1 if Dmap(i,j)>

t1

BDmap(i,j)=0 otherwise

where t1 is a first threshold value; and

determines that at least one edge map block (i, j) is blurred if said corresponding BDmap(i,j)=1.

25. The apparatus as recited in claim 24, wherein said wavelet transform blur detector determines that at least said portion of said digital image is blurred if an applicable percentage of edge map blocks are determined to be blurred exceeds a second threshold value.

26. The apparatus as recited in claim 22, wherein in said difference map a position of a plurality of relatively large amplitude values corresponds to at least one blurred edge.

27. The apparatus as recited in claim 22, wherein said apparatus includes at least one device selected from a group of devices comprising a computer, a camera, a set top box, an optical disc player, an optical disc player recorder, a portable communication device, a display device, a television set, and a projector.