Method and apparatus for block classification and adaptive bit allocation

• US 6,275,614 B1
• Filed: 04/21/1999
• Issued: 08/14/2001
• Est. Priority Date: 06/26/1998
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation), SRI International, Inc.
• Original Assignee: LG Electronics, Inc. (LG Corporation), Sarnoff Corporation (SRI International, Inc.)
• Inventors: Krishnamurthy, Ravi, Sethuraman, Sriram
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/295,878
• Time in Patent Office: 846 Days
• Field of Search: 382/115, 382/181, 382/224, 382/232, 382/236, 382/118, 382/133, 382/165, 382/225, 382/238, 348/14.13, 348/155, 348/384.1, 358/467, 375/240.12, 375/240.11
• US Class Current: 382/224
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/21 with binary alpha-plane coding for video objects, e.g. context-based arithmetic encoding [CAE]
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Apparatus and method for classifying regions of an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources, e.g., bit allocation in an encoding environment.

Claims

1. A method for classifying a block within a current image of an uncompressed input image sequence, said method comprising the steps of:

(a) mapping a chrominance component of a block in the current image of the uncompressed input image sequence in a chrominance plane;

(b) defining a decision region by four straight lines in said chrominance plane; and

(c) classifying said block as a skin-tone block if said chrominance component falls within said decision region.

2. The method of claim 1, wherein said defining step (b) defines said decision region by four straight lines expressed as:

3. The method of claim 2, further comprising the step of:

(c) applying said skin-tone block to generate an importance map.

4. The method of claim 3, wherein said applying step (c) further comprises the step of:

(c1) using an edge block to generate said importance map.

5. The method of claim 3, wherein said applying step (c) further comprises the step of:

(c1) using a motion block to generate said importance map.

6. The method of claim 3, wherein said applying step (c) further comprises the step of:

(c1) using an importance map of a prior image to generate said importance map of the current image.

7. The method of claim 1, further comprising the step of:

(d) using luminance information to qualify said classifying step (c).

8. A method for allocating coding bits to a block within an image of an uncompressed input image sequence, said method comprising the steps of:

(a) obtaining an importance information for the block within the image, by receiving importance information from a block classifier that maps a chrominance component of said block in the image of the uncompressed input image sequence in a chrominance plane to detect a skin-tone block; and

(b) allocating coding bits to said block in accordance with said importance information.

9. The method of claim 8, wherein said obtaining importance information step (a) further comprises the step of:

receiving importance information from a block classifier for detecting an edge block.

10. The method of claim 8, wherein said obtaining importance information step (a) further comprises the step of:

receiving importance information from a block classifier for detecting a motion block.

11. The method of claim 8, wherein said obtaining importance information step (a) further comprises the step of:

receiving importance information from a block classifier using an importance map of a prior image.

12. The method of claim 8, wherein said allocating step (b) comprises the step of:

(b1) allocating the coding bits in accordance with an importance factor that accounts for a significance of the block.

13. The method of claim 12, wherein said allocating step (1) allocates the coding bits in accordance with:

14. The method of claim 12, wherein said importance factor is expressed as:

$\mathrm{imp\_factor}=\left(\frac{\mathrm{\varepsilon <br><br>}+\mathrm{\mu <br><br>}<br><br>\underset{\mathrm{remaining}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{mbs}}{\sum <br><br>}<br><br>\frac{B_i}{N_{r<br><br>\text{\hspace{1em}<br><br>}<br><br>e<br><br>\text{\hspace{1em}<br><br>}<br><br>m}}+B_{\mathrm{curr}}}{\mathrm{\varepsilon <br><br>}+\underset{\mathrm{remaining}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{mbs}}{\sum <br><br>}<br><br>\frac{B_i}{N_{r<br><br>\text{\hspace{1em}<br><br>}<br><br>e<br><br>\text{\hspace{1em}<br><br>}<br><br>m}}+\mathrm{\mu <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>B_{\mathrm{curr}}}\right)$

where ε

is a constant, μ

is a constant, N_rem is a number of remaining blocks, B_i is a significance value of the ith block, and B_curr is a significance value of the current block.

15. The method of claim 12, wherein said allocating step (b) further comprises the step of:

(b2) allocating the coding bits in accordance with a global adjustment that accounts for bits used to encode a previous frame.

16. The method of claim 15, wherein said global adjustment is expressed as:

17. The method of claim 15, wherein said allocating step (b) further comprises the step of:

(b3) allocating the coding bits in accordance with a local adjustment that accounts for a number of bits spent up to a present block.

18. The method of claim 17, wherein said local adjustment is expressed as:

19. The method of claim 18, wherein said projection is expressed as:

$\mathrm{projection}=\frac{T}{\underset{\mathrm{all}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{macroblocks}}{\sum <br><br>}<br><br>w_i<br><br>d_i^2}<br><br>\underset{i=0}{\overset{\mathrm{current}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{macroblock}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>w_i<br><br>d_i^2$

where w_i is a weight for a macroblock i and d_i is a distortion for said macroblock i.

20. The method of claim 8, wherein said obtaining importance information step (a) comprises the step of:

filling or purging said received importance information from said block classifier for detecting a skin-tone block.

21. An apparatus for allocating coding bits to a block within an image of an uncompressed input image sequence said apparatus comprising:

a block classifier for generating an importance map for indicating an importance information of at least one block within the image, where said importance information is generated by mapping a chrominance component of said at least one block in the image of the uncompressed input image sequence in a chrominance plane to detect a skin-tone block; and

a controller, coupled to said block classifier, for allocating coding bits to said block in accordance with said importance information.

22. The apparatus of claim 21, wherein said block classifier comprises:

at least one detector for classifying a block as a skin-tone block, an edge block, or a motion block; and

an importance map generator, coupled to said at least one detector, for generating said importance map.

23. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

(a) mapping a chrominance component of a block in the current image of an uncompressed input image sequence in a chrominance plane;

(b) defining a decision region by four straight lines in said chrominance plane; and

(c) classifying said block as a skin-tone block if said chrominance component falls within said decision region.