Method and apparatus for region-based allocation of processing resources and control of input image formation

• US 20030123751A1
• Filed: 12/12/2002
• Published: 07/03/2003
• Est. Priority Date: 06/26/1998
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Ravi Krishnamurthy, Sriram Sethuraman, Ya-Qin Zhang
• Inventors: Krishnamurthy, Ravi, Sethuraman, Sriram, Zhang, Ya-Qin
• Granted Patent: US 6,917,719 B2
• Time in Patent Office: Days
• US Class Current: 382/282
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Apparatus and method for classifying regions of an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources and input image formation.

Claims

1. A method for forming an input image sequence, said method comprising the steps of:

(a) defining a region of interest, wherein said region of interest is defined in accordance with a detected audio signal;

(b) obtaining a higher resolution of said defined region of interest to form a component of the input image sequence; and

(c) applying said input image sequence to an encoder.

2. The method of claim 1, further comprising the step of:

(d) applying said higher resolution of said defined region of interest to implement a zooming function.

3. The method of claim 1, further comprising the step of:

(d) applying said higher resolution of said defined region of interest to implement a compositing function.

4. The method of claim 1, further comprising the step of:

(d) defining a non-region of interest; and

(e) encoding said non-region of interest at a lower quality.

5. The method of claim 4, wherein said region of interest and said non-region of interest are encoded into two separate streams.

6. The method of claim 1, further comprising the step of:

(d) encoding said region of interest at a different quality.

7. The method of claim 1, further comprising the step of:

(d) changing a resolution of the input image sequence to account for said defined region of interest.

8. The method of claim 1, further comprising the step of:

(d) changing a quality of the input image sequence to account for said defined region of interest.

9. A method for classifying a block within a current image of an input image sequence, said method comprising the steps of:

(a) detecting an audio signal;

(b) associating said audio signal to a block in a current image of the input image sequence; and

(c) classifying said block as an important block..

10. The method of claim 9, wherein said detecting step (a) detects said audio signal using a plurality of spatially offset microphones.

11. The method of claim 10, wherein said associating step (b) associates said audio signal to a block having a speaker.

12. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

(a) defining a region of interest, wherein said region of interest is defined in accordance with a detected audio signal;

(b) obtaining a higher resolution of said defined region of interest to form a component of the input image sequence; and

(c) applying said input image sequence to an encoder.

13. The computer-readable medium of claim 12, further comprising the step of:

(d) applying said higher resolution of said defined region of interest to implement a zooming function.

14. The computer-readable medium of claim 12, further comprising the step of:

(d) applying said higher resolution of said defined region of interest to implement a compositing function.

15. The computer-readable medium of claim 12, further comprising the steps of:

(d) defining a non-region of interest; and

(e) encoding said non-region of interest at a lower quality.

16. The computer-readable medium of claim 12, further comprising the step of:

(d) encoding said region of interest at a different quality.

17. The computer-readable medium of claim 12, further comprising the step of:

(d) changing a resolution of the input image sequence to account for said defined region of interest.

18. The computer-readable medium of claim 12, further comprising the step of:

(d) changing a quality of the input image sequence to account for said defined region of interest.

19. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

(a) detecting an audio signal;

(b) associating said audio signal to a block in a current image of the input image sequence; and

(c) classifying said block as an important block.

20. An apparatus for forming an input image sequence, said apparatus comprising:

an importance map generator for defining a region of interest, wherein said region of interest is defined in accordance with a detected audio signal;

an image capturing device for obtaining a higher resolution of said defined region of interest to form a component of the input image sequence; and

an input image processing module, coupled to said importance map generator and said image capturing device, for applying said input image sequence to an encoder.

21. An apparatus for classifying a block within a current image of an input image sequence, said apparatus comprising:

an audio detector for detecting an audio signal; and

an importance map generator, coupled to said audio detector, for associating said audio signal to a block in a current image of the input image sequence and for classifying said block as an important block.