System for Use of Complexity of Audio, Image and Video as Perceived by a Human Observer

• US 20080159403A1
• Filed: 12/14/2007
• Published: 07/03/2008
• Est. Priority Date: 12/14/2006
• Status: Abandoned Application

• Current Assignee: Corp. One Ltd.
• Original Assignee: Corp. One Ltd.
• Inventors: Dunning, Ted Emerson
• Application Number:
  US11/956,896
• Publication Number:
  US 20080159403A1
• Time in Patent Office: Days
• US Class Current: 375/240.21
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A system and method for determining and using complexity of image, audio, or video information as perceived by a human observer is provided. The system and method may determine complexity of the image, audio or video information by using a perceptual model, such as a lossy compression system. The compression system may remove portions of the information (and reduce the size of the information) in ways nearly imperceptible to a human, while preserving the overall human perception. The size of the information after the compression may provide an indicator of the complexity, such as provide an upper bound on the complexity of the information as perceived by a human. The complexity of the information, once determined, may be used in a variety of ways, such as characterizing the information (including fingerprinting the information), comparing the information with other image, audio or video information, or presenting the information.

Claims

1. A method for characterizing complexity of audio or visual information as perceived by a human, the method comprising:

applying a model of human perception to at least a part of the audio or visual information, the model modifying at least one aspect of the audio or visual information; and

analyzing the aspect modified by the model in order to characterize complexity of the audio or visual information as perceived by the human.

2. The method of claim 1, wherein the model compresses the audio or visual information;

and

wherein analyzing the aspect comprises analyzing a size of the audio or visual information after compressing.

3. The method of claim 2, wherein the model is adapted to reduce the size of the audio or visual information by removing information nearly imperceptible to the human;

and

wherein the model further low-pass filters the audio or visual information.

4. The method of claim 1, wherein the audio or visual information comprises multiple images in a video;

and

wherein the model determines a complexity of at least a part of the multiple images by extracting at least a portion of each of the multiple images in the video.

5. The method of claim 4, wherein the model compresses the audio or visual information;

and

further comprising normalizing the multiple images in the video prior to compressing the audio or visual information.

6. The method of claim 4, wherein the model is applied to differences between the multiple images in the video.

7. The method of claim 4, wherein analyzing comprises comparing the complexity of the multiple images to determine a most complex image.

8. The method of claim 4, wherein the video comprises a plurality of scenes, the scenes each comprising multiple frames;

and

wherein analyzing comprises comparing the complexity of the multiple scenes by analyzing the multiple frames within the scenes in order to determine a most complex scene from the plurality of scenes in the video.

9. The method of claim 1, wherein analyzing further comprises generating a fingerprint of the audio or visual information based on the complexity.

10. The method of claim 9, further comprising comparing the fingerprint of the audio or visual information with a fingerprint of a known audio or visual information in order to determine whether at least a part of the audio or visual information is substantially the same as at least part of the known audio or visual information.

11. The method of claim 1, wherein the audio or visual information comprises a plurality of video clips;

wherein analyzing comprises generating fingerprints of each of the plurality of video clips; and

further comprising;

comparing the fingerprints of each of the plurality of video clips with at least one fingerprint from one or more known video clips in order to determine a sequence list, the sequence list comprising a listing of a sequence of at least some of the plurality of video clips that comprise the one or more known video clips.

12. A method of determining human perception of audio or visual information, the method comprising:

compressing at least part of the audio or visual information, the compressing adapted to reduce size of the audio or visual information; and

analyzing the size of the compressed audio or visual information in order to determine human perception of the audio or visual information.

13. The method of claim 12, wherein the information comprises video information including multiple images.

14. The method of claim 12, wherein the compression methodology comprises lossy compression.

15. The method of claim 12, wherein analyzing the size determines a degree of complexity of the compressed audio or visual information as perceived by a human observer.

16. A system for characterizing complexity of audio or visual information as perceived by a human, the system comprising logic for:

applying a model of human perception to at least a part of the audio or visual information, the model modifying at least one aspect of the audio or visual information; and

analyzing the aspect modified by the model in order to characterize complexity of the audio or visual information as perceived by the human.

17. The system of claim 16, wherein the model compresses the audio or visual information;

and

wherein analyzing the aspect comprises analyzing a size of the audio or visual information after compressing.

18. The system of claim 17, wherein the model is adapted to reduce the size of the audio or visual information by removing information nearly imperceptible to the human;

and

wherein the model further low-pass filters the audio or visual information.

19. The system of claim 16, wherein the audio or visual information comprises multiple images in a video;

and

wherein the model determines a complexity of at least a part of the multiple images by extracting at least a portion of each of the multiple images in the video.

20. The system of claim 19, wherein the model compresses the audio or visual information;

and

further comprising normalizing the multiple images in the video prior to compressing the audio or visual information.

21. The system of claim 19, wherein the model is applied to differences between the multiple images in the video.

22. The system of claim 19, wherein analyzing comprises comparing the complexity of the multiple images to determine a most complex image.

23. The system of claim 19, wherein the video comprises a plurality of scenes, the scenes each comprising multiple frames;

and

wherein analyzing comprises comparing the complexity of the multiple scenes by analyzing the multiple frames within the scenes in order to determine a most complex scene from the plurality of scenes in the video.

24. The system of claim 16, wherein analyzing further comprises generating a fingerprint of the audio or visual information based on the complexity.

25. The system of claim 24, further comprising comparing the fingerprint of the audio or visual information with a fingerprint of a known audio or visual information in order to determine whether at least a part of the audio or visual information is substantially the same as at least part of the known audio or visual information.

26. The system of claim 19, wherein the audio or visual information comprises a plurality of video clips;

wherein analyzing comprises generating fingerprints of each of the plurality of video clips; and

further comprising;

comparing the fingerprints of each of the plurality of video clips with at least one fingerprint from one or more known video clips in order to determine a sequence list, the sequence list comprising a listing of a sequence of at least some of the plurality of video clips that comprise the one or more known video clips.