Encoding and decoding images

• US 20050286777A1
• Filed: 04/28/2005
• Published: 12/29/2005
• Est. Priority Date: 06/27/2004
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Pun, Thomas, Kumar, Roger, Wu, Hsi Jung, Duvivier, Christian
• Application Number:
  US11/119,414
• Publication Number:
  US 20050286777A1
• Time in Patent Office: Days
• US Class Current: 382/232
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Some embodiments provide a method for encoding a first set of pixels in a first image by reference to a second image in a video sequence. In a first search window within a second image, the method searches to identify a first particular portion in the second image that best matches the first set of pixels in the first image. In the first search window within the second image, the method identifies a first location corresponding to the first particular portion. In a second search window within the second image, the method then searches to identify a second particular portion in the second image that best matches the first set of pixels in the first image, where the second search window is defined about the first location.

Claims

1. A method for encoding a first set of pixels in a first image by reference to a second image in a video sequence, the method comprising:

a) in a first search window within a second image, searching to identify a first particular portion in the second image that best matches the first set of pixels in the first image. In the first search window within the second image;

identifying a first location corresponding to the first particular portion; and

b) in a second search window within the second image, searching to identify a second particular portion in the second image that best matches the first set of pixels in the first image, where the second search window is defined about the first location.

2. The method of claim 1, wherein the second search window within the second image is smaller than the first search window in the second image.

3. The method of claim 1, wherein searching to identify the first particular portion comprise searching from more than one start location.

4. The method of claim 1, wherein searching in the first search window comprises a coarse search, wherein searching in the second search window comprises a refined search.

5. The method of claim 1, wherein searching in the second search window further comprises:

a) identifying a plurality of search points within the second search window;

b) for each particular search point, iteratively;

i. identifying a plurality of second groups of pixels

ii. computing a motion vector metric for each identified second group of pixels;

iii. specifying a best second group of pixels for each first group of pixels; and

iv. foregoing the remaining search point if a criterion is satisfied.

6. The method of claim 5 further comprising:

a) determining whether the specified second group of pixels associated with the particular group of pixels, has a computed motion vector metric that is above a threshold value; and

b) excluding from a subsequent search, the particular first group of pixels, after determining that the specified second group of pixels associated with the particular first group of pixels is above the threshold value.

7. The method of claim 6, wherein the threshold value is dynamically defined during searching in the second search window.

8. The method of claim 5, wherein searching in the second search window comprises searching at a first pixel level.

9. The method of claim 8, wherein searching in the second search window further comprises searching at a second pixel level, wherein the first pixel level is an integer pixel level, wherein the second pixel level is a half pixel level.

10. A method for interblock encoding images in a video sequence, where each image in the video sequence has a plurality of integer pixel locations, with each integer pixel location having at least one image value, the method comprising:

a) selecting a first image for encoding by reference to a second image;

b) identifying a first set of non-integer pixel locations in the second image that match a set of pixels in the first image, wherein this identification comprises interpolating the image values associated with the non-integer pixel locations in the second image from the image values of a plurality of integer pixel locations in the second image

c) storing the interpolated image values of the non-integer pixel locations for later use during the encoding of a third image by reference to the second image.

11. The method of claim 10 further comprising interpolating the image values of a group of other non-integer pixel locations in the second image after identifying the set of non-integer pixel locations in the second image.

12. The method of claim 11, wherein the group of non-integer pixel locations are located about the first set of non-integer pixel locations.

13. A method for interblock decoding images in a video sequence, where each image in the video sequence has a plurality of integer pixel locations, with each integer pixel location having at least one image value, the method comprising:

a) selecting a first image for decoding by reference to a second image.;

b) identifying a set of non-integer pixel locations in the second image that correspond to a set of pixels in the first image.

c) interpolating the image values associated with the non-integer pixel locations in the second image from the image values of a plurality of integer pixel locations in the second image; and

d) storing the interpolated image values of the non-integer pixel locations for later use during the decoding of a third image by reference to the second image.

14. The method of claim 13 further comprising interpolating the image values of a group of other non-integer pixel locations in the second image after interpolating the image values associated with the non-integer pixel location.

15. The method of claim 14, wherein the group of non-integer pixel locations are located about the first non-integer pixel location.

16. A method for interblock processing a first portion in a first image by reference to a second image in a sequence of video images, the method comprising:

a) dividing the second image into a set of tiles;

b) storing the tiles in a first non-cache memory storage;

c) retrieving from the first non-cache memory storage the sub-set of tiles, whenever a sub-set of tiles are needed; and

d) storing the retrieved sub-set of tiles in a second cache memory storage between the first portion and portions of the second image that are part of the retrieved sub-set of tiles, wherein the retrieved sub-set of tiles is smaller than the entire set of tiles.

17. The method of claim 16, wherein the method determines that it needs a sub-set of tiles to be retrieved and stored in the second cache memory storage when the method identifies a location in the second image to search to identify a portion in the second image that matches the first portion, wherein this identified location corresponds to the sub-set of tiles.

18. The method of claim 16, wherein the cache memory storage is a random access memory of a computer.

19. The method of claim 16, wherein the cache memory storage is a non-volatile storage device of the computer.

20. The method of claim 16, wherein the interblock processing method is an interblock encoding method.

21. The method of claim 16, wherein the interblock processing method is an interblock decoding method.

22. The method of claim 16, wherein, the set of tiles comprises at least two horizontally adjacent tiles and at least two vertically adjacent tiles.

23. The method of claim 16, wherein the tiles are stored in the cache memory storage sequentially.

24. An interblock encoding method that encodes a first set of pixels in a first video image, the method comprising:

a) selecting a first search pattern from a set of search patterns that each defines a pattern for examining portions of a second image that might match the first set of pixels; and

b) adaptively selecting the first search pattern in the set of search patterns, based on a set of criteria.

25. The method of claim 24, wherein the set of criteria comprises of a resolution encoding of the sequence of images in a media.

26. The method of claim 24, wherein the set of criteria comprises of motion vectors of neighboring motion vectors.

27. The method of claim 24, wherein the set of criteria comprises of a motion field in a set of video images.

28. A method for encoding a first set of pixels in a first image by reference to a second image in a sequence of images, the method comprising:

a) identifying a plurality of second sets of pixels in a second image;

b) computing a first metric score for each of the second set of pixel;

c) identifying a subset of second sets of pixels based on the first metric score;

d) from the subset of identified second sets of pixels;

i. computing a second metric score for each of the identified second set of pixels; and

ii. selecting the identified second set of pixels having the best second metric score, wherein the selected identified second set of pixels best matches the first set of pixels.

29. The method of claim 28, wherein each second set of pixels comprises a plurality of second grouping of pixels, wherein each second grouping of pixels comprises a plurality of second group of pixels.

30. The method of claim 29, wherein computing the first metric score comprises:

a) computing a first metric score for each second grouping of pixels; and

b) computing a first metric score for each second group of pixels.

31. The method of claim 30, wherein identifying the subset of second sets of pixels comprises identifying a subset of second groupings of pixels and a subset of second groups of pixels.

32. The method of claim 31, wherein computing the second metric score comprises computing a second metric score for each second grouping of pixels and each second group of pixels.

33. The method of claim 32, wherein the first metric score is a sum absolute difference (“

SAD”

) metric score.

34. The method of claim 28, wherein identifying the subset of second sets pixels comprises selecting the top N second sets of pixels with the lowest first metric score.

35. The method of claim 34, wherein the second rate metric score is a rate distortion cost that quantifies the amount of data that has to be transmitted and the amount of distortion that is associated with the transmitted data.

36. The method of claim 28 further comprising:

a) computing a third metric score for the top N second set of pixels having the lowest second metric score; and

b) selecting the identified second set of pixels having the best third score, wherein the selected identified second set of pixels best matches the first set of pixels.