Formation of a time-varying signal representative of at least variations in a value based on pixel values

• US 9,025,826 B2
• Filed: 10/03/2013
• Issued: 05/05/2015
• Est. Priority Date: 10/06/2009
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Kirenko, Ihor, Jeanne, Vincent, de Haan, Gerard
• Primary Examiner(s): Carter, Aaron W
• Assistant Examiner(s): Harandi, Siamak
• Application Number:
  US14/044,976
• Publication Number:
  US 20140037163A1
• Time in Patent Office: 579 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/38 Registration of image sequences

Abstract

A method of forming a time-varying signal representative of at test variations in a value based on pixel values from a sequence of images, the signal corresponding in length to the sequence of images, includes obtaining the sequence of images. A plurality of groups of sub-sets of pixel values are formed by selecting a sub-set of at least one pixel value from each of at least two images defining an interval to form a group of associated sub-sets using at least one of a different aperture and a different interval length for groups defined on different intervals of the sequence. Groups of sub-sets are selected to form the signal to cover different intervals of the sequence of images by obtaining spatio-temporal volumes of pixel values from a sequence of images at least based on the received sequence of images. Each volume includes pixel values within a spatial aperture from each image within an interval of the sequence.

Claims

1. A method of forming a time-varying signal representative of at least variations in a value based on pixel values from a sequence of images, the signal corresponding in length to the sequence of images, the method comprising acts of:

receiving the sequence of images;

forming a plurality of groups of sub-sets of pixel values by selecting a sub-set of at least one pixel value from each of at least two images defining an interval to form a group of associated sub-sets using at least one of a different aperture and a different interval length for groups defined on different intervals of the sequence; and

selecting groups of sub-sets to form the signal to cover different intervals of the sequence of images by obtaining spatio-temporal volumes of pixel values from a sequence of images at least based on the received sequence of images, each volume comprising pixel values within a spatial aperture from each image within an interval of the sequence.

2. The method according to claim 1, wherein the act of forming groups comprises an act of laying a grid over at least a region of each of a plurality of images at least based on the received images to obtain spatio-temporal volumes of pixel values, each comprising pixel values of each image within a corresponding spatial aperture defined by the grid.

3. The method according to claim 2, comprising an act of analyzing for each group at least one sub-set of pixel values defined by the spatial aperture to determine whether it represents a part of a surface of a certain type.

4. The method according to claim 1, wherein the act of selecting groups comprise an act of using a different interval length for groups defined on different intervals of the sequence of images.

5. The method according to claim 3, wherein the act of selecting groups comprises an act of selecting groups of sub-sets to form the signal in accordance with at least one selection criterion.

6. The method according to claim 5, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of spatial uniformity.

7. The method according to claim 5, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of temporal uniformity.

8. The method according to claim 1, wherein the act of selecting groups comprises an act of selecting a first sub-set of at least one pixel value from a first image at least based on one of the received sequence of images and locating a similar sub-set in each of at least one further image at least based on a respective one of the received sequence of images.

9. The method according to claim 8, wherein the act of selecting groups comprises an act of obtaining a predicted motion vector and modifying the predicted motion vector to select candidate sub-sets for evaluation against at least one similarity criterion.

10. The method according to claim 1, wherein forming the groups comprises an act of selecting sub-sets from only limited regions of interest in each of a sequence of at least two images at least based on respective images of the received sequence of images.

11. A machine-readable non-transitory medium including a set of instructions configured to cause a system having information processing capability to perform a method according to claim 1.

12. A method of forming a time-varying signal representative of at least variations in a value based on pixel values from a sequence of images, the method comprising acts of:

receiving the sequence of images;

laying a grid over at least a region of each of a plurality of images based on the received images to obtain spatio-temporal volumes of pixel values, each comprising pixel values of each image within a corresponding spatial aperture, wherein dimensions of the grid varies spatially;

forming a plurality of groups of sub-sets of pixel values by selecting a sub-set of at least one pixel value from each spatial aperture of at least two images defining an interval to form a group of associated sub-sets using a different aperture having different dimensions defined by the grid for at least one interval of the sequence of images; and

selecting groups of sub-sets to form the signal to cover different intervals of the sequence of images by obtaining spatio-temporal volumes of pixel values from a sequence of images at least based on the received sequence of images, each volume comprising pixel values within a spatial aperture front each image within an interval of the sequence of images.

13. The method according to claim 12, comprising an act of analyzing for each group at least one sub-set of pixel values defined by the spatial aperture to determine whether it represents a part of a surface of a certain type.

14. The method according to claim 12, wherein forming the groups comprises an act of determining an aperture size separately for each of the intervals based on a uniformity of an image area within a corresponding sequence of images.

15. The method according to claim 14, wherein less uniformity of the image area results in a smaller determined aperture size and more uniformity of the image area results in a larger determined aperture size.

16. The method according to claim 12, wherein forming the groups comprises an act of determining an aperture size separately for each aperture of the grid based on a uniformity of an image area within a corresponding sequence of images.

17. The method according to claim 16, wherein determining the aperture size comprises an act of performing at least one of object segmentation, color segmentation and background-foreground segmentation on at least one image of the corresponding sequence of images.

18. The method according to claim 12, wherein the act of selecting groups comprise an act of using a different interval length for groups defined on different intervals of die sequence of images.

19. The method according to claim 18, wherein the act of selecting groups comprise an act of selecting an interval length based on an amount of motion present in each image of a corresponding sequence of images.

20. A method of forming a time-varying signal representative of at least variations in a value based on pixel values from a sequence of images, the method comprising acts of:

receiving the sequence of images, such that any two images of the sequence define a time interval;

composing one or more spatio-temporal volumes responsive to the received sequence of images, each composed volume comprising at least one selected spatial aperture and at least one selected time interval wherein at least a first one of the composed volumes has at least one of a different aperture and a different interval length from a second one of the composed volumes;

obtaining at least one spatio-temporal volume, each obtained spatio-temporal volume comprising pixel values of the spatial aperture of selected of the composed spatio-temporal volumes, the pixel values being from the received sequence of images including the images of the time interval of the selected of the spatio-temporal volumes; and

forming the time-varying signal responsive to groups of sub-sets of the pixel values of one or more of the obtained spatio-temporal volumes.

21. A system for forming a time-varying signal representative of at least variations in a value based on pixel values from a sequence of images, the signal corresponding in length to the sequence of images, the system comprising:

a processor configured to;

receive the sequence of images;

form a plurality of groups of sub-sets of pixel values by selecting a sub-set of at least one pixel value from each of at least two images defining an interval to form a group of associated sub-sets using at least one of a different aperture and a different interval length for groups defined on different intervals of the sequence; and

select groups of sub-sets to form the signal to cover different intervals of the sequence of images by obtaining spatio-temporal volumes of pixel values from a sequence of images at least based on the received sequence of images, each volume comprising pixel values within a spatial aperture from each image within an interval of the sequence.

22. The system according to claim 21, wherein the processor is configured to form groups by laying a grid over at least a region of each of a plurality of images at least based on the received images to obtain spatio-temporal volumes of pixel values, each comprising pixel values of each image within a corresponding spatial aperture defined by the grid.

23. The system according to claim 22, wherein the processor is configured to analyze for each group at least one sub-set of pixel values defined by the spatial aperture to determine whether it represents a part of a surface of a certain type.

24. The system according to claim 21, wherein the processor is configured to select groups by using a different interval length for groups defined on different intervals of the sequence of images.

25. The system according to claim 21, wherein the processor is configured to select groups by selecting groups of sub-sets to form the signal in accordance with at least one selection criterion.

26. The system according to claim 25, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, wherein the processor is configured to evaluate the at least one selection criterion to determine a measure of spatial uniformity.

27. The system according to claim 25, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, wherein the processor is configured to evaluate the at least one selection criterion to determine a measure of temporal uniformity.

28. The system according to claim 21, wherein the processor is configured to select groups by selecting a first sub-set of at least one pixel value from a first image at least based on one of the received sequence of images and locating a similar sub-set in each of at least one further image at least based on a respective one of the received sequence of images.

29. The system according to claim 28, wherein the processor is configured to select groups by obtaining a predicted motion vector and modifying the predicted motion vector to select candidate sub-sets for evaluation against at least one similarity criterion.

30. The system according to claim 21, wherein the processor is configured to form the groups by selecting sub-sets from only limited regions of interest in each of a sequence of at least two images at least based on respective images of the received sequence of images.

31. A machine-readable non-transitory medium including a set of instructions configured to cause a system having information processing capability to perform a method of forming a time-varying signal representative of at least variations in a value based on pixel values from a sequence of images, the signal corresponding in length to the sequence of images, the method comprising acts of:

receiving the sequence of images;

forming a plurality of groups of sub-sets of pixel values by selecting a sub-set of at least one pixel value from each of at least two images defining an interval to form a group of associated sub-sets using at least one of a different aperture and a different interval length for groups defined on different intervals of the sequence; and

selecting groups of sub-sets to form the signal to cover different intervals of the sequence of images by obtaining spatio-temporal volumes of pixel values from a sequence of images at least based on the received sequence of images, each volume comprising pixel values within a spatial aperture from each image within an interval of the sequence.

32. The medium according to claim 31, wherein the act of forming groups comprises an act of laying a grid over at least a region of each of a plurality of images at least based on the received images to obtain spatio-temporal volumes of pixel values, each comprising pixel values of each image within a corresponding spatial aperture defined by the grid.

33. The medium according to claim 32, the method comprising an act of analyzing for each group at least one sub-set of pixel values defined by the spatial aperture to determine whether it represents a part of a surface of a certain type.

34. The medium according to claim 31, wherein the act of selecting groups comprise an act of using a different interval length for groups defined on different intervals of the sequence of images.

35. The medium according to claim 31, wherein the act of selecting groups comprises an act of selecting groups of sub-sets to form the signal in accordance with at least one selection criterion.

36. The medium according to claim 35, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of spatial uniformity.

37. The medium according to claim 35, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of temporal uniformity.

38. The medium according to claim 31, wherein the act of selecting groups comprises an act of selecting a first sub-set of at least one pixel value from a first image at least based on one of the received sequence of images and locating a similar sub-set in each of at least one further image at least based on a respective one of the received sequence of images.

39. The medium according to claim 38, wherein the act of selecting groups comprises an act of obtaining a predicted motion vector and modifying the predicted motion vector to select candidate sub-sets for evaluation against at least one similarity criterion.

40. The medium according to claim 31, wherein forming the groups comprises an act of selecting sub-sets from only limited regions of interest in each of a sequence of at least two images at least based on respective images of the received sequence of images.

41. A system for forming a time-varying signal representative of at least variations in a value based on pixels and corresponding pixel values from a sequence of images, the signal corresponding in length to the sequence of images, the system comprising:

a processor configured to;

obtain the sequence of images;

form a plurality of groups of sub-sets of pixels and corresponding pixel values by selecting a sub-set of at least one pixel and corresponding pixel value from each of at least two images defining an interval to form a group of associated sub-sets,

wherein, for any pair of groups with only sub-sets from the same images, the sub-sets from more than one image are different pixels; and

select groups of sub-sets to form the signal in accordance with at least one selection criterion,

wherein the selected groups of sub-sets cover different intervals, shorter than a length of the sequence.

42. A method of forming a time-varying signal representative of at least variations in a value based on pixels and corresponding pixel values from a sequence of images, the signal corresponding in length to the sequence of images, the method comprising acts of:

obtaining the sequence of images;

forming a plurality of groups of sub-sets of pixels and corresponding pixel values by selecting a sub-set of at least one pixel and corresponding pixel value from each of at least two images defining an interval to form a group of associated sub-sets, such that images are covered by at least two intervals; and

selecting groups of sub-sets to form the signal in accordance with at least one selection criterion,

wherein the selected groups of sub-sets cover at least one of different pixels and intervals, shorter than a length of the sequence.

43. The method according to claim 42, wherein the act of selecting groups comprise an act of using a different interval length for groups defined on different intervals of the sequence of images.

44. The method according to claim 42, wherein the act of selecting groups comprises an act of selecting groups of sub-sets to form the signal in accordance with at least one selection criterion.

45. The method according to claim 44, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of spatial uniformity.

46. The method according to claim 44, wherein each volume comprises pixel values within a spatial aperture from each image within an interval of the sequence, the method comprising an act of evaluating the at least one selection criterion determining a measure of temporal uniformity.

47. The method according to claim 42, wherein forming the groups comprises an act of selecting sub-sets from only limited regions of interest in each of a sequence of at least two images at least based on respective images of the received sequence of images.

48. A machine-readable non-transitory medium including a set of instructions configured to cause a system having information processing capability to perform a method according to claim 42.