Segmentation of Video Sequences

• US 20090028432A1
• Filed: 12/30/2005
• Published: 01/29/2009
• Est. Priority Date: 12/30/2005
• Status: Abandoned Application

• Current Assignee: Telecom Italia S.p.A.
• Original Assignee: Telecom Italia S.p.A.
• Inventors: Porta, Alessandro, Varesio, Andrea, Rossato, Luca
• Application Number:
  US12/087,269
• Publication Number:
  US 20090028432A1
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

A method of image processing, includes: providing a data processing apparatus, including a central processing unit and a graphics processing unit; feeding at least one video frame of a video sequence to the data processing apparatus, the at least one video frame including at least one foreground subject and a background; and processing the at least one video frame in the data processing apparatus so as to separate the at least one foreground subject from the background. The processing is at least partially performed by the processors of the graphics processing unit.

Claims

1-58. -58. (canceled)

59. A method of image processing, comprising:

providing a data processing apparatus, comprising a central processing unit and a graphics processing unit;

receiving at the data processing apparatus at least one video frame of a video sequence, the at least one video frame comprising at least one foreground subject and a background; and

processing said at least one video frame in the data processing apparatus so as to separate said at least one foreground subject from said background,

said processing being at least partially performed by said graphics processing unit.

60. The method of claim 59, wherein said graphics processing unit comprises a plurality of processors.

61. The method of claim 59, wherein said graphics processing unit supports pixel shading.

62. The method of claim 59, wherein said processing comprises obtaining a reference image of the background.

63. The method of claim 62, wherein said obtaining a reference image of the background comprises obtaining a background statistical model based on a plurality of video frames not containing the at least one foreground subject.

64. The method of claim 62, wherein said obtaining the background statistical model comprises:

providing a description of the video frames of said plurality in a color space comprising an angular coordinate descriptive of a pixel property; and

calculating an average of the angular coordinate for each pixel of the video frames of said plurality.

65. The method of claim 64, wherein said calculating the average of the angular coordinate for each pixel of the video frames of said plurality comprises:

determining, from the angular coordinate of each pixel of the video frames of said plurality, a respective pair of first and second derived linear coordinates lying on a first, and respectively, second directions;

calculating an average of the first and second derived linear coordinates; and

calculating the average of the angular coordinate based on the average of the first derived linear coordinate and the average of the second derived linear coordinate.

66. The method of claim 64, wherein said obtaining the background statistical model comprises:

calculating a variance of the angular coordinate for each pixel of the video frames of said plurality.

67. The method of claim 66, wherein said calculating the variance of the angular coordinate comprises:

associating with each pair of said first and second derived linear coordinates a complex number of unitary modulus; and

calculating a modulus | Z_N|² as a combination of the squares of the average values of the first and second derived linear coordinates.

68. The method of claim 65, comprising, for each pixel, storing in a storage area, running totals of the first and second derived linear coordinates obtained for each of said video frames of said plurality as the video frames are received.

69. The method of claim 64, wherein said angular coordinate is descriptive of a hue of the pixel.

70. The method of claim 64, wherein said providing a description of the captured video frames of said plurality in a color space comprising an angular coordinate comprises associating with the angular coordinate a first and second associated coordinates descriptive of further properties of the pixel.

71. The method of claim 64, wherein said color space is a hue, saturation, value color space.

72. The method of claim 70, wherein said obtaining the background statistical model further comprises:

calculating an average of the first and second associated coordinates for the pixels of said plurality of video frames.

73. The method of claim 70, wherein said obtaining the background statistical model comprises:

calculating a variance of the first and second associated coordinates for the pixels of said plurality of video frames.

74. The method of claim 70, comprising, for each pixel, storing in a storage area, running totals of the values of the first and second associated coordinates obtained for each of said video frames of said plurality as they are received.

75. The method of claim 68, wherein said storing running totals comprises, for one or more coordinates selected from the group of the first and second derived linear coordinates and the first and second associated coordinates:

a) providing a first graphics processing unit texture and a second graphics processing unit texture adapted to store, for each pixel, values corresponding to the running totals of the one or more coordinates for the pixels of said video frames of said plurality;

b) initializing the values of at least said first texture to a predetermined initial value;

c) obtaining a value of the at least one coordinate for each pixel of a generic video frame of said plurality;

d) feeding the first graphics processing unit texture as input to the graphics processing unit;

e) having the graphics processing unit add, for each pixel, the corresponding values stored in the first texture to the value of the one or more coordinates for each pixel of generic video frame of said plurality; and

f) having the graphics processing unit store the result of the addition into the second texture.

76. The method of claim 75, further comprising:

g) exchanging a role of input texture and output texture between the first texture and the second texture; and

h) repeating steps c) to g) for all the video frames of said plurality.

77. The method of claim 62, wherein said processing comprises comparing said at least one video frame to said reference image of the background.

78. The method of claim 77, wherein:

said processing comprises obtaining a reference image of the background;

wherein said obtaining a reference image of the background comprises obtaining a background statistical model based on a plurality of video frames not containing the at least one foreground subject;

wherein said obtaining the background statistical model comprises;

providing a description of the video frames of said plurality in a color space including an angular coordinate descriptive of a pixel property; and

calculating an average of the angular coordinate for each pixel of the video frames of said plurality;

wherein said providing a description of the captured video frames of said plurality in a color space comprising an angular coordinate comprises associating with the angular coordinate a first and second associated coordinates descriptive of further properties of the pixel;

wherein said comparing comprises;

for each pixel of the at least one video frame, comparing at least one among the angular coordinate and the first and second associated coordinates to a corresponding coordinate value in said reference image of the background.

79. The method of claim 78, wherein said comparing at least one among the angular coordinate and the first and second associated coordinates comprises:

for at least one of said angular coordinates, the first and second associated coordinates, calculating an absolute value of a difference between a value of the coordinate in respect to the at least one video frame and the value of said coordinate in respect to the reference image of the background; and

comparing the calculated absolute difference value to a predetermined threshold value.

80. The method of claim 79, wherein said obtaining the background statistical model comprises:

calculating a variance of the first and second associated coordinates for the pixels of said plurality of video frames;

wherein said predetermined threshold value is a value related to the variance or to the standard deviation of the coordinate in the background statistical model.

81. The method of claim 77, wherein said comparing said at least one video frame to said reference image of the background comprises obtaining a foreground mask as a result of said comparing, said foreground mask specifying, for each pixel of the at least one video frame, a first pixel state indicative of the fact that the pixel belongs to the at least one foreground subject, and a second pixel state indicative of the fact that the pixel belongs to the background.

82. The method according to claim 81, wherein said processing comprises obtaining a reference image of the background;

wherein said processing comprises comparing said at least one video frame to said reference image of the background;

wherein said obtaining the foreground mask comprises;

combining a result of said comparing the calculated absolute difference value to the predetermined threshold value for the angular coordinate and the first and second associated coordinates; and

obtaining an approximated foreground mask by determining a pixel state based on said combining.

83. The method of claim 59, wherein the data processing apparatus comprises at least one of a videotelephone and a video-conferencing apparatus.