Video image formatting technique

• US 20030214603A1
• Filed: 05/20/2002
• Published: 11/20/2003
• Est. Priority Date: 05/20/2002
• Status: Active Grant

• Current Assignee: Thomson Licensing (Vantiva SA)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Manning, John Palmer
• Granted Patent: US 7,184,093 B2
• Time in Patent Office: Days
• US Class Current: 348/445
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/0157 with motion estimation, e.g. involving the use of motion vectors

Abstract

The formatting of a video image frame (10) having a first aspect ratio to yield an image frame having a different aspect ratio is facilitated by establishing a pan and scan pixel coordinate (P) within the image that defines an image capture window for formatting purposes. A predictor (22) determines the pan and scan pixel coordinate by examining the image frame to determine the location therein of the most pertinent activity. The predictor (22) determines the location of the most pertinent activity by examining at least one of: video data, audio data, and other data such as closed captioning information.

Claims

1. A method for establishing a pan and scan pixel coordinate (P) that defines an image capture window useful for formatting an image (10) having a first aspect ratio to yield an image having a second aspect ratio, comprising the steps of:

examining an incoming image frame to ascertain a location therein of most pertinent activity; and

establishing the pan and scan pixel coordinate in accordance with the location in the incoming image frame of the most pertinent activity.

2. The method according to claim 1 wherein the image frame comprises a video image.

3. The method according to claim 2 wherein the location of the most pertinent activity in the video image is ascertained in accordance with coordinates of macro blocks in I, P and B pictures when the video image is MPEG encoded.

4. The method according to claim 2 wherein the location of the most pertinent activity in the video image is ascertained, when the video image is MPEG encoded, from a location where motion vectors associated with such MPEG encoding, exhibit a maximum change.

5. The method according to claim 2 wherein the location of the most pertinent activity in the video image is ascertained from a location in the image whose pixel exhibits a maximum bit rate.

6. The method according to claim 1 wherein the step of examining the image frame includes examining audio data associated therewith.

7. The method according to claim 6 wherein the step of examining the audio data includes the step of determining a location within the image data associated with a source of such audio data.

8. The method according to claim 6 wherein the step of examining the audio data includes the step determining a location within the image frame associated with a maximum sound magnitude.

9. The method according to claim 6 wherein the step of examining the audio data includes the step determining a location within the image associated with a surround sound origin.

10. The method according to claim 1 wherein the step of examining the image includes the step of determining where closed captioning information appears in the image.

11. The method according to claim 1 wherein the step of examining the image includes the step of examining video data, audio data and other data associated with image.

12. The method according to claim 11 wherein the pan and scan pixel coordinate (P) is determined from the video, audio, and other data in accordance with the relationship:

P=X1*F(V)+X2*F(A)+X3*F(D)

where

X1=weighting factor assigned to the video data (0<

X1<

1);

X2=weighting factor assigned to the audio data (0<

X2<

1);

X3=weighting factor assigned to the other data(0<

X3<

1);

F(V)=coordinate predicted using video data;

F(A)=coordinate predicted using audio data;

F(D)=coordinate predicted using other data;

with

X1+X2+X3=1.

13. The method according to claim 11 wherein the other data comprises closed captioning information.

14. Apparatus for establishing a pan and scan pixel coordinate (P) that defines an image capture window useful for formatting an image (10) from a first aspect ratio to a second aspect ratio, comprising:

means for examining the image frame to ascertain a location therein of most pertinent activity;

means for establishing the pan and scan pixel coordinate in accordance with the location in the image of the most pertinent activity.

15. The apparatus according to claim 14 wherein the examining means examines at least one of:

video data, audio data, and other data associated with the image.

16. The apparatus according to claim 15 wherein the determining means establishes the pan and scan pixel coordinate (P) in accordance with the relationship:

P=X1*F(V)+X2*F(A)+X3*F(D)

where

X1=weighting factor assigned to the video data (0<

X1<

1)

X2=weighting factor assigned to the audio data (0<

X2<

1)

X3=weighting factor assigned to the other data(0<

X3<

1)

F(V)=coordinate predicted using video data

F(A)=coordinate predicted using audio data

F(D)=coordinate predicted using other data

with

X1+X2+X3=1.  

 

(2)

17. A method for formatting an incoming image frame having a first aspect ratio to yield a formatted image frame having a second aspect ratio, comprising the steps of:

establishing within the image frame a pan and scan pixel coordinate (P) that lies substantially coincident with a location in the image frame associated with most pertinent activity, and

panning and scanning the incoming image frame with an image capture window centered about the pan and scan pixel location, to format incoming image frame to yield the formatted image frame.

18. The method according to claim 17 wherein location in the image frame having the most pertinent activity is determined by examining at least one of:

video data, audio data and other data associated with the image frame.

19. The method according to claim 18 wherein the pan and scan pixel coordinate (P) is determined in accordance with the relationship:

P=X1*F(V)+X2*F(A)+X3*F(D)

where

X1=weighting factor assigned to the video data (0<

X1<

1)

X2=weighting factor assigned to the audio data (0<

X2<

1)

X3=weighting factor assigned to the other data(0<

X3<

1)

F(V)=coordinate predicted using video data

F(A)=coordinate predicted using audio data

F(D)=coordinate predicted using other data

with

X1+X2+X3=1.

20. Apparatus for formatting an incoming image frame having a first aspect ratio to yield a formatted image frame having a second aspect ratio.

means for establishing within the image frame a pan and scan pixel coordinate (P) that lies substantially coincident with a location in the image frame associated with most pertinent activity, and

means for panning and scanning the incoming image frame with an image capture window centered about the pan and scan pixel coordinate to format incoming image frame to yield the formatted image frame.

21. The apparatus according to claim 20 wherein the establishing means establishes the location in the image frame having the most pertinent activity by examining at least one of:

video data, audio data and other data associated with the image frame.

22. The method according to claim 21 wherein establishing means establishes the pan and scan pixel coordinate (P) in accordance with the relationship:

P=X1*F(V)+X2*F(A)+X3*F(D)

where

X1=weighting factor assigned to the video data (0<

X1<

1)

X2=weighting factor assigned to the audio data (0<

X2<

1)

X3=weighting factor assigned to the other data(0<

X3<

1)

F(V)=coordinate predicted using video data

F(A)=coordinate predicted using audio data

F(D)=coordinate predicted using other data

with

X1+X2+X3=1.