Image processing apparatus and method for displaying picture-in-picture with frame rate conversion

• US 7,142,252 B2
• Filed: 01/10/2002
• Issued: 11/28/2006
• Est. Priority Date: 03/10/2001
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Song, Suk-beom
• Primary Examiner(s): Tran, Trang U.
• Application Number:
  US10/044,123
• Publication Number:
  US 20020154658A1
• Time in Patent Office: 1,783 Days
• Field of Search: 348/555, 348/554, 348/556, 348/565, 348/512, 348/564, 348/567, 348714-716, 348/441, 348/445, 348/446
• US Class Current: 348/565
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/440281 by altering the temporal resolution, e.g. by frame skipping television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

An apparatus and method for implementing picture-in-picture with frame rate conversion includes an input buffer unit, a data synchronizing unit, first through third memories, and a memory control unit. The input buffer unit buffers input data and outputs buffered data as first data and first data enabling signals. The data synchronizing unit synchronizes the first data output with an output clock signal and outputs synchronized data as second data and second data enabling signals. The first memory multiplexes the second data and outputs stored data in response to a first memory enabling signal. The second memory writes and reads data output from the first memory in response to a frame buffer control signal. The third memory outputs stored data as a display signal. The memory control unit controls data flow between the memories and frame rates of the input data.

Claims

1. An image processing apparatus for displaying a plurality of input data asynchronously input through different channels on one display device and converting frame rates of the input data in accordance with an output frame rate of the display device, the apparatus comprising:

an input buffer unit for buffering input data which are externally and asynchronously input through at least two channels by different input clock signals and outputting buffered data as first data and first data enabling signals;

a data synchronizing unit for synchronizing the first data output from the input buffer unit with an output clock signal in response to the input clock signals and the first data enabling signals and outputting synchronized data as second data and second data enabling signals in response to each of the first data enabling signals;

a first memory for multiplexing the second data according to time sharing, storing the second data in different regions, and outputting the stored second data in response to a first memory enabling signal;

a second memory for writing and reading data output from the first memory in response to a frame buffer control signal;

a third memory for storing data output from the second memory and comprising a single output terminal for outputting the stored data as a display signal in response to a second memory enabling signal; and

a memory control unit for (i) detecting underflow conditions in the first memory and detecting overflow conditions in the third memory, (ii) generating the first memory enabling signal that is applied to the first memory to control data flow between the first memory and the second memory, (iii) generating the frame buffer control signal that is applied to the second memory to control frame rates of the input data and to control frame rates of the display signal in response to the underflow and overflow conditions, and (iv) generating the second memory enabling signal that is applied to the third memory to control data flow between the second memory and the third memory.

2. The apparatus of claim 1, wherein the data synchronizing unit includes a plurality of data synchronizing units for synchronizing the first data with the output clock signal, and the plurality of data synchronizing units comprises:

a write address counter for counting a write address of the first memory in response to the first data enabling signals and the input clock signals and outputting the counted write address;

a demultiplexer for demultiplexing the first data in response to the counted write address and selectively outputting the demultiplexed data;

a parallel buffer including a plurality of parallel registers for storing the demultiplexed first data;

a frequency conversion module for converting the frequency of the counted write address into the frequency region of the output clock signal and outputting a converted frequency write address;

a read address counter for counting a read address of the first memory in response to the second data enabling signals and the output clock signal and outputting the counted read address;

an underflow detecting unit for detecting underflow of the first memory from the converted frequency write address and the counted read address and outputting a detected result;

a graphic enabling signal generating unit for generating the second data enabling signals in response to the underflow; and

a multiplexer for selectively outputting data stored in the parallel buffer as the second data in response to the counted read address.

3. The apparatus of claim 1, wherein the input buffer unit comprises:

a first input buffer for buffering a graphic data input through a first channel in response to a graphic clock signal and outputting buffered data as a first graphic data and a first graphic enabling signal; and

a second input buffer for buffering a video data input through a second channel in response to a video clock signal and outputting buffered data as a first video data and a first video enabling signal.

4. The apparatus of claim 3, wherein the data synchronizing unit comprises:

a first data synchronizing unit for inputting the first graphic data in response to the graphic clock signal and synchronizing the first graph data with the output clock signal to generate a second graphic data and a second graphic enabling signal; and

a second data synchronizing unit for inputting the first video data in response to the video clock signal and synchronizing the first video data with the output clock signal to generate a second video data and a second video enabling signal.

5. The apparatus of claim 4, wherein the first memory is divided into different storing regions and is realized by a first first-in first-out (FIFO) for selectively storing the second graphic data and the second video data, which are output from the first and second data synchronizing units, in response to the first memory enabling signal.

6. The apparatus of claim 5, wherein the second memory is realized by a frame buffer for storing at least one frame of data stored in the first FIFO.

7. The apparatus of claim 6, wherein the third memory is realized by a second FIFO for outputting data output from the second memory as the display signal in response to the second memory enabling signal.

8. The apparatus of claim 7, wherein the memory control unit comprises:

a first FIFO control unit for generating the first memory enabling signal in response to the second graphic enabling signal, a delayed video enabling signal, and a first frame data enabling signal, detecting underflow between a write address and a read address of the first FIFO, and controlling data write and read of the first FIFO according to a detected result;

a second FIFO control unit for generating the second memory enabling signal in response to a second frame data enabling signal and an output enabling signal, detecting overflow between a write address and a read address of the second FIFO, and controlling data write and read of the second FIFO according to a detected result; and

a frame buffer control unit for comparing a frame rate of data input to the frame buffer with that of an output display signal, generating an input blocking signal for blocking data of the first and second input buffers in response to a compared result to apply the input blocking signal to the first and second input buffers, and generating the first and second frame data enabling signals and the frame buffer control signal in response to the underflow and the overflow.

9. The apparatus of claim 4, wherein the image processing apparatus further comprises a color space converting unit for converting the second video data output from the second data synchronizing unit into a converted video data, outputting the converted video data to a video region of the first memory, and delaying the second video enabling signal for a predetermined time to output the delayed second video enabling signal to the memory control unit and the first data synchronizing unit.

10. An image processing method for displaying a plurality of input data asynchronously input through different channels on one display device and converting frame rates of the input data in accordance with an output frame rate of the display device, the method comprising the steps of:

(a) buffering the plurality of input data by using each of input clock signals and synchronizing each of buffered data with an output clock signal;

(b) storing the plurality of input data synchronized with the output clock signal in a first memory in response to each of input enabling signals;

(c) comparing a write address of the first memory with a read address of the first memory to determine whether data stored in the first memory are stored in a second memory and to detect underflow conditions in the first memory;

(d) comparing frame rates of each of the plurality of input data with that of an output display signal to control data write and read of the second memory; and

(e) comparing a write address of a third memory with a read address of the third memory to determine whether output data of the second memory are stored in the second memory and to detect overflow conditions in the third memory, and outputting data stored in the third memory through a single output terminal of the third memory as a display signal for displaying on the display device.

11. The method of claim 10, wherein the step (b) comprises the steps of:

(b1) determining whether an enabling signal for a first input data among the plurality of input data is enabled;

(b2) storing the first input data synchronized with the output clock signal in the first memory if the enabling signal for the first input data is enabled;

(b3) determining whether an enabling signal for a second input data is enabled if the enabling signal for the first input data is disabled in step (b1); and

(b4) storing the second input data synchronized with the output clock signal in the first memory if the enabling signal for second input data is enabled.

12. The method of claim 10, wherein first and second input data are multiplexed according to time sharing and stored in different regions of the first memory in the step (b).

13. The method of claim 10, wherein the step (c) comprises the steps of:

(c1) determining whether underflow occurs between the write address of the first memory and the read address of the first memory;

(c2) stopping data output from the first memory to the second memory if it is determined that underflow occurs between the write address of the first memory and the read address of the first memory; and

(c3) storing one of a first input data and a second input data, which is stored in the first memory, in the second memory if underflow does not occur between the write address of the first memory and the read address of the first memory.

14. The method of claim 13, wherein the step (c1) comprises the steps of:

(c11) determining whether the write address of the first memory is larger than the read address of the first memory;

(c12) determining that underflow occurs in a case where the following condition;

WADD_—

F1−

RADD_—

F1<

TH1

 

is satisfied, if it is determined that the write address of the first memory is larger than the read address of the first memory; and

(c13) determining that underflow occurs in a case where the following condition;

N1+WADD_—

F1−

RADD_—

F1<

TH1

 

is satisfied, if it is determined that the write address of the first memory is not larger than the read address of the first memory in the step (c11);

wherein WADD_F1 is the write address of the first memory, RADD_F1 is the read address of the first memory, TH1 is a predetermined threshold value, and N1 is the size of an address for one of the first input data and the second input data of the first memory.

15. The method of claim 10, wherein the step (d) comprises the steps of:

(d1) determining whether a frame rate of one of first and second input data is larger than that of the output display signal;

(d2) intercepting input by blocking one frame of one of the first and second input data if it is determined that the frame rate of the input data is larger than that of the output display signal;

(d3) determining whether the frame rate of the input data is smaller than that of the output display signal if it is determined that the frame rate of the input data is not larger than that of the output display signal in the step (d1); and

(d4) reading and repeating one frame of one of the first and second input data, which is stored in the second memory, if it is determined that the frame rate of the input data is smaller than that of the output display signal.

16. The method of claim 15, wherein the first or second input data are blocked in a case where the following condition:

M−

F+Pi+R_—

fr>

F

is not satisfied in the step (d2), wherein M is a storage size of the second memory, F is an image size of one frame, Pi is the position of frame data being read when one frame data begin to be input in the second memory, and R_fr is an output frame rate/input frame rate.

17. The method of claim 15, wherein one frame of data stored in the second memory is repeated and read in a case where the following conditions:

$\mathrm{Pi}+\frac{1}{\mathrm{R\_fr}}><br><br>F,<br><br>\mathrm{and}$
Pi>

M−

F

are not satisfied in the step (d4), wherein M is a storage size of the second memory, F is an image size of one frame, Pi is the position of frame data being read when one frame data being to be input in the second memory, and R_fr is an output frame rate/input frame rate.

18. The method of claim 10, wherein the step (e) comprises the steps of:

(e

     1) determining whether overflow occurs between the write address of the third memory and the read address of the third memory;

(e2) stopping data output from the third memory if it is determined that overflow occurs between the write address of the third memory and the read address of the third memory;

(e3) storing data of the second memory in the third memory if it is determined that overflow does not occur between the write address of the third memory and the read address of the third memory; and

(e4) outputting data stored in the third memory as the display signal in accordance with an output display format.

19. The method of claim 18, wherein the step (e1) comprises the steps of:

(e11) determining whether the write address of the third memory is larger than the read address of the third memory;

(e12) determining that the overflow occurs in a case where the following condition;

N2+RADD_—

F2−

WADD_—

F2<

TH2

 

is satisfied, if it is determined that the write address of the third memory is larger than the read address of the third memory; and

(e13) determining that the overflow occurs in a case where the following condition;

RADD_—

F2−

WADD_—

F2<

TH2

 

is satisfied, if it is determined that the write address of the third memory is not larger than the read address of the third memory in step (c11),

wherein WADD_F2 is the write address of the second memory, RADD_F2 is the read address of the second memory, TH2 is a threshold value, and N2 is the size of an address of the third memory.

20. The method of claim 10, wherein first and second input data are a graphic data for indicating R/G/B and a video data for indicating a luminance signal (Y) and a color-difference signal (U/V), respectively.