Subpicture stream change control

• US 6,512,552 B1
• Filed: 03/29/1999
• Issued: 01/28/2003
• Est. Priority Date: 03/29/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Subramanian, Pattabiraman
• Primary Examiner(s): Miller, John
• Assistant Examiner(s): Lo, Linus H
• Application Number:
  US09/280,727
• Time in Patent Office: 1,401 Days
• Field of Search: H04/N.5448, 348/564, 348/565, 348/512, 348/567, 348/588, 348/473, 348/474, 348/423.1, 386/98, 386/125, 386/126, 386/95, 386/37, 386/84, 386/123
• US Class Current: 348/564
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4384 involving operations to reduce the access time, e.g. fast-tuning for reducing channel switching latency
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

A method of operating a digital video processor to continuously demultiplex, store and decode audio and video data streams and play back corresponding frames of audio and video. The processor further continuously demultiplexes, stores and decodes a first subpicture data stream and plays back first subpicture data in synchronization with the playback of frames of video data. PTS values associated with SPU'"'"'s in the first subpicture data stream are continuously stored. The processor then detects a command instructing a transition from a first subpicture data stream to a second subpicture data stream and stores a PTS value associated with a first complete SPU of the second subpicture data stream so that it follows a PTS value associated with a last complete SPU in the first subpicture data stream. The processor then continuously stores PTS values associated with the SPU'"'"'s in the second subpicture data stream. The first subpicture data stream is played back in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the first subpicture data stream; and the second subpicture data stream is played back in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the second subpicture data stream. The above process is executed without interrupting the continuous decoding and playback of the audio and video data streams.

Claims

1. A method of operating a digital video processor in response to audio, video and subpicture data streams, the digital video processor including a CPU and a memory and the subpicture data streams comprising a serial stream of subpicture units (“

SPU'"'"'s”

) wherein a start of each of the SPU'"'"'s has a presentation time stamp (“

PTS”

) associated therewith, the method comprising;

continuously demultiplexing, storing and decoding the audio and video data streams and playing back corresponding frames of audio and video;

continuously demultiplexing, storing and decoding a first subpicture data stream and playing back first subpicture data in synchronization with the playback of frames of video data;

continuously storing PTS values associated with SPU'"'"'s in the first subpicture data stream;

detecting a command instructing a transition form the first subpicture data stream to a second subpicture data stream;

storing a PTS value associated with a first complete SPU of the second subpicture data stream so that it follows a PTS value associated with a last complete SPU in the first subpicture data stream;

continuously storing PTS values associated with the SPU'"'"'s in the second subpicture data stream;

playing back the first subpicture data stream in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'of the first subpicture data stream; and

playing back the second subpicture data stream in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the second subpicture data stream, thereby providing a transition from the first to the second subpicture data streams without playing back any partial SPU'"'"'s in either of the first or the second subpicture data streams and without any interruption to the continuous demultiplexing, storing and decoding of the audio and video data streams.

2. A method of claim 1 further comprising continuously storing with each stored PTS value associated with SPU'"'"'s in both the first and the second subpicture data streams a memory location of the current PTS value being stored.

3. A method of claim 1 further comprising continuously storing with each stored PTS value associated with SPU'"'"'s in both the first and the second subpicture data streams a number of bytes from an immediately preceding memory location storing a PTS value to a memory location of the current PTS value being stored.

4. A method of claim 1 further comprising:

determining a PTS value of a first complete SPU in the second subpicture data stream;

successively comparing the PTS value of the first complete SPU in the second subpicture data stream to the stored PTS values of the complete SPU'"'"'s in the first subpicture data stream beginning with a most recently stored PTS value; and

identifying a closest stored PTS value of the complete SPU'"'"'s in the first subpicture data stream that immediately precedes the PTS value of the first complete SPU in the second subpicture data stream.

5. A method of claim 1 further comprising detecting a PTS value of the first complete SPU in the second subpicture data stream being greater than the stored PTS values of the complete SPU'"'"'s in the first subpicture data stream to identify the closest stored PTS value of the complete SPU in the first picture data stream.

6. A method of claim 1 further comprising detecting a PTS value of the first complete SPU in the second subpicture data stream being greater than or equal to the stored PTS values of the complete SPU'"'"'s in the first subpicture data stream to identify the closest stored PTS value of the complete SPU in the first picture data stream.

7. A method of operating a digital video processor in response to audio, video and subpicture data streams, the digital video processor including a CPU and a memory and the subpicture data streams comprising a serial stream of subpicture units (“

SPU'"'"'s”

) wherein a start of each of the SPU'"'"'s has a presentation time stamp (“

PTS”

) associated therewith, the method comprising;

continuously demultiplexing, storing and decoding the audio and video data streams and playing back corresponding frames of audio and video;

continuously demultiplexing, storing and decoding a first subpicture data stream and playing back first subpicture data in synchronization with the playback of frames of video data;

continuously storing PTS values associated with SPU'"'"'s in the first subpicture data stream;

detecting a command instructing a transition from the first subpicture data stream to a second subpicture data stream;

determining a PTS value of a first complete SPU in the second subpicture data stream;

comparing the PTS value of the first complete SPU in the second subpicture data stream to the stored PTS values of the complete SPU'"'"'s in the first subpicture data stream;

identifying a closest stored PTS value of the complete SPU'"'"'s in the first subpicture data stream that immediately precedes the PTS value of the first complete SPU in the second subpicture data stream;

storing the PTS value associated with the first complete SPU of the second subpicture data stream so that it follows the closest stored PTS value associated with a last complete SPU in the first subpicture data stream;

continuously storing other PTS values associated with the SPU'"'"'s in the second subpicture data stream;

playing back the SPU'"'"'s in the first subpicture data stream up through the SPU having the closest PTS value in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the first subpicture data stream and up through the closest PTS value; and

playing back the second subpicture data stream starting with the first complete SPU in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the second subpicture data stream, thereby providing a smooth transition from the first to the second subpicture data streams without interfering with the continuous demultiplexing, storing and decoding the audio and video data streams and playing back of corresponding frames of audio and video.

8. A method of operating a digital video processor including a CPU and a memory to change from a first subpicture data stream to a second subpicture data stream, each of the first and second subpicture data streams comprising a respective serial stream of subpicture units (“

SPU'"'"'s”

), a start of each of the SPU'"'"'s having a presentation time stamp (“

PTS”

) associated therewith, the method comprising;

creating a subpicture PTS table in the memory for storing successive PTS values for successive SPU'"'"'s in the first subpicture data stream and respective associated byte counts including a number of bytes between successive SPU'"'"'s;

detecting a command requesting a change from the first to the second subpicture data stream;

comparing a first PTS value in the second subpicture data stream to the PTS values of the first subpicture data stream stored in the subpicture PTS table;

storing the first PTS value in the second subpicture data stream in a subpicture PTS table location having a first stream PTS value greater than the first PTS value in the second subpicture data stream; and

storing in a location in the subpicture PTS table associated with the first PTS value in the second subpicture data stream a byte count representing the sum of all of the byte counts in the first and second subpicture data streams demultiplexed up to the first PTS value in the second subpicture data stream.

9. The method of claim 8 further comprising storing the first PTS value in the second subpicture data stream in a subpicture PTS table location having a first stream PTS value greater than or equal to the first PTS value in the second subpicture data stream.

10. The method of claim 8 further comprising:

receiving a PTS interrupt from a demultiplexer in response to the demultiplexer processing a current SPU;

testing whether the command requesting a change from the first to the second subpicture data stream has been received;

writing a current PTS value associated with the current SPU in the subpicture PTS table in response to determining that the command has not been received;

determining a current byte count including a number of bytes from a start of a prior SPU to the first SPU;

reading a base address in byte counts from the subpicture PTS table associated with the prior SPU, the base address identifying a location of subpicture data in the memory associated with the prior SPU;

combining the current byte count with the base address to form a current base address identifying a location of subpicture data in memory associated with the current SPU; and

writing the current base address into a location in the subpicture PTS table associated with the current PTS value of the current SPU.

11. The method of claim 10 wherein the subpicture PTS table has a read pointer and a write pointer, the method further comprising:

comparing the locations of the read and write pointers in the subpicture PTS table in response to determining that the command requesting a change from the first to the second subpicture data stream has been received;

decrementing the write pointer one location within the subpicture PTS table in response to the read and write pointers not being at adjacent locations within the subpicture PTS table;

saving a reference base address associated with a current location of the write pointer;

comparing again the locations of the read and write pointers in the subpicture PTS table;

decrementing the write pointer another location within the subpicture PTS table in response to the read and write pointers again not being at adjacent locations within the subpicture PTS table; and

comparing another PTS value from the first subpicture in response stored in the other location in the subpicture PTS table with the first PTS value in the second subpicture data stream.

12. The method of claim 11 further comprising:

writing the reference base address at the other location of the subpicture PTS table in response to the value of the current PTS not being less than the first PTS value in the second subpicture data stream;

incrementing the write pointer to the one location in the subpicture PTS table; and

writing in the one location of the subpicture PTS table, the value of the first PTS value in the second subpicture data stream and a base address value equal to a sum of the reference base address and a number of byte counts detected from the reference base address to the first PTS value in the second subpicture data stream.

13. The method of claim 12 comprising:

comparing again the locations of the read and write pointers in the subpicture PTS table in response to the value of the first PTS value in the second subpicture data stream being less than the other PTS value;

decrementing the write pointer a further location within the subpicture PTS table in response to the read and write pointers again not being at adjacent locations within the subpicture PTS table; and

comparing a further PTS value from the first subpicture data stream stored in the other location in the subpicture PTS table with a value of the first PTS value in the second subpicture data stream.

14. The method of claim 13 further comprising iterating the methods of claims 5 and 6 for all PTS values in the subpicture PTS table that are greater than the first PTS value in the second subpicture data stream.

15. The method of claim 14 further comprising iterating the methods of claims 5 and 6 for all PTS values in the subpicture PTS table that are greater than or equal to the first PTS value in the second subpicture data stream.

16. A method of operating a digital video processor including a CPU and a memory to change from one subpicture data stream to a second subpicture data stream, each of the first and second subpicture data streams comprising a respective serial stream of subpicture units (“

SPU'"'"'s”

), a start of each of the SPU'"'"'s being defined by a presentation time stamp (“

PTS”

) associated therewith, the method comprising;

creating a subpicture PTS table in the memory for storing successive PTS values for successive SPU'"'"'s in the first subpicture data stream and respective associated byte counts including a number of bytes between successive SPU'"'"'s;

detecting a command requesting a change from the first to the second subpicture data stream;

comparing the locations of the read and write pointers in the subpicture PTS table in response to determining that the command has been received;

decrementing the write pointer in the subpicture PTS through successive locations in the subpicture PTS table in response to the read and write pointers not being in adjacent locations in the subpicture PTS table with each decrement of the write pointer;

with each decrement of the write pointer, comparing a first PTS value in the second subpicture data stream to a PTS value associated with a current location of the write pointer;

storing the first PTS value in the second subpicture data stream in the subpicture PTS table location having a first stream PTS value greater than the first PTS value in the second subpicture data stream; and

storing in a location in the subpicture PTS table associated with the first PTS value in the second subpicture data stream a byte count representing the sum of all of the byte counts in the first and second subpicture data streams demultiplexed up to the first PTS value in the second subpicture data stream.

17. The method of claim 16 further comprising storing the first PTS value in the second subpicture data stream in the subpicture PTS table location having a first stream PTS value greater than or equal to the first PTS value in the second subpicture data stream.

18. A method of operating a digital video processor including a CPU and a memory to change from one subpicture data stream to a second subpicture data stream, each of the first and second subpicture data streams comprising a respective serial stream of subpicture units (“

SPU'"'"'s”

), each of the SPU'"'"'s having a presentation time stamp (“

PTS”

) associated therewith, the method comprising;

demultiplexing the first subpicture data stream including first stream PTS values;

storing successive first stream PTS values in successive locations in a PTS table in the memory;

storing in locations in the PTS table associated with each of the first stream PTS values a respective byte count including a sum of byte counts between successive SPU'"'"'s in the first subpicture data stream;

detecting a command requesting a change from the first to the second subpicture data stream;

demultiplexing the second subpicture data stream including second stream PTS values;

comparing a first second stream PTS value to first stream PTS values stored in the PTS table;

storing the first second stream PTS value in a PTS table location having a first stream PTS value greater than the first second stream PTS value; and

storing in a location in the PTS table associated with the first second stream PTS value stream a byte count including a sum of all of the byte counts in the first and second subpicture data streams demultiplexed up to the first second stream PTS value.

19. The method of claim 18 further comprising storing the first second stream PTS value in a subpicture PTS table location having a first stream PTS value greater than or equal to the first second stream PTS value.

20. A digital video processor being responsive to audio, video and subpicture data streams, the subpicture data streams comprising a serial stream of subpicture units (“

SPU'"'"'s”

) wherein a start of each of the SPU'"'"'s has a presentation time stamp (“

PTS”

) associated therewith, the processor comprising;

a demultiplexer for continuously demultiplexing the audio and video data streams and subpicture data streams;

a memory electrically connected to the demultiplexer for storing demultiplexed audio and video data and demultiplexed subpicture data, the memory further storing PTS values associated with SPU'"'"'s in a first subpicture data stream;

a CPU electrically connected to the memory and the demultiplexer for detecting a command requesting a transition from a first subpicture data stream to a second subpicture data stream, the CPU controlling the demultiplexer and the memory to store a PTS value associated with a first complete SPU of the second subpicture data stream so that it follows a PTS value associated with a last complete SPU in the first subpicture data stream and thereafter continuously storing PTS values associated with the SPU'"'"'s in the second subpicture data stream;

a decoder electrically connected to the CPU and the memory for decoding the demultiplexed audio and video data and continuously playing back corresponding frames of audio and video;

a subpicture generator and blender electrically connected to the CPU, the decoder and the memory and responsive to the demultiplexed subpicture data streams for playing back the first subpicture data stream in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the first subpicture data stream and also playing back the second subpicture data stream in synchronization with corresponding frames of video according to the stored PTS values associated with the SPU'"'"'s of the second subpicture data stream, thereby providing a transition from the first to the second subpicture data streams without playing back any partial SPU'"'"'s in either of the first or the second subpicture data streams and without any interruption to the continuous demultiplexing, storing and decoding of the audio and video data streams.