Method and apparatus for processing a data series including processing priority data

• US 20040120345A1
• Filed: 07/24/2003
• Published: 06/24/2004
• Est. Priority Date: 03/17/1997
• Status: Active Grant

• Current Assignee: Hiroshi Arakawa, Minoru Etoh, Takao Yamaguchi
• Original Assignee: Hiroshi Arakawa, Minoru Etoh, Takao Yamaguchi
• Inventors: Yamaguchi, Takao, Arakawa, Hiroshi, Etoh, Minoru
• Granted Patent: US 7,502,070 B2
• Time in Patent Office: Days
• US Class Current: 370/465
• CPC Class Codes: H04L 1/1854 Scheduling and prioritising arrangements
• CPC Class Codes: H04L 1/1887 Scheduling and prioritising arrangements
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2368 : Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/238 : Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4341 : Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4363 : Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/4381 : Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/44008 : involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/6137 : involving transmission via a telephone network, e.g. POTS
• CPC Class Codes: H04N 21/643 : Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 21/6437 : Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/6543 : for forcing some client operations, e.g. recording remote booting in general G06F9/4416
• CPC Class Codes: H04N 21/6547 : comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/854 : Content authoring
• CPC Class Codes: H04N 21/85406 : involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 : Multipoint control units therefor
• CPC Class Codes: H04N 7/24 : Systems for the transmission of television signals using pulse code modulation H04N21/00 takes precedence
• CPC Class Codes: H04N 7/52 : Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

A reception control section 11 for receiving the information including data and its transmission format information from a memory or communication channel, a separating section 12 for analyzing and separating received information, a transmitting section 13 for transmitting information to a memory or transmission channel, a video extending section 14 for extending a video, and video-extension control section 15 control the processing state of said video extending section 14 for extending at least one or more videos and a video synthesizing apparatus constituted with a video synthesizing section 16 for synthesizing videos in accordance with extended information, an output section 17 for outputting a synthesized result, and a terminal control section 18 for controlling the above means makes it possible to synthesize a plurality of videos at the same time and correspond to a dynamic change of transmission format information.

Claims

1. An audio-video transmitting apparatus comprising transmitting means for transmitting the content concerned with a transmitting method and/or the structure of data to be transmitted or an identifier showing the content as transmission format information through a transmission line same as that of the data to be transmitted or a transmission line different from the data transmission line;

wherein

said data to be transmitted is video data and/or audio data.

2. The audio-video transmitting apparatus according to claim 1, wherein said transmission format information is included in at least one of data control information added to said data to control said data, transmission control information added to said data to transmit said data, and information for controlling the processing of the terminal side.

3. The audio-video transmitting apparatus according to claim 2, wherein at least one of said data control information, transmission control information, and information for controlling the processing of said terminal side is dynamically changed.

4. The audio-video transmitting apparatus according to claim 3, wherein

said data is divided into a plurality of packets, and

said data control information or said transmission control information is added not only to the head packet of said divided packets but also to a middle packet of them.

5. The audio-video transmitting apparatus according to claim 1, wherein an identifier showing whether to use timing information concerned with said data as information showing the reproducing time of said data is included in said transmission format information.

6. The audio-video transmitting apparatus according to claim 1, wherein said transmission format information is the structural information of said data and a signal which is output from a receiving apparatus receiving the transmitted structural information of said data and which can be received is confirmed and thereafter, said transmitting means transmits corresponding data to said receiving apparatus.

7. The audio-video transmitting apparatus according to claim 1, wherein said transmission format information include (1) an identifier for identifying a program or data to be used by a receiving apparatus later and (2) at least one of a flag, counter, and timer as information for knowing the point of time in which said program or data is used or the term of validity for using said program or data.

8. The audio-video transmitting apparatus according to claim 7, wherein said point of time in which said program or data is used is transmitted as transmission control information by using a transmission serial number for identifying a transmission sequence or as information to be transmitted by a packet differ nt from that of data to control terminal-side processing.

9. The audio-video transmitting apparatus according to claim 2 or 3, wherein

storing means for storing a plurality of contents concerned with said transmitting method and/or said structure of data to be transmitted and a plurality of its identifiers are included, and

said identifier is included in at least one of said data control information, transmission control information, and information for controlling terminal-side processing as said transmission format information.

10. The audio-video transmitting apparatus according to claim 2 or 3, wherein storing means for storing a plurality of contents concerned with said transmitting method and/or said structure of data to be transmitted are included, and

said contents are included in at least one of said data control information, transmission control information, and information for controlling terminal-side processing as said transmission format information.

11. The audio-video transmitting apparatus according to claim 1, 2, or 3, wherein a default identifier showing whether to change the contents concerned with said transmitting method and/or structure of data to be transmitted is added.

12. The audio-video transmitting apparatus according to claim 9, 10, or 11, wherein said identifier or said default identifier is added to a predetermined fixed-length region of information to be transmitted or said predetermined position.

13. An audio-video receiving apparatus comprising:

receiving means for receiving said transmission format information transmitted from the audio-video transmitting apparatus of any one of claims 1 to 12; and

transmitted-information interpreting means for interpreting said received transmission-format information.

14. The audio-video receiving apparatus according to claim 13, wherein

storing means for storing a plurality of contents concerned with said transmitting method and/or said structure of data to be transmitted and a plurality of its identifiers are included, and

the contents stored in said storing means are used to interpret said transmission format information.

19. The audio-video transmitting apparatus according to claim 1, wherein priority values can be changed in accordance with the situation by transmitting the offset value of information showing the priority for processing of said data.

15. An audio-video transmitting apparatus comprising:

information multiplexing means for controlling start and end of multiplexing the information for a plurality of logical transmission lines for transmitting data and/or control information is included;

wherein,

not only said data and/or control information multiplexed by said information multiplexing means but also control contents concerned with start and end of said multiplexing by said information multiplexing means are transmitted as multiplexing control information, and

said data includes video data and/or audio data.

16. The audio-video transmitting apparatus according to claim 15, wherein it is possible to select whether to transmit said multiplexing control information by arranging said information without multiplexing it before said data and/or control information or transmit said multiplexing control information through a transmission line different from the transmission line for transmitting said data and/or control information.

17. An audio-video receiving apparatus comprising:

receiving means for receiving said multiplexing control information transmitted from the audio-video transmitting apparatus of claim 15 and said multiplexed data and/or control information; and

separating means for separating said multiplexed data and/or control information in accordance with said multiplexing control information.

18. An audio-video receiving apparatus comprising:

main looking-listening means for looking at and listening to a broadcast program; and

auxiliary looking-listening means for cyclically detecting the state of a broadcast program other than the broadcast program looked and listened through said main looking-listening means;

wherein

said detection is performed so that a program and/or data necessary when said broadcast program looked and listened through said main looking-listening means is switched to other broadcast program can be smoothly processed, and

said data includes video data and/or audio data.

20. An audio-video receiving apparatus comprising:

receiving means for receiving encoded information to which the information concerned with the priority for processing under an overload state is previously added; and

priority deciding means for deciding a threshold serving as a criterion for selecting whether to process an object in said information received by said receiving means;

wherein

the timing for outputting said received information is compared with the elapsed time after start of processing or the timing for decoding said received information is compared with the elapsed time after start of processing to change said threshold in accordance with the comparison result, and

video data and/or audio data are or is included as said encoding object.

21. The audio-video receiving apparatus according to claim 20, wherein

retransmission-request-priority deciding means for deciding a threshold serving as a criterion for selecting whether to request retransmission of some of said information not received because it is lost under transmission when it is necessary to retransmit said information is included, and

said decided threshold is decided in accordance with at least one of the priority controlled by said priority deciding means, retransmission frequency, lost factor of information, insertion interval between in-frame-encoded frames, and grading of priority.

22. An audio-video transmitting apparatus comprising:

retransmission-priority deciding means for deciding a threshold serving as a criterion for selecting whether to request retransmission of some of said information not received because it is lost under transmission when retransmission of said unreceived information is requested is included, wherein

said decided threshold is decided in accordance with at least one of the priority controlled by the priority deciding means of said audio-video receiving apparatus of claim 20, retransmission frequency, lost factor of information, insertion interval between in-frame-encoded frames, and grading of priority.

23. An audio-video transmitting apparatus for transmitting said encoded information by using the priority added to said encoded information and thereby thinning it when (1) an actual transfer rate exceeds the target transfer rate of information for a video or audio or (2) it is decided that writing of said encoded information into a transmitting buffer is delayed as the result of comparing the elapsed time after start of transmission with a period to be decoded or output added to said encoded information.

24. A data processing method comprising the steps of:

inputting a data series including (1) time-series data for audio or video, (2) an inter-time-series-data priority showing the priority of the processing between said time-series-data values, and (3) a plurality of in-time-series-data priorities for dividing said time-series data value to show the processing priority between divided data values; and

performing processing by using said inter-time-series-data priority and said in-time-series-data priority together when pluralities of said time-series-data values are simultaneously present.

30. The data processing method according to any one of claims 24, 26, and 28, wherein said in-time-series-data priority is described in the header of a packet to perform priority processing.

32. The data processing method according to any one of claims 24, 26, and 28, wherein the range of a value capable of expressing said in-time-series-data priority is made variable to perform priority processing.

25. A data processing apparatus comprising:

receiving means for receiving a data series including (1) time-series data for audio or video, (2) an inter-time-series-data priority showing the priority of the processing between said time-series-data values, and (3) a plurality of in-time-series-data priorities for dividing said time-series data value to show the processing priority between divided data values; and

data processing means for performing processing by using said inter-time-series-data priority and said in-time-series-data priority together when pluralities of said time-series-data values are simultaneously present.

31. The data processing apparatus according to any one of claims 25, 27, and 29, wherein said in-time-series-data priority is described in the header of a packet to perform priority processing.

33. The data processing apparatus according to any one of claims 25, 27, and 29, wherein the range of a value capable of expressing said in-time-series-data priority is made variable to perform priority processing.

26. A data processing method comprising the steps of:

inputting a data series including (1) time-series data for audio or video, (2) an inter-time-series-data priority showing the priority of the processing between said time-series-data values, and (3) a plurality of in-time-series-data priorities for dividing said time-series data value to show the processing priority between divided data values; and

distributing throughput to each of said time-series-data values in accordance with said inter-time-series-data priority and moreover, adaptively deteriorating the processing quality of the divided data in said time-series data in accordance with said in-time-series-data priority so that each of said time-series-data values is kept within said distributed throughput.

27. A data processing apparatus comprising:

receiving means for receiving a data series including (1) time-series data for audio or video, (2) an inter-time-series-data priority showing the priority of the processing between said time-series-data values, and (3) a plurality of in-time-series-data priorities for dividing said time-series data value to show the processing priority between divided data values; and

data processing means for distributing throughput to each of said time-series-data values in accordance with said inter-time-series-data priority and moreover, adaptively deteriorating the processing quality of the divided data in said time-series data in accordance with said in-time-series-data priority so that each of said time-series-data values is kept within said distributed throughput.

28. A data processing method characterized by, when an in-time-series-data priority for a video is added every frame of said video and said video for each frame is divided into a plurality of packets,

adding said in-time-series-data priority only to the header portion of a packet for transmitting the head portion of a frame of said video accessible as independent information.

29. A data processing apparatus characterized by, when an in-time-series-data priority for a video is added every frame of said video and said video for each frame is divided into a plurality of packets,

adding said in-time-series-data priority only to the header portion of a packet for transmitting the head portion of a frame of said video accessible as independent information.

34. A data processing method comprising the steps of:

inputting a data series including time-series data for audio or video and an inter-time-series-data priority showing the processing priority between said time-series data values; and

processing priorities by using said inter-time-series-data priority as the value of a relative or absolute priority.

42. The data processing method according to claim 34 or 36, wherein

said priority processing is performed (1) by using said inter-time-series-data priority as the value of a relative priority when accumulating and using said inter-time-series-data priority and (2) by using said inter-time-series-data priority as the value of an absolute priority when transmitting said data.

44. The data processing method according to claim 34 or 36, wherein

an identifier classifies whether to express the value of said priority as a relative value or an absolute value.

48. The data processing method according to any one of claims 34, 36, and 46, wherein a packet-constituting method is decided in accordance with any one of said inter-time-series-data priority, in-time-series-data priority, and relational description between said time-series data values.

53. The data processing method according to claim 48, wherein high error protection is applied to a packet including the information for said high in-time-series-data priority or inter-time-series-data priority.

55. The data processing method according to claim 34 or 36, wherein

a priority added to a packet is used as a packet priority, and

said priority processing is performed by relating at least either of the values of said in-time-series-data priority and said inter-time-series-data priority to said packet priority.

57. The data processing method according to claim 34 or 36, wherein

said priority processing is performed by assigning a value lower than a character or control information to said time-series data as the value of said in-time-series-data priority or said inter-time-series-data priority.

35. A data processing apparatus characterized by:

inputting a data series including time-series data for audio or video and an inter-time-series-data priority showing the processing priority between said time-series data values; and

processing priorities by using said inter-time-series-data priority as the value of a relative or absolute priority.

43. The data processing apparatus according to claim 35 or 37, wherein

said priority processing is performed (1) by expressing said inter-time-series-data priority as the value of a relative priority when accumulating and using said inter-time-series-data priority and (2) by expressing said inter-time-series-data priority as the value of an absolute priority when transmitting said inter-time-series-data priority.

45. The data processing apparatus according to claim 35 or 37, wherein

an identifier classifies whether to express the value of said priority as a relative value or an absolute value.

49. The data processing apparatus according to any one of claims 35, 37, and 47, wherein a packet-constituting method is decided in accordance with any one of said inter-time-series-data priority, in-time-series-data priority, and relational description between said time-series data values.

54. The data processing apparatus according to claim 49, wherein high error protection is applied to a packet including the information for said high in-time-series-data priority or inter-time-series-data priority.

56. The data processing apparatus according to claim 35 or 37, wherein

a priority added to a packet is used as a packet priority, and

said priority processing is performed by relating at least either of the values of said in-time-series-data priority and said inter-time-series-data priority to said packet priority.

58. The data processing apparatus according to claim 35 or 37, wherein said priority processing is performed by assigning a value lower than a character or control information to said time-series data as the value of said in-time-series-data priority or said inter-time-series-data priority.

36. A data processing method comprising the steps of:

classifying time-series data values for audio or video;

inputting a data series including said time-series data and a plurality of in-time-series-data priorities showing the processing priority between said classified data values; and

processing priorities by using said in-time-series-data priority as the value of a relative or absolute priority.

37. A data processing apparatus characterized by:

classifying time-series data values for audio or video;

inputting a data series including said time-series data and a plurality of in-time-series-data priorities showing the processing priority between said classified data values; and

processing priorities by using said in-time-series-data priority as the value of a relative or absolute priority.

38. A data processing method comprising the steps of:

classifying time-series data values for audio or video;

encoding said classified data values;

inputting a data series describing an in-time-series-data priority serving as the value of an absolute priority in said encoded information and a in-time-series-data priority serving as the value of a relative priority in the header portion of a packet constituted with said encoded information; and

processing priorities.

39. A data processing apparatus characterized by:

classifying time-series data values for audio or video;

encoding said classified data values;

inputting a data series describing an in-time-series-data priority serving as the value of an absolute priority in said encoded information and a in-time-series-data priority serving as the value of a relative priority in the header portion of a packet constituted with said encoded information; and

processing priorities.

40. A data processing method comprising the steps of:

inputting a data series including time-series data for audio or video and an inter-time-series-data priority showing the processing priority between time series data values; and

processing priorities by relating one said inter-time-series-data priority or more to a TCP/IP logical channel.

41. A data processing apparatus characterized by:

inputting a data series including time-series data for audio or video and an inter-time-series-data priority showing the processing priority between time series data values; and

processing priorities by relating one said inter-time-series-data priority or more to a TCP/IP logical channel.

46. A data processing method comprising the steps of:

when one time-series data includes a plurality of sub-time-series data values, describing the relation between said sub-time-series data values and thereby defining a method for processing said sub-time-series data to perform priority processing.

47. A data processing apparatus characterized by, when one time-series data includes a plurality of sub-time-series data values, describing the relation between said sub-time-series data values and thereby defining a method for processing said sub-time-series data to perform priority processing.

50. A data processing method characterized by relating the sliced structure of a video to the structure of a packet and thereby, making a re-sync marker for resynchronization unnecessary.

51. A data processing apparatus characterized by relating the sliced structure of a video to the structure of a packet and thereby, making a re-sync marker for resynchronization unnecessary.

52. A data processing apparatus characterized by transmitting a method for relating time-series data for audio or video to a packet together with control information or said time-series data and thereby, defining relating of said time-series data to said packet.

59. A data processing method comprising the steps of:

successively inputting classified time-series data and its priority information; and

(1) when the information for said classified time-series data is damaged, performing retransmission request processing in order to request retransmission of said damaged data and (2) when said classified time-series data is continuously or frequently lost, applying said retransmission request processing only to high-priority data.

60. A data processing apparatus characterized by, successively inputting classified time-series data and its priority information;

and

(1) when the information for said classified time-series data is damaged, performing retransmission request processing in order to request retransmission of said damaged data and, (2) when said classified time-series data is continuously or frequently lost, applying said retransmission request processing only to high-priority data.

61. A data processing method comprising the step of:

successively inputting classified time-series data and its priority information; and

preferentially transmitting said high-priority data in accordance with the amount of said classified time-series data to be transmitted.

62. A data processing apparatus characterized by:

successively inputting classified time-series data and its priority information; and

preferentially transmitting said high-priority data in accordance with the amount of said classified time-series data to be transmitted.

63. A waveform data transmitting method comprising the steps of:

(a) dividing a plurality of decoding units constituting the waveform-data decoding process into a plurality of groups in accordance with the significance for maintaining quality and counting the execution frequency of an encoding unit corresponding to the decoding unit belonging to each group;

(b) receiving said counted result and transforming said result into a data string when encoding of waveform data for a predetermined period is completed; and

(c) outputting a code which is a waveform-data encoding result and said data string and transmitting the execution frequency of each processing unit every a plurality of groups to the receiving apparatus.

65. The waveform data transmitting method according to claim 63, wherein

pluralities of decoding units constituting a plurality of waveform-data decoding processes are divided into at least one indispensable processing or more and at least one dispensable processing or more (when this processing is omitted, waveforms are deteriorated but waveforms can be decoded), the execution frequency of said indispensable processing and that of dispensable processing are counted, and the execution frequencies of said indispensable and dispensable processings for each processing unit are transmitted to said receiving apparatus.

67. The video waveform data transmitting method according to claim 63, wherein a video is input as said waveform data.

77. The waveform data transmitting method according to claim 63, wherein

(d) a data string including the execution time of each group is input, and

(e) the execution frequency of each group is calculated in accordance with each execution time of said receiving means in order to complete decoding within the time required to transmit a code length decided by the designation by a rate controller or the like.

79. The video waveform data transmitting method according to claim 67, wherein

(d) the execution time of each group is estimated in accordance with the processing time required to encode a video and said each execution frequency;

and

(e) the processing time required to encode a video is estimated by using said execution time and the execution frequency of each group is calculated in which said processing time does not exceed the time usable to process a sheet of video determined in accordance with a frame rate given as the designation by a user.

81. The video waveform data transmitting method according to claim 63, wherein said counting result and the length of a code corresponding to waveform data for a predetermined period are received when generation of said code is completed to transform the result and length into a data string.

82. The video waveform data transmitting apparatus according to claim 63, wherein transforming means is included which receives said counting result of said counting means and the length of a code corresponding to waveform data for a predetermined period when generation of said code is completed to transform the result and length into a data string.

64. A waveform data transmitting apparatus comprising:

(a) counting means for dividing a plurality of decoding units constituting the waveform-data decoding process into a plurality of groups in accordance with the significance for maintaining quality and counting the execution frequency of an encoding unit corresponding to the decoding unit belonging to each group;

(b) transforming means for receiving said counted result and transforming said result into a data string when encoding of waveform data for a predetermined period is completed; and

(c) transmitting means for outputting a code which is a waveform-data encoding result and said data string;

wherein

the execution frequency of each processing unit is transmitted to the receiving apparatus every a plurality of groups.

66. The waveform data transmitting apparatus according to claim 64, wherein counting means for dividing a plurality of decoding units constituting a plurality of waveform-data decoding processes into at least one indispensable processing or more and at least one dispensable processing or more (when this processing is omitted, waveforms are deteriorated but waveforms can be decoded) and counting the execution frequency of said indispensable processing and that of dispensable processing is included and the execution frequencies of said indispensable and dispensable processings for each processing unit are transmitted to said receiving apparatus.

68. The video waveform data transmitting apparatus according to claim 64, wherein a video is input as said waveform data.

80. A video waveform data transmitting apparatus according to claim 68, wherein

(d) estimating means for estimating the execution time of each group in accordance with the processing time required to encode a video and each execution time output by counting means;

and

(e) deciding means for estimating the processing time required to encode a video by using said execution time and calculating the execution frequency of each group in which said processing time does not exceed the time usable to process a sheet of video determined in accordance with a frame rate given as the designation by a user.

69. A waveform data receiving method comprising the steps of:

(a) receiving a data string including the code of waveform data and the execution frequency of each decoding unit grouped in accordance with the significance for maintaining the quality of the waveform data decoded from said code and outputting said code and said execution frequency;

(b) estimating the execution time of each group in accordance with the processing time until obtaining a waveform after decoding said code and each of said execution frequencies obtained from said data string; and

(c) estimating the processing time required to decode a waveform by using the execution frequency and said execution time, calculating the reduced number of execution frequencies of groups in which said processing time does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time) in accordance with each execution time output by said receiving means and each execution time output by said estimating means, estimating the time required for decoding, and reducing the execution frequency of each group so as to complete decoding within said designated time.

73. The video waveform data receiving method according to claim 69, wherein a video is output as said waveform data.

75. The video waveform data receiving method according to claim 69, wherein (d) the execution time of each group obtained through estimation is output.

83. The waveform data receiving method according to claim 69, wherein a data string including a code corresponding to the waveform data for a predetermined period, the execution frequency of each decoding unit grouped in accordance with the significance for maintaining the quality of the waveform data decoded from said code, and the length of said code is received and said code, execution frequency, and code length are output to reduce the execution frequency of dispensable processing so that the time required for decoding does not exceed a code transmission time obtained from the length and transmission rate of said code.

70. A waveform data receiving apparatus comprising:

(a) receiving means for receiving a data string including the code of waveform data and the execution frequency of each decoding unit grouped in accordance with the significance for maintaining the quality of the waveform data decoded from said code and outputting said code and said execution frequency;

(b) estimating means for estimating the execution time of each group in accordance with the processing time until obtaining a waveform after decoding said code and each of said execution frequencies obtained from said data string; and

(c) frequency reducing means for estimating the processing time required to decode a waveform by using said execution frequency and said execution time, calculating the reduced number of execution frequencies of the groups in which said processing time does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time) in accordance with each execution time output by said receiving means and each execution time output by said estimating means;

wherein the time required for decoding is estimated and the execution frequency of each group is reduced so as to complete decoding within said designated time.

74. The video waveform data receiving apparatus according to claim 70, wherein a video is output as said waveform data.

76. The video waveform data receiving apparatus according to claim 70, wherein (d) the execution time of each group obtained by estimating means is output.

84. The waveform data receiving apparatus according to claim 70, wherein receiving means is included which receives a data string including a code corresponding to the waveform data for a predetermined period, the execution frequency of each decoding unit grouped in accordance with the significance for maintaining the quality of the waveform data decoded from said code, and the length of said code and outputs said code, execution frequency, and code length to reduce the execution frequency of dispensable processing so that the time required for decoding does not exceed a code transmission time obtained from the length and transmission rate of said code.

71. A waveform data receiving method comprising the steps of:

(a) receiving a data string including the code of waveform data and the execution frequencies of indispensable and dispensable processings for decoding and outputting said code and said execution frequencies;

(b) estimating the execution frequencies of said indispensable and dispensable processings in accordance with the processing time until obtaining a waveform after decoding said code and each of said execution frequencies obtained from said data string; and

(c) estimating the processing time required to decode a waveform by using said execution frequency and said execution time, calculating the reduced number of execution frequencies of said dispensable processing in which said processing time does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time) in accordance with each execution frequency output by said receiving means and each execution time output by said estimating means, estimating the time required for decoding in accordance with each estimated execution time, and reducing the execution frequency of said dispensable processing so as to complete decoding within said designated time.

72. A waveform data receiving apparatus comprising:

(a) receiving means for receiving a data string including the code of waveform data and the execution frequencies of indispensable and dispensable processings for decoding and outputting said code and said execution frequencies;

(b) estimating means for estimating the execution frequencies of said indispensable and dispensable processings in accordance with the processing time until obtaining a waveform after decoding said code and each of said execution frequencies obtained from said data string; and

(c) frequency reducing means for estimating the processing time required to decode a waveform by using said execution frequency and said execution time and calculating the reduced number of execution frequencies of said dispensable processing in which said processing time does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time) in accordance with each execution frequency output by said receiving means and each execution time output by said estimating means;

wherein

the time required for decoding is estimated in accordance with each estimated execution time and the execution frequency of said dispensable processing is reduced so as to complete decoding within said designated time.

78. The waveform data transmitting apparatus comprising:

(d) receiving means for inputting a data string constituted with the execution time of each group; and

(e) deciding means for calculating the execution frequency of each group in accordance with each execution time of said receiving means in order to complete decoding within the time required to transmit a code decided by the designation by a rate controller or the like.

85. A waveform data receiving method for receiving the code of waveform data and decoding and outputting the waveform, comprising the steps of:

(a) constituting a data string including the designation for selecting a processing unit having an execution time shorter than that of the encoding unit included in said code every encoding unit corresponding to a processing unit constituting the decoding process so that the processing time required to decode a waveform does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(b) transmitting said data string to communicate to the transmitting side that a code for completing decoding within said designated time is transmitted.

86. A waveform data receiving apparatus for receiving the code of waveform data and decoding and outputting said waveform, comprising:

(a) designated data constituting means for constituting a data string including the designation for selecting a processing unit having an execution time shorter than that of the encoding unit included in said code every encoding unit corresponding to a processing unit constituting the decoding process so that the processing time required to decode a waveform does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(b) transmitting means for transmitting said data string;

wherein

it is communicated to the transmitting side that a code for completing decoding within said designated time is transmitted.

87. A waveform data transmitting method for encoding a waveform and outputting said code, comprising the steps of:

(a) receiving a data string including the designation for a processing unit to be selected for each processing unit constituting the encoding process; and

(b) extracting said designation from said data string, encoding a waveform by using the processing unit specified in accordance with said designation, and outputting a code.

88. A waveform data transmitting apparatus for encoding a waveform and outputting said code, comprising:

(a) receiving means for receiving a data string including the designation for a processing unit to be selected for each processing unit constituting the encoding process; and

(b) extracting means for extracting said designation from said data string;

wherein

a waveform is encoded by using the processing unit specified in accordance with said designation to output a code.

89. A waveform data receiving method for receiving the code of waveform data and decoding and outputting a waveform, comprising the steps of:

(a) counting the execution frequency of each processing unit constituting the waveform decoding process;

(b) estimating the execution time for each processing unit in accordance with said execution frequency and processing time required to decode a waveform;

(c) constituting a data string including the designation for selecting a processing unit having an execution time shorter than that of the encoding unit included in said code every encoding unit corresponding to the processing unit constituting the decoding process so that the processing time required to decode a waveform does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(d) transmitting said data string;

wherein

it is communicated to the transmitting method that a code for completing decoding within said designated time is transmitted.

90. A waveform data receiving apparatus for receiving the code of waveform data and decoding and outputting a waveform, comprising:

(a) counting means for counting the execution frequency of each processing unit constituting the waveform decoding process;

(b) estimating means for estimating the execution time for each processing unit in accordance with said execution frequency and processing time required to decode a waveform;

(c) designated-data constituting means for constituting a data string including the designation for selecting a processing unit having an execution time shorter than that of the encoding unit included in said code every encoding unit corresponding to the processing unit constituting the decoding process so that the processing time required to decode a waveform does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(d) transmitting means for transmitting said data string;

wherein

it is communicated to the transmitting side that a code for completing decoding within said designated time is transmitted.

91. A video waveform data receiving method for receiving the code of a video and decoding and outputting said video, comprising the steps of:

(a) constituting a data string including the designation for replacing the movement compensating method used to encode a video with the movement compensation processing having an execution time shorter than that of the movement compensation processing included in said code so that the processing time required to decode a video does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(b) transmitting said data string;

wherein

it is communicated to the transmitting side that a code for completing encoding within said designated time is transmitted.

92. A video receiving apparatus for receiving the code of a video and decoding and outputting said video, comprising:

(a) designated-data constituting means for constituting a data string including the designation for replacing the movement compensating method used to encode a video with the movement compensation processing having an execution time shorter than that of the movement compensation processing included in said code so that the processing time required to decode a video does not exceed the time required to receive said code or the time from start of receiving said code up to start of receiving the next code (this is referred to as designated time); and

(b) transmitting means for transmitting said data string;

wherein

it is communicated to the transmitting side that a code for completing encoding within said designated time is transmitted.

93. A video transmitting method for encoding a video and outputting said code, comprising the steps of:

(a) receiving a data string including the designation for the processing to be selected by using the movement compensating processing constituting the decoding process; and

(b) extracting said designation from said data string;

wherein

encoding of a video is executed by using the movement compensating processing specified in accordance with said designation to output a code.

94. A video transmitting apparatus for encoding a video and outputting said code, comprising the steps of:

(a) receiving means for receiving a data string including the designation for the processing to be selected by using the movement compensating processing constituting the decoding process; and

(b) extracting means for extracting said designation from said data string;

wherein

encoding of a video is executed by using the movement compensating processing specified in accordance with said designation to output a code.