Transmitting apparatus and method, receiving apparatus and method, transmitting and receiving apparatus and method, record medium and signal

• US 7,009,579 B1
• Filed: 08/09/2000
• Issued: 03/07/2006
• Est. Priority Date: 08/09/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kanemaru, Masanori, Kondo, Tetsujiro, Kobayashi, Naoki
• Primary Examiner(s): Eisen, Alexander
• Assistant Examiner(s): NGUYEN, KIMNHUNG T
• Application Number:
  US09/807,114
• Time in Patent Office: 2,036 Days
• Field of Search: 345/1.2, 345 33- 34, 345/2.1, 345698-699, 455424-426, 455/428, 348/14.12
• US Class Current: 345/1.2
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 : for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6131 : involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/835 : Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

On the transmission side, a background picture and objects #1 to #3 are transmitted at a transmission rate R/4 each. On the reception side, a picture composed of the objects #1 to #3 and the background picture is displayed with a particular spatial resolution and a particular temporal resolution. In this case, on the reception side, when the object #1 is dragged at particular time t1, on the transmission side, as shown in FIG. 16 (A), the transmission of the background picture and the objects #2 and #3 is stopped. Only the object #1 is transmitted at the transmission rate R of the transmission path. Thus, a picture of which the spatial resolution of the object #1 dragged is improved is displayed at the sacrifice of the temporal resolution of the picture.

Claims

1. A transmitting apparatus for transmitting data to a receiving apparatus, comprising:

receiving means for receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

controlling means for improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction and the level direction corresponding to the control information containing the particular point of the data; and

transmitting means for transmitting the data, of which the resolutions have been controlled corresponding to the control information, to the receiving apparatus.

2. The transmitting apparatus as set forth in claim 1,

wherein said controlling means controls the resolutions in the temporal direction and the spatial direction of the picture data transmitted to the receiving apparatus corresponding to the control information.

3. The transmitting apparatus as set forth in claim 2,

wherein said transmitting means transmits the data to the receiving apparatus through a predetermined transmission path at a predetermined transmission rate, and

wherein said controlling means controls the resolutions of the data corresponding to the control information so that the transmission rate of the data does not exceed the predetermined transmission rate.

4. The transmitting apparatus as set forth in claim 3,

wherein said transmitting means transmits picture data to the receiving apparatus through a predetermined transmission path at a predetermined transmission rate,

wherein the receiving apparatus displays the picture data transmitted from said transmitting means,

wherein the control information contains a temporal and special position of the picture data displayed by the receiving apparatus, and

wherein said controlling means improves the spatial resolution of a considered area that contains the temporal and spatial position of the picture data and deteriorates the temporal resolution corresponding to the control information so that the transmission rate of the picture data does not exceed the predetermined transmission rate.

5. The transmitting apparatus as set forth in claim 4, further comprising:

background picture data extracting means for extracting background picture data from the picture data transmitted to the receiving apparatus,

wherein said controlling means improves the spatial resolution of the background picture data when the temporal and spatial position contained in the control information represents the background picture data.

6. The transmitting apparatus as set forth in claim 5, further comprising:

object picture data extracting means for extracting object picture data from the picture data corresponding to the difference between the picture data and the background picture data transmitted to the receiving apparatus,

wherein said controlling means improves the spatial resolution of the object picture data when the temporal and spatial position contained in the control information represents the object picture data.

7. The transmitting apparatus as set forth in claim 6, further comprising:

combining means for combining the background picture data and the object picture as combined data,

wherein said transmitting means transmits the combined data to the receiving apparatus.

8. The transmitting apparatus as set forth in claim 1, further comprising:

inputting means for inputting the data.

9. The transmitting apparatus as set forth in claim 8,

wherein the data is picture data, and

wherein said inputting means is photographing means for photographing a picture and outputting the picture data.

10. The transmitting apparatus as set forth in claim 1,

wherein the transmitting apparatus is a portable telephone.

11. The transmitting apparatus as set forth in claim 1, further comprising:

analyzing means for analyzing the preferences of the user of the receiving apparatus,

wherein said controlling means controls the resolutions of the data corresponding to the analyzed result of said analyzing means.

12. The transmitting apparatus as set forth in claim 11,

wherein the receiving apparatus outputs the data transmitted from said transmitting means,

wherein said controlling means contains a considered point of the data that is output to the receiving apparatus, and

wherein said analyzing means analyzes the preferences of the user corresponding to the considered point.

13. The transmitting apparatus as set forth in claim 12,

wherein said analyzing means has:

feature amount extracting means for extracting a feature amount of a considered area that contains a considered point of the data; and

area detecting means for detecting a predetermined area corresponding to the preference of the user from the data corresponding to the feature amount, and

wherein said controlling means controls the resolutions of the predetermined area of the data.

14. The transmitting apparatus as set forth in claim 13, further comprising:

histogram storing means for storing a histogram of the feature amount,

wherein said area detecting means detects the predetermined area corresponding to the histogram.

15. The transmitting apparatus as set forth in claim 13,

wherein said transmitting means transmits picture data to the receiving apparatus through a predetermined transmission path at a predetermined transmission rate,

wherein the receiving apparatus displays the picture data transmitted from said transmitting means, and

wherein said controlling means improves the spatial resolution of the predetermined area of the picture data and deteriorates the temporal resolution so that the transmission rate of the picture data does not exceed the predetermined transmission rate.

16. The transmitting apparatus as set forth in claim 14,

wherein said area detecting means detects an area having the same as or similar to the feature amount with the largest frequency of the histogram as the predetermined area.

17. The transmitting apparatus as set forth in claim 16,

wherein said transmitting means transmits picture data to the receiving apparatus through a predetermined transmission path at a predetermined transmission rate,

wherein the receiving apparatus displays the picture data transmitted from said transmitting means, and

wherein said controlling means improves the spatial resolution of the predetermined area of the picture data and deteriorates the temporal resolution so that the transmission rate of the picture data does not exceed the predetermined transmission rate.

18. The transmitting apparatus as set forth in claim 15,

wherein said feature amount extracting means extracts at least one of motion information, depth information, position information, color information, and shape information of a considered area that contains the considered point of the picture data as the feature amount.

19. The transmitting apparatus as set forth in claim 18,

wherein said feature amount extracting means extracts a plurality of motion information, depth information, position information, color information, and shape information of a considered area that contains the considered point of the picture data as a plurality of feature amounts, and generates a feature amount vector composed of the plurality of feature amounts.

20. The transmitting apparatus as set forth in claim 12,

wherein said analyzing means has:

categorizing means for categorizing the data corresponding to a considered area that contains the considered point of the data,

wherein said analyzing means analyzes the preference of the user corresponding to the analyzed result of said categorizing means.

21. The transmitting apparatus as set forth in claim 2,

wherein the picture data is object encoded.

22. A receiving apparatus for receiving data transmitted from a transmitting apparatus, comprising:

outputting means for outputting the received data;

transmitting means for transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving means for receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

23. The receiving apparatus as set forth in claim 22,

wherein the data is picture data;

and

wherein said outputting means is displaying means for displaying the picture data.

24. The receiving apparatus as set forth in claim 23, further comprising:

considered point detecting means for detecting a considered point of the user from the picture data displayed by said displaying means,

wherein said transmitting means transmits the considered point as the control information to the transmitting apparatus.

25. The receiving apparatus as set forth in claim 24,

wherein said considered point detecting means detects the position designated by said designating means as the considered point.

26. The receiving apparatus as set forth in claim 23, further comprising:

picture data storing means for storing picture data received by said receiving means; and

controlling means for causing picture data stored in said picture data storing means to be displayed by said displaying means when the resolutions of the picture data stored in said picture data storing means are higher than the resolutions of the picture data received by said receiving means.

27. The receiving apparatus as set forth in claim 26,

wherein said controlling means causes the picture data received by said receiving means to be overwritten to said picture data storing means the picture data received by said receiving means to be displayed by said displaying means when the resolutions of the picture data stored in said picture data storing means are lower than the resolutions of the picture data received by said receiving means, the picture data stored in said picture data storing means corresponding to the picture data received by said receiving means.

28. A transmitting and receiving apparatus having a transmitting apparatus for transmitting data and a receiving apparatus for receiving the data,

wherein the transmitting apparatus comprises:

receiving means for receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

controlling means for improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in at least one of the rest of those directions corresponding to the control information containing the particular point of the data; and

transmitting means for transmitting the data of which the resolutions have been controlled corresponding to the control information to the receiving apparatus, and

wherein the receiving apparatus comprises;

outputting means for outputting the received data;

transmitting means for transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving means for receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

29. The transmitting apparatus as set forth in claim 28,

wherein the data is picture data,

wherein the receiving apparatus displays the picture data transmitted from said transmitting means,

wherein the control information contains a considered point of picture data displayed by the receiving apparatus,

wherein said categorizing means categorizes the picture data corresponding to a considered area that contains the considered point of the picture data.

30. The transmitting apparatus as set forth in claim 29,

wherein the picture data is object encoded.

31. A transmitting method for transmitting data to a receiving apparatus, comprising the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in at least another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting the data of which the resolutions have been controlled corresponding to the control information to the receiving apparatus.

32. A receiving method for receiving data transmitted from a transmitting apparatus, comprising the steps of:

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in at least another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

33. A transmitting and receiving method having a process of a transmitting apparatus for transmitting data and a process of a receiving apparatus for receiving the data,

wherein the process of the transmitting apparatus comprises the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

improving resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting the data, of which the resolutions have been controlled corresponding to the control information, to the receiving apparatus, and

wherein the process of the receiving apparatus comprises the steps of;

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in at least another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

34. A record medium for recording a program that causes a computer to perform a transmitting process for transmitting data to a receiving apparatus, the transmitting process comprising the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting the data, of which the resolutions have been controlled corresponding to the control information, to the receiving apparatus.

35. A record medium for recording a program that causes a computer to perform a receiving process for receiving data transmitted from a transmitting apparatus, the receiving process comprising the steps of:

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

36. A record medium for recording a program that causes a computer to perform a transmitting process of a transmitting apparatus for transmitting data and a receiving process of a receiving apparatus for receiving the data,

wherein the transmitting process of the transmitting apparatus comprises the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of data;

improving resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution of another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

transmitting the data of which the resolutions have been controlled corresponding to the control information to the receiving apparatus, and

wherein the receiving process of the receiving apparatus comprises the steps of;

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in at least another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

37. A signal for containing a program that causes a computer to perform a transmitting process for transmitting data to a receiving apparatus, the transmitting process comprising the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting the data of which the resolutions have been controlled corresponding to the control information to the receiving apparatus.

38. A signal for containing a program that causes a computer to perform a receiving process for receiving data transmitted from a transmitting apparatus, the receiving process comprising the steps of:

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

39. A signal for containing a program that causes a computer to perform a transmitting process of a transmitting apparatus for transmitting data and a receiving process of a receiving apparatus for receiving the data,

wherein the transmitting process of the transmitting apparatus comprises the steps of:

receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting the data, of which the resolutions have been controlled corresponding to the control information, to the receiving apparatus, and

wherein the receiving process of the receiving apparatus comprises the steps of;

outputting the received data;

transmitting control information containing a particular point of the data, which has been output, to the transmitting apparatus that improves the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data and deteriorates the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information; and

receiving the data transmitted from the transmitting apparatus, the resolutions of the data having been controlled corresponding to the control information.

40. A mobile transmitting apparatus for transmitting data to a receiving apparatus, comprising:

receiving means for receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

controlling means for improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting means for transmitting the data of which the resolutions have been controlled corresponding to the control information in the receiving apparatus.

41. A mobile transmitting apparatus for transmitting data to a receiving apparatus, comprising:

receiving means for receiving control information transmitted from the receiving apparatus, the control information containing a particular point of the data;

controlling means for improving the resolution in at least one of the temporal direction, the spatial direction, and the level direction of a particular area containing the particular point of the data transmitted to the receiving apparatus and deteriorating the resolution in another one or more of the temporal direction, the spatial direction, and the level direction corresponding to the control information containing the particular point of the data; and

transmitting means for prioritizingly transmitting the data of the particular area of which the resolutions have been controlled corresponding to the control information in the receiving apparatus.