Transmitting apparatus, transmitting method, receiving apparatus, receiving method, transmitting and receiving apparatus, transmitting and receiving method, record medium, and signal

• US 7,760,156 B2
• Filed: 10/30/2006
• Issued: 07/20/2010
• Est. Priority Date: 08/09/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kobayashi, Naoki, Kanemaru, Masanori, Kondo, Tetsujiro
• Primary Examiner(s): NGUYEN, KIMNHUNG T
• Application Number:
  US11/590,554
• Publication Number:
  US 20070103387A1
• Time in Patent Office: 1,359 Days
• Field of Search: 345/1.2, 345 21- 23, 455/426.1, 455/426.2, 455/561, 455/562.1, 455/73, 455/82, 455/83, 455/88, 455/546.6
• US Class Current: 345/1.2
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 : for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6131 : involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/835 : Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

On the transmission side, a background picture and objects #1 to #3 are transmitted at a transmission rate R/4 each. On the reception side, a picture composed of the objects #1 to #3 and the background picture is displayed with a particular spatial resolution and a particular temporal resolution. Thus, on the reception side, when the object #1 is dragged at particular time t1, on the transmission side, the transmission of the background picture and the objects #2 and #3 is stopped. Only the object #1 is transmitted at the transmission rate R of the transmission path. Therefore, a picture of which the spatial resolution of the object #1 dragged is improved is displayed at the sacrifice of the temporal resolution of the picture.

Claims

1. A transmitting apparatus for transmitting data to a receiving apparatus, comprising:

receiving means for receiving control information transmitted from the receiving apparatus;

controlling means for controlling the resolutions in at least two directions of a temporal direction, a spatial direction, and a level direction of the data transmitted to the receiving apparatus corresponding to the control information;

transmitting means for transmitting the data of which the resolutions in at least two directions have been controlled corresponding to the control information to the receiving apparatus;

said receiving apparatus outputting the data transmitted from said transmitting means;

analyzing means for analyzing the preferences of the user of the receiving apparatus,

wherein said controlling means controls the resolutions of the data corresponding to the analyzed result of said analyzing means; and

wherein said controlling means contains a considered point of the data that is output to the receiving apparatus, and

wherein said analyzing means analyzes the preferences of the user corresponding to the considered point;

said analyzing means including categorizing means for categorizing the data corresponding to a considered area that contains said considered point;

wherein said analyzing means analyzes the preference of the user corresponding to the analyzed result of said categorizing means; and

wherein said data is picture data,

and said analyzing means further includes;

still area and moving area determining means for determining whether the considered area of the picture data is a still area or a moving area; and

continuity determining means for determining whether the considered point is continuous in the temporal and spatial directions of the considered point, and

wherein said categorizing means categorizes the picture data corresponding to the determined results of the still area and moving area determining means and said continuity means.

2. The transmitting apparatus as set forth in claim 1, further comprising:

considered point storing means for storing a considered point that is contained in the considered area that is still and that is continuous in the temporal and spatial directions and a considered point that is contained in the considered area that is moving and that is continuous in the temporal and spatial directions; and

category identifier adding means for obtaining a category identifier added to a considered point stored to the said considered point storing means and adding the category identifier to the considered point.

3. The transmitting apparatus as set forth in claim 2, wherein in the case that the current considered point is in the considered area that is still and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing means is contained in the considered area that is still and that is continuous in the temporal and spatial directions, said category identifier adding means obtains a category identifier added to the current considered point corresponding to the relation of the spatial positions between the current considered point and the area that contains the immediately preceding considered point.

4. The transmitting apparatus as set forth in claim 3, wherein in the case that the current considered point is in the considered area that is moving and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing means is contained in the considered area that is moving and the is continuous in the temporal and spatial directions, said category identifier adding means obtains a category identifier added to the current considered point corresponding to the similarity of a predetermined feature amount of the considered area that contains the current considered point and that of the considered area that contains the immediately preceding considered point.

5. The transmitting apparatus as set forth in claim 2, wherein said categorizing means categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points stored in said considered point storing means.

6. The transmitting apparatus as set forth in claim 5, wherein said categorizing means categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points assigned the same category identifier, stored in said considered point storing means, and contained in the considered area that is still.

7. The transmitting apparatus as set forth in claim 5, wherein said categorizing means categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points stored in said considered point storing means, contained in the considered area that is moving, assigned the same category identifier, and motion compensated.

8. The transmitting apparatus as set forth in claim 5, wherein said controlling means improves the resolutions of the area categorized as the object.

9. The transmitting apparatus as set forth in claim 1, wherein said continuity determining means determines whether or not the current considered point is continuous in the temporal and spatial directions corresponding to the time difference between the current considered point and a past considered point.

10. The transmitting apparatus as set forth in claim 1, wherein said still area and moving area determining means determines whether a considered area that contains the current considered point is still or moving corresponding to the difference between the considered area that contains the considered point of the current frame and the considered area that contains the considered point of a past frame.

11. The transmitting apparatus as set forth in claim 1, wherein said continuity determining means determines whether or not the current considered point is continuous corresponding to the distances in the temporal direction and the spatial direction between the current considered point and a past considered point at which the same still are and moving are determined result as the considered area that contains the current considered point is obtained.

12. The transmitting apparatus as set forth in claim 11, wherein said categorizing means categorizes the picture data corresponding to weighted values for the distances in the temporal direction and the spatial direction.

13. The transmitting apparatus as set forth in claim 12, further comprising:

picture data storing means for storing picture data in the considered area that contains a considered point that is continuous in the temporal direction and the spatial direction.

14. The transmitting apparatus as set forth in claim 13, wherein when the current considered point is not continuous in the temporal direction and the spatial direction, after the content of said picture data storing means is read, the content is erased and the picture data is the considered area that contains the current considered point is stored to said picture data storing means.

15. The transmitting apparatus as set forth in claim 14, wherein said controlling means improves the resolutions of the picture data that is read from said picture data storing means.

16. The transmitting apparatus as set forth in claim 1, wherein the control information is used for a charging process.

17. A transmitting apparatus for transmitting picture data to a receiving apparatus comprising:

receiving means for receiving control information transmitted from the receiving apparatus;

categorizing means for categorizing the picture data corresponding to the control information;

transmitting means for transmitting for display at the receiving apparatus picture data corresponding to the categorized result of the picture data;

wherein the control information contains a considered point of the picture data displayed by the receiving apparatus,

still area and moving area determining means for determining whether or not the considered area of the picture data is still or moving; and

continuity determining means for determining whether or not the considered point is continuous in a temporal direction and a spatial direction,

wherein said categorizing means categorizes the picture data corresponding to a considered area that contains the considered point of the picture data, and

wherein said categorizing means categorizes the picture data corresponding to the determined results of said still area and moving area determining means and said continuity determining means.

18. The transmitting apparatus as set forth in claim 17, further comprising:

considered point storing means for storing a considered point that is contained in the considered area that is still and that is continuous in the temporal direction and the spatial direction and a considered point that is contained in the considered area that is moving and that is continuous in the temporal direction and the spatial direction; and

category identifier adding means for obtaining a category identifier added to a considered point stored in said considered point storing means and adding the category identifier to the considered point.

19. The transmitting apparatus as set forth in claim 18, wherein in the case that the current considered point is in the considered area that is still and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing means is contained in the considered area that is still and that is continuous in the temporal and spatial directions, said category identifier adding means obtains a category identifier added to the current considered point corresponding to the relation of the spatial positions between the current considered point and the area that contains the immediately preceding considered point.

20. The transmitting apparatus as set forth in claim 18, wherein in the case that the current considered point is in the considered area that is moving and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing means is contained in the considered area that is moving and that is continuous in the temporal and spatial directions, said category identifier adding means obtains a category identifier added to the current considered point corresponding to the similarity of a predetermined feature amount of the considered area that contains the current considered point and that of the considered area that contains the immediately preceding considered point.

21. The transmitting apparatus as set forth in claim 18, wherein said categorizing means categorizes a predetermined area of the picture data as one object corresponding to the density of considered points stored in said considered point storing means.

22. The transmitting apparatus as set forth in claim 21, wherein said categorizing means categorizes a predetermined area of the picture data as one object corresponding to the density of considered points assigned the same category identifier, stored in said considered point storing means, and contained in the considered area that is still.

23. The transmitting apparatus as set forth in claim 21, wherein said categorizing means categorizes a predetermined area of the picture data as one object corresponding to the density of considered points stored in said considered point storing means, contained in the considered area that is moving, assigned the same category identifier, and motion compensated.

24. The transmitting apparatus as set forth in claim 21, wherein said controlling means improves the resolutions of the area categorized as the object.

25. The transmitting apparatus as set forth in claim 17, wherein said still area and moving area determining means determines whether a considered area that contains the current considered point is still or moving corresponding to the difference between the considered area that contains the considered point of the current frame and the considered area that contains the considered point of a past frame.

26. The transmitting apparatus as set forth in claim 17, wherein said continuity determining means determines whether or not the current considered point is continuous in the temporal and spatial directions corresponding to the time difference between the current considered point and a past considered point.

27. The transmitting apparatus as set forth in claim 17, wherein said continuity determining means determines whether or not the current considered point is continuous corresponding to the distances in the temporal direction and the spatial direction between the current considered point and a past considered point at which the same still area moving area determined result as the considered area that contains the current considered point is obtained.

28. The transmitting apparatus as set forth in claim 27, wherein said categorizing means categorizes the picture data corresponding to weighted values for the distances in the temporal direction and the spatial direction.

29. The transmitting apparatus as set forth in claim 27, further comprising:

picture data storing means for storing picture data in the considered area that contains a considered point that is continuous in the temporal direction and the spatial direction.

30. The transmitting apparatus as set forth in claim 29, wherein when the current considered point is not continuous in the temporal direction and the spatial direction, after the content of said picture data storing means is read, the content is erased and the picture data in the considered area that contains the current considered point is stored to said picture data storing means.

31. The transmitting apparatus as set forth in claim 30, wherein said controlling means improves the resolutions of the picture data that is read from said picture data storing means.

32. The transmitting apparatus as set forth in claim 17, wherein the control information is used for a charging process.

33. A transmitting apparatus for transmitting data to a receiving apparatus, comprising:

a receiving unit configured to receive control information transmitted from the receiving apparatus;

a controlling unit configured to control the resolutions in at least two directions of a temporal direction, a spatial direction, and a level direction of the data transmitted to the receiving apparatus corresponding to the control information;

a transmitting unit configured to transmit the data of which the resolutions in at least two directions have been controlled corresponding to the control information to the receiving apparatus;

said receiving apparatus outputting the data transmitted from said transmitting unit;

an analyzing unit configured to analyze the preferences of the user of the receiving apparatus,

wherein said controlling unit controls the resolutions of the data corresponding to the analyzed result of said analyzing unit; and

wherein said controlling unit contains a considered point of the data that is output to the receiving apparatus, and

wherein said analyzing unit analyzes the preferences of the user corresponding to the considered point;

said analyzing unit including categorizing unit configured to categorize the data corresponding to a considered area that contains said considered point;

wherein said analyzing unit analyzes the preference of the user corresponding to the analyzed result of said categorizing unit; and

wherein said data is picture data,

and said analyzing unit further includes;

a still area and moving area determining unit configured to determine whether the considered area of the picture data is a still area or a moving area; and

a continuity determining unit configured to determine whether the considered point is continuous in the temporal and spatial directions of the considered point, and

wherein said categorizing unit categorizes the picture data corresponding to the determined results of the still area and moving area determining unit and said continuity unit.

34. The transmitting apparatus as set forth in claim 33, further comprising:

a considered point storing unit configured to store a considered point that is contained in the considered area that is still and that is continuous in the temporal and spatial directions and a considered point that is contained in the considered area that is moving and that is continuous in the temporal and spatial directions; and

a category identifier adding unit configured to obtain a category identifier added to a considered point stored to the said considered point storing unit and adding the category identifier to the considered point.

35. The transmitting apparatus as set forth in claim 34, wherein in the case that the current considered point is in the considered area that is still and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing unit is contained in the considered area that is still and that is continuous in the temporal and spatial directions, said category identifier adding unit obtains a category identifier added to the current considered point corresponding to the relation of the spatial positions between the current considered point and the area that contains the immediately preceding considered point.

36. The transmitting apparatus as set forth in claim 35, wherein in the case that the current considered point is in the considered area that is moving and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing unit is contained in the considered area that is moving and the is continuous in the temporal and spatial directions, said category identifier adding unit obtains a category identifier added to the current considered point corresponding to the similarity of a predetermined feature amount of the considered area that contains the current considered point and that of the considered area that contains the immediately preceding considered point.

37. The transmitting apparatus as set forth in claim 34, wherein said categorizing unit categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points stored in said considered point storing unit.

38. The transmitting apparatus as set forth in claim 37, wherein said categorizing unit categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points assigned the same category identifier, stored in said considered point storing unit, and contained in the considered area that is still.

39. The transmitting apparatus as set forth in claim 37, wherein said categorizing unit categorizes a predetermined area of the picture data as an object corresponding to the preference of the user corresponding to the density of considered points stored in said considered point storing unit, contained in the considered area that is moving, assigned the same category identifier, and motion compensated.

40. The transmitting apparatus as set forth in claim 37, wherein said controlling unit improves the resolutions of the area categorized as the object.

41. The transmitting apparatus as set forth in claim 33, wherein said still area and moving area determining unit determines whether a considered area that contains the current considered point is still or moving corresponding to the difference between the considered area that contains the considered point of the current frame and the considered area that contains the considered point of a past frame.

42. The transmitting apparatus as set forth in claim 33, wherein said continuity determining unit determines whether or not the current considered point is continuous in the temporal and spatial directions corresponding to the time difference between the current considered point and a past considered point.

43. The transmitting apparatus as set forth in claim 33, wherein said continuity determining unit determines whether or not the current considered point is continuous corresponding to the distances in the temporal direction and the spatial direction between the current considered point and a past considered point at which the same still are and moving are determined result as the considered area that contains the current considered point is obtained.

44. The transmitting apparatus as set forth in claim 43, wherein said categorizing unit categorizes the picture data corresponding to weighted values for the distances in the temporal direction and the spatial direction.

45. The transmitting apparatus as set forth in claim 44, further comprising:

a picture data storing unit configured to store picture data in the considered area that contains a considered point that is continuous in the temporal direction and the spatial direction.

46. The transmitting apparatus as set forth in claim 45, wherein when the current considered point is not continuous in the temporal direction and the spatial direction, after the content of said picture data storing unit is read, the content is erased and the picture data is the considered area that contains the current considered point is stored to said picture data storing unit.

47. The transmitting apparatus as set forth in claim 46, wherein said controlling unit improves the resolutions of the picture data that is read from said picture data storing unit.

48. The transmitting apparatus as set forth in claim 33, wherein the control information is used for a charging process.

49. A transmitting apparatus for transmitting picture data to a receiving apparatus comprising:

a receiving unit configured to receive control information transmitted from the receiving apparatus;

a categorizing unit configured to categorize the picture data corresponding to the control information;

a transmitting unit configured to transmit for display at the receiving apparatus picture data corresponding to the categorized result of the picture data;

wherein the control information contains a considered point of the picture data displayed by the receiving apparatus,

a still area and moving area determining unit configured to determine whether or not the considered area of the picture data is still or moving; and

a continuity determining unit configured to determine whether or not the considered point is continuous in a temporal direction and a spatial direction,

wherein said categorizing unit categorizes the picture data corresponding to a considered area that contains the considered point of the picture data, and

wherein said categorizing unit categorizes the picture data corresponding to the determined results of said still area and moving area determining unit and said continuity determining unit.

50. The transmitting apparatus as set forth in claim 49, further comprising:

a considered point storing unit configured to store a considered point that is contained in the considered area that is still and that is continuous in the temporal direction and the spatial direction and a considered point that is contained in the considered area that is moving and that is continuous in the temporal direction and the spatial direction; and

a category identifier adding unit configured to obtain a category identifier added to a considered point stored in said considered point storing unit and adding the category identifier to the considered point.

51. The transmitting apparatus as set forth in claim 50, wherein in the case that the current considered point is in the considered area that is still and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing unit is contained in the considered area that is still and that is continuous in the temporal and spatial directions, said category identifier adding unit obtains a category identifier added to the current considered point corresponding to the relation of the spatial positions between the current considered point and the area that contains the immediately preceding considered point.

52. The transmitting apparatus as set forth in claim 50, wherein in the case that the current considered point is in the considered area that is moving and that is continuous in the temporal and spatial directions, when an immediately preceding considered point stored in said considered point storing unit is contained in the considered area that is moving and that is continuous in the temporal and spatial directions, said category identifier adding unit obtains a category identifier added to the current considered point corresponding to the similarity of a predetermined feature amount of the considered area that contains the current considered point and that of the considered area that contains the immediately preceding considered point.

53. The transmitting apparatus as set forth in claim 50, wherein said categorizing unit categorizes a predetermined area of the picture data as one object corresponding to the density of considered points stored in said considered point storing unit.

54. The transmitting apparatus as set forth in claim 53, wherein said categorizing unit categorizes a predetermined area of the picture data as one object corresponding to the density of considered points assigned the same category identifier, stored in said considered point storing unit, and contained in the considered area that is still.

55. The transmitting apparatus as set forth in claim 53, wherein said categorizing unit categorizes a predetermined area of the picture data as one object corresponding to the density of considered points stored in said considered point storing unit, contained in the considered area that is moving, assigned the same category identifier, and motion compensated.

56. The transmitting apparatus as set forth in claim 53, wherein said controlling unit improves the resolutions of the area categorized as the object.

57. The transmitting apparatus as set forth in claim 49, wherein said still area and moving area determining unit determines whether a considered area that contains the current considered point is still or moving corresponding to the difference between the considered area that contains the considered point of the current frame and the considered area that contains the considered point of a past frame.

58. The transmitting apparatus as set forth in claim 49, wherein said continuity determining unit determines whether or not the current considered point is continuous in the temporal and spatial directions corresponding to the time difference between the current considered point and a past considered point.

59. The transmitting apparatus as set forth in claim 49, wherein said continuity determining unit determines whether or not the current considered point is continuous corresponding to the distances in the temporal direction and the spatial direction between the current considered point and a past considered point at which the same still area moving area determined result as the considered area that contains the current considered point is obtained.

60. The transmitting apparatus as set forth in claim 59, wherein said categorizing unit categorizes the picture data corresponding to weighted values for the distances in the temporal direction and the spatial direction.

61. The transmitting apparatus as set forth in claim 59, further comprising:

a picture data storing unit configured to store picture data in the considered area that contains a considered point that is continuous in the temporal direction and the spatial direction.

62. The transmitting apparatus as set forth in claim 61, wherein when the current considered point is not continuous in the temporal direction and the spatial direction, after the content of said picture data storing unit is read, the content is erased and the picture data in the considered area that contains the current considered point is stored to said picture data storing unit.

63. The transmitting apparatus as set forth in claim 62, wherein said controlling unit improves the resolutions of the picture data that is read from said picture data storing unit.

64. The transmitting apparatus as set forth in claim 49, wherein the control information is used for a charging process.