SCREEN DATA TRANSMITTING SYSTEM, SCREEN DATA TRANSMITTING SERVER, SCREEN DATA TRANSMITTING METHOD AND PROGRAM RECORDING MEDIUM

• US 20100322523A1
• Filed: 06/27/2008
• Published: 12/23/2010
• Est. Priority Date: 06/29/2007
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Senda, Shuji, Mitsuhashi, Akitake
• Granted Patent: US 8,224,099 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/507 using conditional replenishment
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

A difference region extracting section 122 extracts differential regions based on screen data of a previous screen stored in transmission screen data storage section 124 and a current screen stored in a screen data storage section 121. A priority determining section 123 calculates change data density indicating a degree of a change of pixel values for every differential region. For example, when a user operates an icon on the screen, the change data density of the differential region corresponding to the icon becomes larger than the change data density of the differential region corresponding to the background portion of a video image. An encoding section 125 writes screen code data of each differential region in a transmission buffer 141 in descending order of the change data density.

Claims

1-30. -30. (canceled)

31. A screen data transmitting system comprising:

a screen data transmission server; and

a screen data reception client configured to generate a screen data based on image data received from said screen data transmission server and display the generated screen data on a client screen,

wherein said screen data transmission server comprises;

a differential region extracting section configured to compare a current screen currently displayed and a previous screen previous to the current screen, and to extract as differential regions, regions on the current screen having changed from the previous screen;

a priority determining section configured to determine a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region for every differential region; and

a communication section configured to transmit the image data of the differential regions in descending order of the priority.

32. The screen data transmitting system according to claim 31, wherein said priority determining section calculates a change data density indicating a degree of a change of the pixel values to the previous screen based on the pixel values and the area, and determines the priorities of the differential regions based on the change data densities.

33. The screen data transmitting system according to claim 32, wherein the change data density is calculated by dividing a total of change amounts of the pixel values by a square of the area.

34. The screen data transmitting system according to claim 32, wherein said screen data transmission server further comprises:

a transmission history storage section configured to store a transmission history indicating for each of blocks obtained by dividing the previous or current screen that the image data in which at least a part of each of the differential regions is contained in the block have been transmitted; and

a weight generating section configured to calculate a weight to the priority based on the transmission history for the blocks covered by the differential regions, and

wherein said priority determining section weights the change data density with the weight to determine the priority.

35. The screen data transmitting system according to claim 34, wherein the transmission history comprises the latest transmission time indicating a time at which the image data for the differential region whose part is contained in the blocks has been transmitted last, and

said priority determining section sets the weight such that the weight takes a larger value when a difference between the latest transmission time and the current time is larger.

36. The screen data transmitting system according to claim 34, wherein the transmission history contains a transmission frequency indicates the number of times of transmission of the image data contained in the blocks, and

said priority determining section sets the weight such that the weight takes a larger value when the transmission frequency of the blocks covered by the differential region is smaller.

37. The screen data transmitting system according to claim 32, wherein said screen data reception client comprises:

an operation data input section configured to transmit an operation data indicating an operation performed by a user, to said screen data transmission server, and

said priority determining section calculates the weight based on the operation data transmitted from said screen data reception client, and weights the change data density by the weight to determine the priority.

38. The screen data transmitting system according to claim 37, wherein the operation data comprises a position of a cursor displayed on said client screen, and

said priority determining section determines the weight such that the weight takes a larger value when a distance between a position of the differential region and the position of said cursor shown by the operation data is smaller.

39. The screen data transmitting system according to claim 37, wherein the operation data contains a movement speed of the cursor displayed on the client screen, and

said priority determining section determines the priority without weighting by the weight when the movement speed is lower than a threshold value, and determines the priority by weighting the change data density by the weight such that the weight takes a larger value as the area is smaller, when the movement speed is equal to or higher than the threshold value.

40. The screen data transmitting system according to claim 37, wherein the operation data includes an operation frequency of a key operated by the user, and

said priority determining section determines the priority without weighting by the weight when the operation frequency is lower than a threshold value, and determines the priority by weighting the change data density by the weight such that the weight takes a larger value as the area is smaller, when the operation frequency is equal to or higher than the threshold value.

41. The screen data transmitting system according to claim 31, wherein said screen data transmission server further comprises a transmission rate calculating section configured to calculate a transmission rate when said communication section communicates with said screen data reception client,

said communication section comprises a transmission buffer configured to temporarily hold the image data to be transmitted to said screen data reception client, and

a maximum data size of the image data to be written in said transmission buffer is determined based on the transmission rate, and the image data for the differential regions are written in said transmission buffer in the maximum data size or below in descending order of the priority.

42. The screen data transmitting system according to claim 31, wherein said differential region extracting section detects the pixel whose value has changed from the pixel value of the previous screen more than a pixel threshold value, and extracts as the differential region, a region covered by a set of the detected pixels, every two of which are connected by a line segment and a distance between every two of which is equal to or shorter than a distance threshold value.

43. The screen data transmitting system according to claim 31, wherein said differential region extracting section extracts said differential region based on weighted pixel values obtained by weighting color components by different weights.

44. The screen data transmitting system according to claim 32, wherein said screen data transmission server further comprises:

an encoding section configured to encode pixel data of the differential region to generate a code data; and

a transmission code generating section configured to divide the code data of a size smaller equal to or smaller than a predetermined size to generate a division code data, and relate a transmission priority obtained by dividing the change data density by a divisional number of the divisional code data to the division code data, and

said communication section transmits the division code data based on the transmission priority.

45. The screen data transmitting system according to claim 44, wherein the division code data comprises connection order data indicating an order in which the division code data is to be connected, and

said transmission code generating section selects a transmission target code data, the division code data which contains the connection order data indicating the smallest order, transmits the transmission target code data by using said communication section, deletes the transmitted transmission target code data, deletes all of the division code data when the division code data to be first connected in generation of the screen code data by connecting the division code data is contained and the division code data to be connected next to the division code data to be first connected from among the division code data which cannot be transmitted by said communication section, and deletes the screen code data which cannot be transmitted by said communication section.

46. A screen data transmission server which transmits image data for differential regions to a screen data reception client configured to generate a screen data based on the image data and display the generated screen data on a client screen, comprising:

a differential region extracting section configured to compare a current screen currently displayed and a previous screen previous to the current screen and extract as differential regions, regions on the current screen having changed from the previous screen;

a priority determining section configured to determine a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region, for every differential region; and

a communication section configured to transmit the image data for the differential regions in descending order of the priority.

47. The screen data transmission server according to claim 46, wherein said priority determining section calculates a change data density indicating a degree of a change of the pixel values to the previous screen based on the pixel values and the area, and determines the priorities of the differential regions based on the change data densities.

48. The screen data transmission server according to claim 47, further comprising:

a transmission history storage section configured to store a transmission history indicating for each of blocks obtained by dividing the previous or current screen that the image data in which at least a part of the differential regions is contained in the blocks have been transmitted; and

a weight generating section configured to calculate a weight to the priority based on the transmission history for the blocks covered by the differential regions,

wherein said priority determining section weights the change data density with the weight to determine the priority.

49. The screen data transmission server according to claim 47, further comprising:

an operation data acquiring section configured to hold an operation data indicating an operation performed by a user, and received by said screen data reception client,

wherein said priority determining section calculates the weight based on the operation data received by said screen data reception client, and weights the change data density by the weight to determine the priority.

50. The screen data transmission server according to claim 46, further comprising:

a communication rate calculating section configured to calculate a communication rate when said communication section communicates with said screen data reception client,

said communication section comprises a transmission buffer configured to temporarily hold the image data to be transmitted to said screen data reception client, and

wherein a maximum data size of the image data to be written in said transmission buffer is determined based on the communication rate, and the image data for the differential regions are written in said transmission buffer in the maximum data size or below in descending order of the priority.

51. The screen data transmission server according to claim 46, further comprises:

an encoding section configured to encode pixel data of said differential region to generate a code data; and

a transmission code generating section configured to divide the code data of a size smaller equal to or smaller than a predetermined size to generate a division code data, and relate a transmission priority obtained by dividing the change data density by a divisional number of the divisional code data to the division code data, and

wherein said communication section transmits the division code data based on the transmission priority.

52. The screen data transmission server according to claim 51, wherein the division code data comprises connection order data indicating an order in which the division code data is to be connected, and

said transmission code generating section selects a transmission target code data, the division code data which contains the connection order data indicating the smallest order, transmits the transmission target code data by using said communication section, deletes the transmitted transmission target code data, deletes all of the division code data when the division code data to be first connected in generation of the screen code data by connecting the division code data is contained and the division code data to be connected next to the division code data to be first connected from among the division code data which cannot be transmitted by said communication section, and deletes the screen code data which cannot be transmitted by said communication section.

53. A screen data transmitting method comprising:

comparing a current screen currently displayed and a previous screen previous to the current screen and extracting as differential regions, regions on the current screen having changed from the previous screen;

determining a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region, for every differential region;

transmitting image data for the differential regions in descending order of the priority; and

generating a screen data based on the image data received from a screen data transmission server and displaying the generated screen data on a client screen.

54. The screen data transmitting method according to claim 53, wherein said determining comprises:

calculating a change data density indicating a degree of a change of the pixel values to the previous screen based on the pixel values and the area, and determining the priorities of the differential regions based on the change data densities.

55. The screen data transmitting method according to claim 54, further comprising:

managing a transmission history indicating for each of blocks obtained by dividing the previous or current screen that the image data in which at least a part of the differential regions is contained in the blocks have been transmitted,

wherein said determining comprises;

calculating a weight to the change data density based on the transmission history; and

determining the priority of each of the differential regions based on the weighted change data density obtained by weighting the change data density.

56. The screen data transmitting method according to claim 54, wherein said determining comprises:

calculating the weight based on the operation data transmitted from said screen data reception client; and

determining the priority based on the weighted change data density.

57. The screen data transmitting method according to claim 53, further comprising:

temporarily holding the image data in a transmission buffer;

calculating a transmission rate of a server which transmits the image data for the differential regions,

wherein said communicating comprises;

determining a data size of the image data to be written in said transmission buffer based on the transmission rate; and

writing the image data for the differential regions in said transmission buffer in a range of the data size in descending order of the priority.

58. The screen data transmitting method according to claim 53, further comprising:

generating code data by encoding pixel data of the differential region;

generating division code data by dividing the code data of a size smaller equal to or smaller than a predetermined size when the code data size is larger than the predetermined size; and

relating a transmission priority obtained by dividing the change data density by a divisional number of the divisional code data to the division code data,

wherein said communicating comprises;

transmitting the division code data based on the transmission priority.

59. The screen data transmitting method according to claim 58, wherein the to division code data comprises connection order data indicating an order in which the division code data is to be connected, and

said communicating comprises:

transmitting the division code data based on code the connection order data.

60. A computer-readable recording medium in which a computer-executable program code is stored to make a computer to execute a screen data transmitting method which comprises:

comparing a current screen currently displayed and a previous screen previous to the current screen and extracting as differential regions, regions on the current screen having changed from the previous screen;

determining a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region, for every differential region;

transmitting image data for the differential regions in descending order of the priority; and

generating a screen data based on the image data received from a screen data transmission server and displaying the generated screen data on a client screen.

61. A screen data reception client comprising:

a section configured to receive image data for differential regions;

a generating section configured to generate a screen data based on the received image data for the differential regions; and

a client screen configured to display the generated screen data,

wherein a current screen currently displayed and a previous screen previous to the current screen are compared, regions on the current screen having changed from the previous screen are extracted as the differential regions, a priority is determined based on pixel values of pixels contained in each of the differential regions and an area of the differential region for every differential region, and the image data for the differential regions are transmitted in descending order of the priority.

62. A screen data transmitting method comprising:

comparing a current screen currently displayed and a previous screen previous to the current screen and of extracting as differential regions, regions on the current screen having changed from the previous screen;

determining a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region, for every differential region; and

transmitting the image data for the differential regions in descending order of the priority.

63. A computer-readable recording medium in which a computer-executable program code is stored to make a computer to executed a screen data transmitting method which comprises:

comparing a current screen currently displayed and a previous screen previous to the current screen;

extracting as differential regions, regions on the current screen having changed from the previous screen;

determining a priority based on pixel values of pixels contained in each of the differential regions and an area of the differential region, for every differential region; and

transmitting the image data for the differential regions in descending order of the priority.

64. A screen data receiving method comprising:

receiving image data for differential regions; and

generating a screen data based on the received image data for the differential regions and displaying the generated screen data,

wherein a current screen currently displayed and a previous screen previous to the current screen are compared, regions on the current screen having changed from the previous screen are extracted as the differential regions, a priority is determined based on pixel values of pixels contained in each of the differential regions and an area of the differential region for every differential region, and the image data for the differential regions are transmitted in descending order of the priority.

65. A computer-readable recording medium in which a computer-executable program code is stored to make a computer to execute a screen data receiving method which comprises:

receiving image data for differential regions; and

generating a screen data based on the received image data for the differential regions and displaying the generated screen data,

wherein a current screen currently displayed and a previous screen previous to the current screen are compared, regions on the current screen having changed from the previous screen are extracted as the differential regions, a priority is determined based on pixel values of pixels contained in each of the differential regions and an area of the differential region for every differential region, and the image data for the differential regions are transmitted in descending order of the priority.