Graphical-interactive-screen display apparatus and peripheral units

• US 20020047945A1
• Filed: 10/30/2001
• Published: 04/25/2002
• Est. Priority Date: 10/20/1993
• Status: Abandoned Application

• Current Assignee: Hidekazu Tanigawa, Seiji Tsuji, Yoshio Nakano
• Original Assignee: Hidekazu Tanigawa, Seiji Tsuji, Yoshio Nakano
• Inventors: Nakano, Yoshio, Tsuji, Seiji, Tanigawa, Hidekazu
• Application Number:
  US10/016,434
• Publication Number:
  US 20020047945A1
• Time in Patent Office: Days
• US Class Current: 348/734
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42209 for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/42226 Reprogrammable remote control devices
• CPC Class Codes: H04N 21/42228 the reprogrammable keys being displayed on a display screen in order to reduce the number of keys on the remote control device itself
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4438 Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/6547 comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A program receiver for displaying a graphical interactive picture by receiving a program transmitted from a program transmitter, the program receiver comprising: a storage unit for storing a plurality of basic picture elements in advance, the plurality of basic picture elements being figures composing the graphical interactive picture manipulated by a user; a signal receipt unit for receiving a signal transmitted from the program transmitter, the signal being a multiplex signal including a program and data specifying a structure of the graphical interactive picture; a signal separation unit for separating the signal received by the signal receipt unit into a program signal and a graphical-interactive-picture-structure specification data signal; a first graphical interactive picture generation unit for generating the graphical interactive picture by combining the basic picture elements stored in the storage unit based on the graphical-interactive-picture-structure specification data signal from the signal separation unit; and a display unit for displaying the graphical interactive picture generated by the first graphical interactive picture generation unit.

Claims

1. A remote controller comprising structure information supply means for holding structure information for various types of appliances in advance, and portable remote control means for controlling said appliances,

(1) said structure information supply means including:

a transmitter'"'"'s structure information storage unit for storing the structure information, the structure information including control information for each appliance and object information, said control information being used by said remote control means in controlling said appliances, said object information representing manipulation objects forming a manipulation unit of said remote control means; and

a structure information transmission unit for retrieving the control information and the object information from said transmitter'"'"'s structure information storage unit to send the retrieved information to said remote control means,

(2) said remote control means including;

a structure information receipt unit for receiving the control information and the object information from said structure information transmission unit;

a receiver'"'"'s structure information storage unit for storing the control information and the object information received by said structure information receipt unit,

a display unit for displaying a picture having a manipulation area used for manipulating said appliances; and

a first display control unit for controlling said display unit to place the manipulation objects on the manipulation area based on the object information stored in said receiver'"'"'s structure information storage unit.

2. The remote controller of claim 1, wherein said remote control means further includes:

a manipulation detection unit for detecting which manipulation object is manipulated on the manipulation area;

a first control signal transmission unit for transmitting a control signal assigned for the manipulation object detected by said manipulation detection unit to said appliances based on the control information stored in said receiver'"'"'s structure information storage unit,

whereby said appliances operate in accordance with the control signal from said remote control means.

3. The remote controller of claim 2, wherein said structure information supply means is installed in each appliance controlled by the control signal from said remote control means.

4. The remote controller of claim 3, wherein

said structure information supply means further includes:

a structure information generation unit for generating state-based control information depending on an action state of each appliance, said state-based control information being transmitted to said remote control means by said structure information transmission unit;

an object information generation unit for generating the object information depending on the action state of each appliance, the object information being transmitted to said remote control means by said structure information transmission unit, and wherein said remote control means further includes;

a generated information receipt unit for receiving the state-based control information from said structure information transmission unit;

a generated information update unit for updating the control information and the object information in said receiver'"'"'s structure information storage unit in accordance with the state-based control information received by said generated information receipt unit;

a second control signal transmission unit for transmitting a control signal assigned to the manipulated object detected by said manipulation detection unit to said appliances based on updated control information stored in said receiver'"'"'s structure information storage unit;

a generated object information receipt unit for receiving updated object information from said structure information transmission unit; and

a second display control unit for controlling said display unit to place updated manipulation objects on the manipulation area based on the updated object information received by said generated object information receipt unit.

5. The remote controller of claim 2, wherein said remote control means further includes a structure information request unit for sending a request to said structure information supply means to transmit the structure information and the object information for said appliances, and

wherein said structure information supply means further includes a structure information request direction unit for directing said structure information transmission unit to transmit the control information and the object information for said appliances upon receipt of the request from said structure information request unit, and

wherein said transmitter'"'"'s structure information storage unit stores the control information in relation with the object information for each appliance.

6. A remote controller comprising structure information supply means for holding structure information for various types of appliances in advance, first remote control means for controlling said appliances, and second portable remote control means for controlling said appliances,

(1) said structure information supply means including:

a transmitter'"'"'s structure information storage unit for storing the structure information, the structure information including control information for each appliance and object information, said control information being used by said second remote control means in controlling said appliances, said object information representing manipulation objects forming a manipulation unit of said second remote control means; and

a structure information transmission unit for retrieving the control information and the object information from said transmitter'"'"'s structure information storage unit to send the retrieved information to said second remote control means,

(2) said first remote control means including;

a manipulation acceptance unit for accepting a manipulation direction; and

a signal transmission unit for transmitting a control signal assigned to the manipulation direction accepted by said manipulation acceptance unit, whereby said appliances operate in accordance with the control signal from said first remote control means,

(3) said second remote control means including;

a structure information receipt unit for receiving the control information and the object information from said structure information transmission unit;

a receiver s structure information storage unit for storing the control information and the object information received by said structure information receipt unit,

a display unit for displaying a picture having a manipulation area used for manipulating said appliances;

a first display control unit for controlling said display unit to place the manipulation objects on the manipulation area based on the object information stored in said receiver'"'"'s structure information storage unit;

a manipulation detection unit for detecting which manipulation object is manipulated on the manipulation area;

a first control signal transmission unit for transmitting a control signal assigned for the manipulation object detected by said manipulation detection unit to said appliances based on the control information stored in said receiver'"'"'s structure information storage unit,

whereby said appliances operate in accordance with the control signal from said second remote control means.

7. A synthesis apparatus including a plurality of appliances for synthesizing menu information withheld by each appliance, said menu information being hierarchical information enabling a user to control said appliances interactively,

(1) one of said plurality of appliances including:

first menu storage means for storing first menu information, said first menu information being hierarchical information to be used to control said one appliance;

menu information receipt means for receiving the menu information from the other appliances;

menu information synthesis means for synthesizing the menu information received by said menu information receipt means and first menu information stored in said first menu information storage unit, resulting synthesized menu information being used as a menu-information selection menu at a highest position in a hierarchy; and

menu display means for displaying the menu-information selection menu generated by said menu information synthesis means,

(2) each of the other appliances including;

menu storage means for storing the menu information unique to each appliance, the menu information being hierarchical information to be used to control each appliance; and

menu transmission means for transmitting the unique menu information in said menu storage means to said one appliance.

8. The synthesis apparatus of claim 7, wherein said one appliance further includes synthesis rule storage means for storing rules regulating menu-information synthesis by said menu information synthesis means,

whereby said menu information synthesis means synthesizes the menu information from the other appliances and the first menu information in accordance with the synthesis rules.

9. The synthesis apparatus of claim 8, wherein said menu transmission means includes:

a menu information retrieval direction unit for detecting a connection with said one appliance and for giving a direction to retrieve the menu information;

a menu information retrieval unit for retrieving the menu information from said menu storage means upon receipt of the direction from said menu information retrieval direction unit; and

a menu transmission unit for transmitting the menu information retrieved by said menu information retrieval unit to said menu information receipt means.

10. The synthesis apparatus of claim 9, wherein said menu information synthesis means includes:

a menu selection acceptance unit for accepting a selection of an appliance subject to control manipulation using the menu-information selection menu displayed by said menu display means;

a menu information retrieval unit for retrieving the menu information of the appliance accepted by said menu selection acceptance unit from said menu information synthesis means; and

a menu switch control unit for changing a display on said menu display means in accordance with the menu information retrieved by said menu information retrieval unit.

11. The synthesis apparatus of claim 10, wherein said menu information synthesis means further includes:

a menu information temporarily storage unit for temporarily storing the menu information received by said menu information receipt means;

a first menu information retrieval unit for retrieving the first menu information from said first menu storage means;

a synthesis rule retrieval unit for retrieving the synthesis rules from said synthesis rule storage means, said synthesis rules including a synthesis rule for a hierarchial structure, a selection rule for a displayed shape, a synthesis rule for a panel object, and a synthesis rule for control of said appliances; and

a hierarchial structure synthesis unit for synthesizing the menu information stored in said menu information temporarily storage unit and the first menu information retrieved by said first menu information retrieval unit in accordance with the hierarchical synthesis rule retrieved by said synthesis rule retrieval unit.

12. The synthesis apparatus of claim 9, wherein said menu information synthesis means further includes:

a menu information temporarily storage unit for temporarily storing the menu information received by said menu information receipt means;

a first menu information retrieval unit for retrieving the first menu information from said first menu storage means;

a synthesis rule retrieval unit for retrieving the synthesis rules from said synthesis rule storage means, said synthesis rules including a synthesis rule for a hierarchial structure, a selection rule for a displayed shape, a synthesis rule for a panel object, and a synthesis rule for control of said appliances; and

a hierarchial structure synthesis unit for synthesizing the menu information stored in said menu information temporarily storage unit and the first menu information retrieved by said first menu information retrieval unit in accordance with the hierarchical synthesis rule retrieved by said synthesis rule retrieval unit.

14. The remote control system of claim 13, wherein said menu information synthesis means includes:

a menu information storage unit for storing the menu information of each appliance received by said menu receipt means, the menu information being hierarchical information including a panel object name and shape information;

a synthesis rule storage unit for storing synthesis rules regulating menu-information synthesis, said synthesis rules including a synthesis rule for a hierarchial structure, a synthesis rule for a panel object, a selection rule for a displayed shape, and a synthesis rule for control of said appliances;

a menu synthesis unit for synthesizing the menu information of said appliances stored in said menu information storage unit into one hierarchical structure in accordance with the hierarchical synthesis rules in said synthesis rule storage unit;

a panel object synthesis unit for synthesizing the menu information of said appliances in said menu information storage unit into one panel object in accordance with the synthesis rule for the panel object stored in said synthesis rule storage unit, the synthesis rule for the panel object being to synthesize the menu information identified by an identical panel object name;

a shape synthesis unit for making the shape information uniform for each appliance for the menu information in said menu information storage unit in accordance with the selection rule for the displayed shape in said synthesis rule storage unit, said selection rule being to select the shape information of a specified appliance and apply the same to the shape information for each appliance for the menu information in said menu information storage unit; and

an action append unit for inserting data as to an control action in accordance with the synthesis rule for the control of said appliances, the synthesis rule for the control being to insert the data as to the control action described by the synthesis rule for the control at a position specified by the synthesis rule for the control.

15. The remote control system of claim 14, wherein said menu display means includes:

a bit map data conversion unit for converting the menu information synthesized by said menu synthesis unit, panel synthesis unit, shape synthesis unit, and action append unit into bit map data for a visual display;

a display unit for displaying the menu information converted into the bit map data by said bit map data conversion unit;

a first manipulation acceptance unit for accepting a user'"'"'s selection of menu displayed by said display unit; and

a selected menu notice unit for notifying a menu selected and accepted by said first manipulation acceptance unit to said menu information synthesis means.

16. The remote control system of claim 15, wherein said menu information synthesis means further includes a menu information retrieval unit for retrieving the menu information from said menu storage unit for the appliance corresponding to the menu notified by said selected menu notice unit, retrieved menu information including information as to manipulation objects displayed by said display unit, and wherein said menu display means further includes:

a menu information conversion unit for converting the menu information retrieved by said menu information retrieval unit into the bit map data, the menu information retrieved by said menu information retrieval unit including information as to a display of manipulation objects displayed by said display unit;

a change-menu display control unit for controlling said display unit to display the menu information converted into the bit map data by said menu information conversion unit;

a second manipulation acceptance unit for accepting a user'"'"'s control manipulation related to a control over the appliance using the manipulation objects displayed by said display unit; and

a manipulation notice unit for notifying the control manipulation accepted by said second manipulation acceptance unit to said control sinal transmission unit.

18. The program receiver of claim 17 further comprising:

interactive manipulation means for inputting manipulation to the graphical interactive picture displayed by said display means;

basic action storage means for storing a content of an action for updating the graphical-interactive-picture-structure specification data; and

second graphical interactive picture generation means for retrieving the content of the action from said basic action storage means based on action information directing an update of the graphical interactive picture upon receipt of the input manipulation from said interactive manipulation means to update the graphical-interactive-picture-structure specification data to generate an updated graphical interactive picture.

19. The program receiver of claim 18, wherein said basic picture elements in said storage means are composed of file names identifying each basic picture element and the bit map data for each basic picture element, and

wherein said first graphical interactive picture generation means includes:

a receipt•

decode unit for receiving the graphical-interactive-picture-structure specification data signal from said signal separation means to decode the same;

a storage unit for storing decoded graphical-interactive-picture-structure specification data from said receipt∩

decode unit, the graphical-interactive-picture-structure specification data being composed of a class definition including a plurality pieces of class attribute information, a panel object definition including a plurality pieces of panel object information, an action definition including a plurality pieces of action information, and a shape definition including a plurality pieces of shape information;

a first process unit for retrieving the graphical-interactive-picture-structure specification data from said storage unit, extracting the basic picture elements corresponding to the file names by referring to the shape information from said storage unit in accordance with the retrieved graphical-interactive-picture-structure specification data, and for placing the extracted basic picture elements by referring to the panel object information; and

a first display control unit for controlling the display means to display the basic picture elements placed by said first process unit as the graphical interactive picture.

20. The program receiver of claim 19, wherein said interactive manipulation means includes:

an input manipulation acceptance unit for accepting a user'"'"'s input manipulation to the graphical interactive picture; and

an interactive signal transmission unit for transmitting the input manipulation accepted by said manipulation acceptance unit to said second graphical interactive picture generation unit as an interactive signal, and

wherein said second graphical interactive picture generation means includes;

an interactive signal receipt unit for receiving the interactive signal from said interactive signal transmission unit;

an interactive signal interpretation unit for interpreting the interactive signal received by said interactive signal receipt unit;

a graphical-interactive-picture-structure specification data update unit for retrieving a content of an action from said basic action storage means in accordance with the interactive signal interpreted by said interactive signal interpretation unit to update the graphical-interactive-picture-structure specification data in said storage unit;

a second process unit for retrieving updated graphical-interactive-picture-structure specification data from said storage unit, and for extracting the basic picture elements corresponding to the file names from said storage means to place the extracted display elements; and

a second display control unit for controlling said display means to display the basic picture elements placed by said second process unit as an updated graphical interactive picture.

21. The program receiver of claim 20 further comprising information transmission means for transmitting the data of the graphical interactive picture updated by said interactive manipulation means to said program transmitter.

22. The program receiver of claim 19 further comprising information transmission means for transmitting the data of the graphical interactive picture updated by said interactive manipulation means to said program transmitter.

23. The program receiver of claim 18, wherein

said basic picture elements in said storage means are composed of file names identifying each basic picture element and the bit map data for each basic picture element, and

wherein said first graphical interactive picture generation means includes:

a receipt•

decode unit for receiving the graphical-interactive-picture-structure specification data signal from said signal separation means to decode the same;

a data division unit for dividing the decoded graphical-interactive-picture-structure specification data from said receipt•

decode unit into a first graphical-interactive-picture-structure specification data and a second graphical-interactive-picture-structure specification data, said first graphical-interactive-picture-structure specification data excluding a synthesis instruction, said second graphical-interactive-picture-structure specification data including the synthesis instruction, the graphical-interactive-picture-structure specification data being composed of a panel object definition including a plurality pieces of panel object information, an action definition including a plurality pieces of action information, a shape definition including a plurality pieces of shape information, and synthesis direction data including a synthesis command in case of the second graphical-interactive-picture-structure specification data;

a first graphical-interactive-picture-structure specification data storage unit for storing the first graphical-interactive-picture-structure specification data;

a second graphical-interactive-picture-structure specification data storage unit for storing the second graphical-interactive-picture-structure specification data;

a graphical interactive picture synthesis unit for synthesizing the first graphical-interactive-picture-structure specification data in said first graphical-interactive-picture-structure specification data storage unit and the second graphical-interactive-picture-structure specification data in said second graphical-interactive-picture-structure specification data storage unit in accordance with the synthesis command of the synthesis direction data stored in said second graphical-interactive-picture-structure specification data storage unit;

a graphical-interactive-picture-structure specification data update unit for updating the first graphical-interactive-picture-structure specification data in said first graphical-interactive-picture-structure specification data storage unit with the synthesized graphical-interactive-picture-structure specification data from said graphical interactive picture synthesis unit;

a first process unit for retrieving updated graphical-interactive-picture-structure specification data from said first graphical-interactive-picture-structure specification data storage unit, and for extracting the basic picture elements corresponding to the file names by referring to the shape information in said storage means to place the extracted basic picture elements by referring to the panel object information; and

a first display control unit for controlling said display means to display the basic picture elements placed by said first process unit as an updated graphical interactive picture.

24. The program receiver of claim 23 further comprising information record means for outputting data related to the graphical interactive picture as per manipulation form said interactive manipulation means to make a record thereof.

25. The program receiver of claim 18 further comprising information transmission means for transmitting the data of the graphical interactive picture updated by said interactive manipulation means to said program transmitter.

26. The program receiver of claim 18 further comprising information record means for outputting data related to the graphical interactive picture as per manipulation form said interactive manipulation means to make a record thereof.

28. The system of claim 27, wherein said graphical interactive picture generation means includes:

a temporarily storage unit for temporarily storing the manipulation direction signal received by said manipulation direction signal receipt means;

a retrieval unit for retrieving the manipulation direction signal from said temporarily storage unit;

a graphical interactive picture structure data extract unit for extracting the graphical interactive structure data from said display'"'"'s graphical interactive picture structure data storage means in accordance with the manipulation direction signal retrieved by said retrieval unit to generate the graphical interactive picture, the graphical interactive picture structure data including object information defining objects that form the graphical interactive picture, shape information defining shapes of the objects, position information defining positions of the objects, action information defining the objects'"'"' actions that change the graphical interactive picture based on the manipulation direction signal;

a graphical display element extract unit for extracting bit map data from said graphical display element storage means based on the shape information extracted by said graphical interactive picture structure data extract unit, said bit map data being stored in said graphical display element storage means in relation with the shape information;

a synthesis generation unit for generating a graphical display by synthesizing the bit map data extracted by said graphical display element extract unit and an attribute value of a character string attribute of each object, said attribute value of the character string attribute displayed on the bit map data being included in the object information;

a placement unit for placing the graphical display synthesized by said synthesis generation unit based oh the position information in the graphical interactive picture structure data.

29. The system in claim 28, wherein said graphical interactive picture generation means further includes a graphical interactive picture structure data update unit for updating a content of the graphical interactive picture structure data in said display'"'"'s graphical interactive picture structure data storage unit based on the action information extracted by said graphical interactive picture structure data extract unit when the manipulation direction signal retrieved by said retrieval unit is an update direction.

30. The system of claim 29, wherein said display unit further includes:

TV receiver means for converting a received program into an image;

TV manipulation means for accepting a manual manipulation to said TV receiver means;

TV'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data related to said TV receiver means in advance; and

TV'"'"'s interactive picture generation means for retrieving the graphical interactive picture structure data from said TV'"'"'s graphical interactive picture structure data storage means when said TV manipulation means receives the manual manipulation to generate the graphical interactive picture by combining the graphical display elements in said graphical display element storage means to update the graphical interactive picture each time a manual manipulation is received by said TV manipulation means.

31. The system of claim 30, wherein each peripheral appliance includes:

manipulation means for accepting a user'"'"'s action direction addressed to a self'"'"'s peripheral appliance;

user s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data to display the graphical interactive picture as per action direction accepted by said manipulation means;

control means for controlling said self'"'"'s peripheral appliance to set a function by giving a direction to transmit the graphical interactive picture structure data and generating a manipulation direction signal to generate the graphical interactive picture and a manipulation direction signal to updated graphical interactive picture upon acceptance of the action direction from said manipulation means;

graphical interactive picture structure data transmission means for retrieving the graphical interactive picture structure data from said user'"'"'s graphical interactive picture structure data storage means upon receipt of the transmission direction from said control means, and for transmitting the retrieved graphical interactive picture structure data to said display unit; and

manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said control means to said display unit.

32. The system of claim 31, wherein said control means includes:

a first signal generation unit for generating the manipulation direction signal as per action direction from said manipulation means; and

a second signal generation unit for generating a predetermined subsidiary data signal depending on current action state of said self'"'"'s peripheral appliance, said subsidiary data being a supplement of the graphical interactive picture structure data and constituting an integral part thereof, said subsidiary data signal being generated by said first signal generation unit together with the manipulation direction signal.

33. The system of claim 32, wherein said manipulation means is portable.

13. In a remote control system comprising a plurality of appliances and a remote controller for transmitting control signals to said plurality of appliances,

(1) said remote controller including:

menu information request transmission means for transmitting a menu information request to each appliance, said menu information being hierarchial information unique to each appliance for enabling a user to control each appliance interactively;

menu receipt means for receiving the menu information transmitted from each appliance;

menu information synthesis means for synthesizing the menu information received by said menu receipt means;

menu display means for visually displaying synthesized menu information generated by said menu information synthesis means;

input manipulation means for accepting a user'"'"'s input manipulation to control said appliances; and

control signal transmission means for transmitting the control signals to said appliances based on the input manipulation accepted by said input manipulation means,

(2) each appliance including;

menu information request receipt means for receiving the menu information request from said menu information request transmission means;

menu information transmission means for transmitting the menu information to said menu receipt means when said menu information request receipt means receives the menu information request;

control signal receipt means for receiving the control signals from said control signal transmission means; and

control means for controlling said appliances as per control signals received by said control signal receipt means.

17. A program receiver for displaying a graphical interactive picture by receiving a program transmitted from a program transmitter, said program receiver comprising:

storage means for storing a plurality of basic picture elements in advance, said plurality of basic picture elements being figures composing the graphical interactive picture manipulated by a user;

signal receipt means for receiving a signal transmitted from said program transmitter, said signal being a multiplex signal including a program and data specifying a structure of the graphical interactive picture;

signal separation means for separating the signal received by said signal receipt means into a program signal and a graphical-interactive-picture-structure specification data signal;

first graphical interactive picture generation means for generating the graphical interactive picture by combining the basic picture elements stored in said storage means based on the graphical-interactive-picture-structure specification data signal from said signal separation means; and

display means for displaying the graphical interactive picture generated by said first graphical interactive picture generation means.

27. A system including peripheral appliances holding graphical interactive picture structure data and a display unit for displaying a graphical interactive picture for said peripheral appliances by receiving a unique manipulation direction signal from each peripheral appliance, said display unit including:

graphical interactive picture structure data receipt means for receiving the graphical interactive picture structure data from said peripheral appliances to generate the graphical interactive picture;

display'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data received by said graphical interactive picture structure data receipt means;

graphical display element storage means for storing a plurality of graphical display elements to generate the graphical interactive picture;

manipulation direction signal receipt means for receiving the manipulation direction signal from said peripheral appliances;

graphical interactive picture generation means for receiving the manipulation direction signal received by said manipulation direction signal receipt means, and for retrieving the graphical interactive picture structure data from said display'"'"'s graphical interactive picture structure data storage means to generate the graphical interactive picture by combining the graphical display elements stored in said graphical display element storage means; and

display means for displaying the graphical interactive picture generated by said graphical interactive picture generation means.

34. A system comprising a remote controller for controlling peripheral appliances and a display unit for displaying a graphical interactive picture as per manipulation direction signal from said remote controller,

(1) said remote controller including:

manipulation means for accepting an action direction addressed to said remote controller;

control means for controlling said remote controller to set a function upon acceptance of the action direction from said manipulation means, and for generating a manipulation direction signal to generate the graphical interactive picture and a manipulation direction signal to update the graphical interactive picture;

manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said control means to said display unit,

(2) said display unit including;

appliance'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data in advance to generate the graphical interactive picture for said peripheral appliances;

graphical display element storage means for storing a plurality of graphical display elements to generate the graphical interactive picture;

manipulation direction signal receipt means for receiving the manipulation direction signal from said remote controller;

graphical interactive picture generation means for retrieving the graphical interactive picture structure data from said appliance'"'"'s graphical interactive picture structure data storage means upon receipt of the manipulation direction signal received by said manipulation direction signal receipt means to generate the graphical interactive picture by combining the graphical-display elements stored in said graphical display element storage means based on the retrieved graphical interactive picture structure data;

display means for displaying the graphical interactive picture generated by said graphical interactive picture generation means;

TV receiver means for converting a received TV program into an image;

TV manipulation means for accepting a manual manipulation to said TV receiver means;

TV'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data as to said TV receiver means in advance;

TV'"'"'s graphical interactive picture generation means for retrieving the graphical interactive picture structure data from said TV'"'"'s graphical interactive picture structure data storage means when said TV manipulation means receives the manual manipulation to generate the graphical interactive picture by combining the graphical display elements stored in said graphical display element storage means based on the retrieved graphical interactive picture structure data to update the graphical interactive picture each time said TV manipulation means receives a manual manipulation.

35. A system comprising a relay, a remote controller, and a display unit, said relay being a peripheral appliance for transferring an action direction signal from said remote controller to said display unit, said relay'"'"'s function being set by said remote controller, said display unit displaying a graphical interactive picture upon receipt of the signal from said relay,

(1) said remote controller including:

first manipulation means for accepting an action of said remote controller;

first graphical interactive picture structure data storage means for storing graphical interactive picture structure data in advance to generate the graphical interactive picture as per action direction;

first control means for controlling said remote controller to set a function upon receipt of the action direction from said first manipulation means by giving a direction to transmit the graphical interactive picture structure data and generating a manipulation direction signal to generate the graphical interactive picture and a manipulation direction signal to update the graphical interactive picture;

first graphical interactive picture structure data transmission means for retrieving the graphical interactive picture structure data from said first graphical interactive picture structure data storage means upon receipt of the transmission direction from said first control means to transmit the same to said relay; and

first manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said first control means to said relay,

(2) said relay including;

second manipulation means for accepting an action direction addressed to said relay;

user'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data in advance to generate the graphical interactive picture as per action direction addressed to said relay;

second control means for controlling said relay to set a function upon receipt of the action direction from said second manipulation means by giving a direction to transmit the graphical interactive picture structure data and generating a manipulation direction signal to generate the graphical interactive picture and a manipulation direction signal to update the graphical interactive picture;

graphical interactive picture structure data transmission means for retrieving the graphical interactive picture structure data from said user s graphical interactive picture structure data storage means upon receipt of the transmission direction from said second control means to transmit the same to said display unit;

second manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said second control means to said display unit;

data relay means for receiving the graphical interactive picture structure data from said first graphical interactive picture structure data transmission means to transmit the same to said display unit; and

signal relay means for receiving the manipulation direction signal from said first manipulation direction signal transmission means to transmit the same to said display unit,

(3) said display unit including;

graphical display element storage means for storing a plurality of graphical display elements to generate the graphical interactive picture;

graphical interactive picture structure data receipt means for receiving the graphical interactive picture structure data to generate the graphical interactive picture from said relay;

display'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data received by said graphical interactive picture structure data receipt means;

manipulation direction signal receipt means for receiving the manipulation direction signal from said relay;

graphical interactive picture generation means for retrieving the graphical interactive picture structure data from said display'"'"'s graphical interactive picture structure data storage means upon receipt of the manipulation direction signal received by said manipulation direction signal receipt means to generate the graphical interactive picture by combining the graphical display elements stored in said graphical display element storage means based on the retrieved graphical interactive picture structure data, and for updating the graphical interactive picture each time said manipulation direction signal receipt means receives a manipulation direction signal; and

display means for displaying the graphical interactive picture generated by said graphical interactive picture generation means.

36. A system comprising a relay, a remote controller, and a display unit, said relay being a peripheral appliance for transferring an action direction signal from said remote controller to said display unit, said relay'"'"'s function being set by said remote controller, said display unit displaying a graphical interactive picture upon receipt of the signal from said relay,

(1) said remote controller including:

first manipulation means for accepting an action direction addressed to said remote controller;

first control means for controlling said remote controller to set a function by giving a direction to transmit graphical interactive picture structure data to said relay and generating a manipulation direction signal to generate a graphical interactive picture and to update the graphical interactive picture; and

first manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said first control means to said relay,

(2) said relay including;

second manipulation means for accepting an action direction addressed to said relay;

user'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data in advance to display the graphical interactive picture as per action direction addressed to said relay;

first graphical interactive picture structure data storage means for storing in advance the graphical interactive picture structure data to display the graphical interactive picture as per graphical-interactive-picture-structure data transmission direction from said remote controller;

second control means for controlling said relay to set a function upon receipt of the action direction from said second manipulation means by giving a direction to transmit the graphical interactive picture structure data and generating a manipulation direction signal to generate the graphical interactive picture and to update the graphical interactive picture;

graphical interactive picture structure data transmission means for retrieving the graphical interactive picture structure data from said user'"'"'s graphical interactive picture structure data storage means and said first graphical interactive picture structure data storage means upon receipt of the transmission direction from said second control means and said remote controller respectively to transmit the same to said display unit;

second manipulation direction signal transmission means for transmitting the manipulation direction signal generated by said second control means to said display unit; and

signal relay means for receiving the manipulation direction signal received by said remote control means to transmit the same to said display unit,

(3) said display unit including;

graphical display element storage means for storing a plurality of graphical display elements to generate the graphical interactive picture;

graphical interactive picture structure data receipt means for receiving the graphical interactive picture structure data to generate the graphical interactive picture from said relay;

display'"'"'s graphical interactive picture structure data storage means for storing the graphical interactive picture structure data received by said graphical interactive picture structure data receipt means,

manipulation direction signal receipt means for receiving the manipulation direction signal from said relay;

graphical interactive picture generation means for retrieving the graphical interactive picture structure data from said display'"'"'s graphical interactive picture structure data storage means upon receipt of the manipulation direction signal received by said manipulation direction signal receipt means to generate the graphical interactive picture by combining the graphical display elements stored in said graphical display elements based on the manipulation direction, and for updating the graphical interactive picture each time said manipulation direction signal receipt means receives a manipulation direction signal; and

display means for displaying the graphical interactive picture generated by said graphical interactive picture generation means.