System and method of dual-screen interactive digital television

• US 7,992,187 B2
• Filed: 09/28/2007
• Issued: 08/02/2011
• Est. Priority Date: 05/15/2007
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chiao, Hsin-Ta
• Primary Examiner(s): Koenig; Andrew
• Assistant Examiner(s): CHUNG, JASON
• Application Number:
  US11/863,248
• Publication Number:
  US 20080284907A1
• Time in Patent Office: 1,404 Days
• Field of Search: 725/80, 725/81, 725/82, 725/133, 725/141, 725/153, 715/238, 715/252, 715716-719, 348/734
• US Class Current: 725/153
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

Disclosed is a system and method of dual-screen interactive digital television (IDTV), which is implemented by applying modality-independent remote console technology. The system has two ends, referred to as host IDTV and handheld device. The host IDTV includes a host IDTV content, an IDTV middleware, a host graphic user interface (host GUI), and a server-side remote console control protocol. The handheld device includes a handheld IDTV content, an interface generator, a handheld GUI, and a client-side remote console control protocol. In the invention, the DTV content may only have a dual-screen execution mode. The DTV content may also switch between single-screen and dual-screen modes.

Claims

1. A dual-screen interactive digital television (IDTV) system, realized with a single-screen mode and a dual-screen mode by applying a modality-independent remote console technology, said system comprising:

a host IDTV, including a host IDTV content, an IDTV middleware in communication with and managing said host IDTV content, a host graphic user interface (GUI) in communication with said IDTV middleware, and a server-side remote console control protocol in communication with said IDTV middleware and said host GUI, said host IDTV content at least including a computer program or a combination of a computer program and data; and

a handheld device, including a handheld IDTV content comprising two user interface (UI) abstract descriptions that contain UI functional information without detailed presentation information of a concrete UI, an interface generator parsing said UI abstract descriptions and constructing a handheld concrete UI, a handheld GUI in communication with said interface generator, and a client-side remote console control protocol in communication with said interface generator, said two UI abstract descriptions including a UI abstract description of a single-screen IDTV input device realized in said single-screen mode, and a UI abstract description of said host IDTV content realized in said dual-screen mode;

wherein said server-side remote console control protocol and said client-side remote console control protocol are connected by a multi-layer networking technology, UI input information to said host IDTV content is relayed through said client-side remote console control protocol and said multi-layer networking technology to said server-side remote control protocol, and transmitted by said IDTV middleware to said host IDTV content, UI output information from said host IDTV content is transmitted through said IDTV middleware to either said handheld GUI or said host GUI, and in said dual-screen mode, an AV content is presented on said host GUI but not on said handheld GUI and said UI output information from said host IDTV content is presented on said handheld GUI but not on said host GUI, said server-side remote control protocol enables said host IDTV content to become a remote console server with said handheld concrete UI of said handheld device being a remote console when said host IDTV content is a computer program or a combination of a computer program and data, and said server-side remote control protocol enables said IDTV middleware to become a remote console server when said host IDTV content is data.

2. The system as claimed in claim 1, wherein said multi-layer networking technology is the combination of a network technology and a connection technology.

3. The system as claimed in claim 1, wherein said system automatically generates said UI abstract descriptions required by said modality-independent remote console technology.

4. The system as claimed in claim 1, wherein said multi-layer networking technology is realized with two communication protocol components, two pieces of communication hardware, and a physical communication network.

5. The system as claimed in claim 1, wherein said host GUI is realized with an OS GUI module and a host GUI hardware.

6. The system as claimed in claim 1, wherein said handheld GUI is realized with an OS GUI module and a handheld GUI hardware.

7. The system as claimed in claim 1, wherein said system is switchable between said single-screen mode and said dual-screen mode.

8. The system as claimed in claim 1, wherein UI of said host IDTV content in said dual-screen mode is generated by said interface generator according to said UI abstract description of said host IDTV content and is presented on a screen of said handheld device by said handheld GUI.

9. The system as claimed in claim 1, wherein in said single-screen mode, said interface generator, according to said UI abstract description of said single-screen IDTV input device through said handheld GUI, presents UI on a screen of said handheld device.

10. The system as claimed in claim 1, wherein when said host IDTV content is a computer program, said host IDTV content includes an application logic component, a UI abstraction layer, a host UI adaptation layer, and a remote console adaptation layer.

11. The system as claimed in claim 1, wherein when said IDTV content is a computer program, said IDTV middleware includes a host API and a remote console API.

12. The system as claimed in claim 1, wherein when said host IDTV content is data, said host IDTV content includes markup language data and script, said markup language data includes presentation-irrelevant data and presentation data, and said presentation data describe how to present the combination of said presentation-irrelevant data and said script on both a handheld device remote console and said host GUI, respectively.

13. The system as claimed in claim 12, wherein said IDTV middleware is realized with a content parser, an interpreting engine, a host presentation engine and a remote console presentation engine, said content parser is responsible for loading and delivering said host IDTV content to said interpreting engine, said interpreting engine interprets and maintains said presentation-irrelevant data and executes said script, said host presentation engine presents an interpreted data page on a screen of said host IDTV, and said remote console presentation engine presents said interpreted data page in a remote console form on a handheld device screen.

14. The system as claimed in claim 1, wherein when said host IDTV content is data, said host IDTV content includes markup language data and script, and presentation-irrelevant data and presentation data of said host GUI are mixed in said markup language data.

15. The system as claimed in claim 14, wherein said IDTV middleware is realized with a content parser, a legacy interpreting and rendering engine, and a remote console plugin, said legacy interpreting and rendering engine interprets said presentation-irrelevant data and said presentation-related data of said host GUI, and executes said script, and said remote console plugin turns off the presentation function of said legacy interpreting and rendering engine on said host IDTV, and intercepts internal events of said legacy interpreting and rendering engine and presents said host IDTV content in a remote console form on a handheld device screen.

16. The system as claimed in claim 1, wherein components of said host IDTV apply said server-side remote console control protocol to become a remote console server and allow said handheld device to operate said components.

17. The system as claimed in claim 1, wherein said interface generator parses said UI abstract descriptions, and constructs a handheld UI concrete description according to the context of said handheld device.

18. The system as claimed in claim 1, wherein said client-side remote console control protocol provides a remote console client API that allows said interface generator to call.

19. A method for operating dual-screen interactive digital television (IDTV) realized with a single-screen mode and a dual-screen mode, applicable to a host IDTV and a handheld device, said method comprising:

applying a modality-independent technology to generate user interface (UI) abstract descriptions containing UI functional information without detailed presentation information of a concrete UI according to a DTV content, where said DTV content including a host IDTV content and a handheld IDTV content, said host IDTV content including a computer program or a combination of a computer program and data, said UI abstract descriptions including a UI abstract description of said host IDTV content realized in said dual-screen mode, and a UI abstract description of a single-screen IDTV input device realized in said single-screen mode;

re-directing said UI abstract descriptions to said handheld device; and

arranging said DTV content to execute in either said single-screen mode or said dual-screen mode;

wherein in said dual-screen mode, an AV content of said DTV content is presented on a host GUI of said host IDTV but not on a handheld GUI of said handheld device and UI output information from said host IDTV content is presented on said handheld GUI but not on said host GUI, said host IDTV content is a remote console server with a handheld concrete UI of said handheld device being a remote console when said host IDTV content is a computer program or a combination of a computer program and data, and an IDTV middleware is a remote console server when said host IDTV content is data.

20. The method as claimed in claim 19, wherein said UI abstract descriptions are stored in said handheld IDTV content.

21. The method as claimed in claim 19, wherein said DTV content is chosen from any one combination of at least a host IDTV content, at least a handheld IDTV content, an audio content, and a video content.

22. The method as claimed in claim 19, wherein said UI abstract description of said single-screen IDTV input device is generated before the broadcasting of said DTV content, and is attached to said handheld IDTV content.

23. The method as claimed in claim 19, wherein said UI abstract description of said single-screen IDTV input device is generated by said host IDTV, and is attached to said handheld IDTV content of said DTV content when said host IDTV transmits said handheld IDTV content to said handheld device.

24. The method as claimed in claim 19, wherein said method in said dual-screen mode further includes the steps of:

receiving said handheld IDTV content, translating said UI abstract description of said host IDTV content into a concrete UI of said handheld device, and presenting said concrete UI on a screen of said handheld device; and

using said concrete UI on said handheld device to input information to said host IDTV content, and presenting output information from said host IDTV content on said concrete UI of said handheld device.

25. The method as claimed in claim 19, wherein said method in said single-screen mode further includes the steps:

parsing said UI abstract description of said single-screen IDTV input device, and translating said UI abstract description of said single-screen IDTV input device into a concrete UI of said handheld device, said concrete UI being a remote console for said single-screen IDTV input device; and

using said concrete UI to input information to said host IDTV content, and presenting output information from said host IDTV content on a screen of said host IDTV.

26. The method as claimed in claim 24, wherein said UI abstract description of said host IDTV content is translated into said concrete UI of said handheld device according to the context of said handheld device.

27. The method as claimed in claim 19, wherein said method uses a server-side remote console control protocol, a client-side remote console control protocol and a multi-layer networking technology to operate on said host IDTV and said handheld device.

28. The method as claimed in claim 19, wherein said UI abstract description of said host IDTV content is automatically generated by a DTV authoring tool during the production of said DTV content.

29. The method as claimed in claim 28, wherein automatic generation of said UI abstract description of said host IDTV content further includes the steps of:

generating a concrete UI of said host IDTV content; and

removing detailed presentation information from said concrete UI, and leaving only UI function description and UI presentation hint to obtain said UI abstract description.