Method and system for presenting user tasks for the control of electronic devices

• US 8,185,427 B2
• Filed: 09/22/2004
• Issued: 05/22/2012
• Est. Priority Date: 09/22/2004
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Messer, Alan, Kunjithapatham, Anugeetha
• Primary Examiner(s): Loftis, Johnna
• Assistant Examiner(s): Parker, Brandi P
• Application Number:
  US10/947,774
• Publication Number:
  US 20060064693A1
• Time in Patent Office: 2,799 Days
• Field of Search: 705/7.21
• US Class Current: 705/7.21
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06Q 10/1097 Task assignment
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612

Abstract

A method of representing user tasks to be performed by one or more of a plurality of electronic devices in a task orchestration system. Each device includes therein a function description specifying a function that the device can perform. The function descriptions are obtained from one or more of the devices, task descriptions that describe device functionalities required for certain tasks are obtained, and task suggestion are generated based on the obtained device function descriptions and task descriptions. Each task suggestion represents a user task as an abstraction of on or more of the obtained device descriptions and task descriptions. The tasks suggestion are displayed on a display for the user to select from, wherein the user selected task suggestion is to be performed by orchestration (automation) of one or more of the devices based on task descriptions that use device functionalities described in the device function descriptions.

Claims

1. A method of representing user tasks to be performed by one or more of a plurality of electronic devices in a task orchestration system, the method comprising:

employing a controller device for;

obtaining device function descriptions, wherein each device function description describes a function that a device can perform;

obtaining task descriptions from multiple electronic devices of a plurality of electronic devices, wherein the task descriptions are dynamically determined at run-time of the task orchestration system, wherein each task description describes the device functionality a certain task requires;

generating task suggestions based on the obtained task descriptions and device function descriptions, wherein each task suggestion represents a user task as an abstraction of one or more of the obtained device functions descriptions and task descriptions;

applying user preferences over the generated task suggestions to determine a rank order for the generated task suggestions; and

displaying task suggestions in the rank order on a display for the user to select from, wherein the user selected task suggestion is to be performed by one or more of the devices.

2. The method of claim 1, wherein a task suggestion is represented using a language structure organized as a set of terms to describe user tasks as abstractions of the obtained device function descriptions and task descriptions.

3. The method of claim 2, wherein in the language structure one or more terms modify one another.

4. The method of claim 3, wherein a modifying term reduces the scope of another term.

5. The method of claim 4, wherein the terms comprise ‘

verbs’ and

‘

subjects’

organized wherein ‘

subjects’

modify ‘

verbs’

.

6. The method of claim 5, wherein the ‘

verbs’

define user task actions.

7. The method of claim 3, wherein the terms comprise ‘

verbs’

, ‘

subjects’ and

‘

modifiers’

, wherein ‘

subjects’

modify ‘

verbs’ and

‘

modifiers’

modify ‘

subjects’

.

8. The method of claim 3, wherein the displaying the task suggestions further includes displaying task suggestions for tasks that can be performed by the available devices.

9. The method of claim 8, wherein the displaying the task suggestions further includes displaying said terms, and in selecting task suggestions, allowing the user to select terms such that the user selected task suggestion can be performed by the available devices.

10. The method of claim 2, wherein the language structure provides a standardized vocabulary that allows the users to express user tasks.

11. The method of claim 2, wherein the task suggestions mask device functions from the user.

33. The method of claim 1, wherein the modifiers describe a time of day to apply to the task suggestions.

34. The method of claim 1, wherein the modifiers describe a particular location to apply to the task suggestions.

35. The method of claim 34, wherein the task descriptions further comprise task functionalities that express the functionalities required by a task.

36. The method of claim 35, wherein the device function descriptions further comprise device functionalities, wherein the device functionalities further comprise device attributes and device control information.

37. The method of claim 36, wherein the task descriptions are de-coupled from the device function descriptions, wherein the task functionalities in the task descriptions map to the device functionalities in the device function descriptions.

38. The method of claim 37, wherein the task descriptions are generated independently of the device descriptions.

39. The method of claim 38, wherein generating task suggestions further comprises:

determining available devices that meet requirements of the task functionalities; and

,

recursing through the available devices to determine combinations of the available devices that fulfill the requirements of the task functionalities.

12. A task orchestration system for a network of interconnected devices, comprising:

a task-driven controller that obtains device function descriptions which describe functions that the devices can perform, obtains task descriptions from multiple electronic devices of a plurality of electronic devices, wherein the task descriptions are dynamically determined at run-time of the task orchestration system, wherein the task descriptions describe the device functionalities that tasks require, and generates task suggestions based on the obtained device function descriptions, task descriptions and user preferences, wherein each task suggestion represents a user task as an abstraction of one or more of the obtained device descriptions and task descriptions; and

a user interface device that displays-the task suggestions in rank order based on the user preferences on a display for the user to select from, wherein the user selected task suggestion is to be performed by one or more of the available devices.

13. The system of claim 12, wherein a task suggestion is represented using a language structure organized as a set of terms to describe user tasks as abstractions of device functions descriptions and task descriptions.

14. The system of claim 13, wherein in the language structure one or more terms modify each other.

15. The system of claim 14, wherein a modifying term reduces the scope of another term.

16. The system of claim 14, wherein the terms comprise ‘

verbs’ and

‘

subjects’

, wherein ‘

subjects’

modify ‘

verbs’

.

17. The system of claim 16, wherein the ‘

verbs’

define user task actions.

18. The system of claim 14, wherein the terms comprise ‘

verbs’

, ‘

subjects’ and

‘

modifiers’

, wherein ‘

subjects’

modify ‘

verbs’ and

‘

modifiers’

modify ‘

subjects’

.

19. The system of claim 14, wherein the user interface device displays task suggestions that can be performed by the devices.

20. The system of claim 14, wherein the user interface device displays said terms that represent task suggestions, and in selecting task suggestions, allows the user to select terms to modify other terms such that the user selected task suggestion can be performed by the available devices.

21. The system of claim 13, wherein the language structure provides a standardized vocabulary that allows the users to express user tasks.

22. The system of claim 12, wherein the tasks mask device function descriptions and task descriptions from the user.

23. The system of claim 12 wherein the task driven controller further controls one or more of the devices to perform the user selected task suggestion.

24. The system of claim 23, wherein the task driven controller controls one or more of the devices to perform the selected task suggestion based on task descriptions that use device functionalities described in the device function descriptions.

25. The system of claim 12, wherein:

device function descriptions are framed as device ontologies;

task descriptions are framed as task ontologies; and

the controller uses the device and task ontologies to map the user selected task suggestions to tasks that can be performed by the available devices.

26. The system of claim 12, wherein:

device function descriptions are framed as device ontologies;

task descriptions are framed as task ontologies;

device instances are framed based on device ontologies;

task instances are framed based on task ontologies; and

the controller further uses the device and task ontologies and instances to map the user selected task to tasks that can be performed by the available devices.

27. The system of claim 13, wherein the language structure is represented in a form which closely matches a natural language.

28. The system of claim 27, wherein selecting terms reduces the scope of a task suggestion.

29. The system of claim 27, wherein only task suggestions are displayed which can be performed by the devices.

30. The system of claim 27, wherein task suggestions are displayed by displaying said terms that represent the task suggestions, and the user is allowed to select terms such that the task suggestions represented by the user selected terms can be performed by the available devices.

31. The system of claim 27, wherein in the language structure one or more terms modify one another.

32. The system of claim 31, wherein a modifying term reduces the scope of another term.

40. A method of representing user tasks to be performed by one or more of a plurality of electronic devices in a task orchestration system, the method comprising:

employing a controller device for;

asserting device and task ontologies into a knowledge base;

asserting instances of devices and tasks into the knowledge base;

extracting verbs and subjects contained in the knowledge base;

extracting device functionalities required to perform tasks from the knowledge base; and

,

generating task suggestions based on the tasks in the knowledge base and the extracted functionalities;

displaying the task suggestions to the user employing a user interface, wherein displaying task suggestions further comprises;

displaying terms comprising verbs, subjects and related content;

in selecting task suggestions, allowing a user to select from said terms such that a user selected task suggestion can be performed by the available devices;

forming a user selection request based on user the selected verb, subject, and content;

identifying solutions associated with the user selection request, wherein identifying the solutions further comprises identifying devices that posses the required functionalities associated with the user selection request;

applying user preferences over the solutions and ranking the solutions; and

employing the user interface for displaying the ranked solutions to the user.