Task computing

• US 20040230636A1
• Filed: 12/12/2003
• Published: 11/18/2004
• Est. Priority Date: 12/19/2002
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Song, Zhexuan, Masuoka, Ryusuke, Labrou, Yannis
• Granted Patent: US 8,561,069 B2
• Time in Patent Office: Days
• US Class Current: 708/800
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A computer-based system includes task computing enabling users to define tasks by combining available functionality and to execute such tasks. The computer-based system of includes available functionality which originates in devices, computing applications and electronic services available through local and remote procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously defined tasks. All available functionality is abstracted to the user as a service and each service is expressed in a service description language, and the services have a semantic description associated with them.

Claims

1. A computer-based system comprising:

a computer system executing task computing, thus enabling users to define tasks by combining available functionality and to execute such tasks.

2. The computer-based system of claim 1, wherein available functionality originates in devices, computing applications and electronic services available through remote procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously defined tasks.

3. The computer-based system of claim 2, wherein all available functionality is abstracted to the user as a service and each service is expressed in a service description language.

4. The computer-based system of claim 3 wherein each service has at least one semantic description associated with it, each semantic description can be provided by any combination of the creator of the service, the owner of the service or some other third party and the service can be made available or unavailable by making available or unavailable one or more of its semantic descriptions.

5. The computer-based system of claim 4 further comprising:

means for discovering available services;

means for filtering the services based upon relevance of the discovered services to the user and her context;

means for specifying the task possibly in collaboration with the user;

means for executing the specified task; and

optionally means for creating, removing, and managing services.

6. The computer-based system of claim 5 wherein the specified tasks are saved, automatically, or as instructed by the user, as new available services.

7. The computer-based system of claim 6, wherein the previously specified tasks saved as services are available to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

8. The computer-based system of claim 7, wherein the means for discovering services utilize a service discovery protocol, including any one or any combination of the following:

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR).

9. The computer-based system of claim 8, wherein the means for filtering discovered services include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of services when considered as a whole, individual service features.

10. The computer-based system of claim 9, wherein the means for specifying a task include any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile

11. The computer-based system of claim 10, wherein the means for executing a specified task utilizes a service execution mechanism including any one or any combination of the following:

Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed (IR), Local function call, and Local object call.

12. The computer-based system of claim 11, wherein the means for creating, removing, and managing services include any one or any combination of the following:

Graphical user interface, service execution, device interaction, events from the operating system of the user-operated device and applications executing on the user-operated device.

13. The computer-based system of claim 12, further comprising:

a module for discovering services;

a module for filtering services;

a module for automatically specifying tasks according to user'"'"'s goals or for assisting the user in specifying tasks by providing the user with information about how the services comprising the tasks can be combined;

a module for assisting the user with executing tasks, by invoking the services that the task is composed of and providing the user with means for interacting with the invoked services as they are executed, and

a User Interface that assists the user with Service Discovery, Service Filtering, Service Composition, Service Execution and the saving of defined tasks for future use

14. The computer-based system of claim 13, wherein the modules execute on a single computing device or wherein the modules, or subcomponents of the modules, are distributed across multiple computing devices and are accessible by programmatic interfaces that are accessible to the User Interface that is preferably running on the device operated by the user.

15. The computer-based system of claim 12, wherein the user engages in Service Discovery, followed by Service Filtering, followed by Task Specification, followed by Task Execution.

16. The computer-based system of claim 15, wherein the user while engaged in Task Specification iterates one or more times through Service Discovery, or Service Filtering, or both.

17. The computer-based system of claim 16, wherein the user while engaged in Task Execution iterates one or more times through Service Discovery, or Service Filtering, or Task Specification, or all three, or any combination of any successive two.

18. The computer-based system of any one of claims 15, 16, and 17, further comprising a User Interface that supports the workflows of Service Discovery, Service Filtering, Service Specification and Service Execution.

19. The computer-based system of claim 18, wherein the User Interface is a graphical one and comprises any combination of:

a Discovery Pane for displaying discovered services and filtered services;

a Details Pane for displaying additional information of a selected service from the Discovery Pane;

a Composition Pane for displaying listings of matched services from Discovery Pane;

a Construction Pane for constructing a composition of services;

an Information Pane for displaying general information about the processes of the tool, including information about the progress of the execution;

a Save Pane for saving a composition of services;

wherein all or a subset of the panes are simultaneously displayable to the user and actions on each of the panes results in automatically updating the related information displayed in other panes.

20. The computer-based system of claim 19, wherein the user can move to the Construction pane by either selecting a service from Discovery pane and choosing the construct function, or by selecting a pair of matched services and choosing the construct function.

21. The computer-based system of claim 19, wherein the user can add or remove services from the currently composed task in the Construct pane by choosing the plus or minus function for each possible position that an insertion or deletion is possible, as is automatically determined by the back-end system.

22. The computer-based system of claim 19, wherein the user can execute a specified task by choosing the execution function when that task becomes executable, as automatically determined by the underlying back-end system, while either at the Construct pane or the Composition pane.

23. The computer-based system of claim 19, wherein the system automatically appends any, pre-defined or user-defined compatible composition of available services before or after a partially specified composition by the user and executes the complete composition whenever the user chooses the execution function.

24. The computer-based system of claim 19, wherein a composition to be saved can be extended so that its constituent services satisfy requirements relating to their properties.

25. The computer-based system of claim 18, wherein the User Interface is a graphical one and comprises panes including any combination of:

a Discovery Pane for displaying discovered services and filtered services;

a Details Pane for displaying additional information of a selected service from the Discovery Pane;

a Composition Pane for Displaying listings of matched services from Discovery Pane;

a Construction Pane for constructing a composition of services;

an Information Pane for displaying general information about the processes of the tool, including information about the progress of the execution;

a Save Pane for saving a composition of services in terms of criteria;

wherein at any given time only one of the tabs is visible to the user and actions on each of the panes results in automatically updating the related information displayed in other panes.

26. The computer-based system of claim 25, wherein the user can specify a task pane by either selecting a service from Discovery pane and choosing the construct function, or by selecting a pair of matched services from the Composition tab and choosing the construct function

27. The computer-based system of claim 25, wherein the user can add or remove services from the currently composed task in the Construct pane by choosing the plus or minus function for each possible position that an insertion or deletion is possible, as is automatically determined by the back-end system.

28. The computer-based system of claim 25, wherein the user can execute a specified task by choosing the execute function when that task becomes executable, as automatically determined by the underlying back-end system, while either at the Construct pane or the Composition pane.

29. The computer-based system of claim 25, wherein the system automatically appends any, pre-defined or user-defined compatible composition of available services before or after a partially specified composition by the user and executes the complete composition whenever the user chooses the execution function.

30. The computer-based system of claim 25, wherein a composition to be saved can be extended so that its constituent services satisfy requirements relating to their properties.

31. The computer-based system of claim 18, wherein the User Interface is a graphical one and comprises a first pane for displaying discovered services and filtered services and a second pane for displaying the current composition as composed by the user and controls for going back and forth in the history of the composition and a control for executing composition whenever possible.

32. The computer-based system of claim 31, wherein the User Interface automatically places a selected service to the proper place of the composition sequence.

33. The computer-based system of claim 32, wherein the User Interface will automatically complete the composition whenever possible, including the case when the available filtered services can only be combined in a single manner for a composition that can be executed.

34. The computer-based system of either claim 18, wherein the User Interface is a web client.

35. The computer-based system of claim 34, wherein as the task is executed the user'"'"'s browser might pop-up a new window or re-direct the browser to a new link that allows the user to enter information related to the execution of the task or feedback regarding the executed task.

36. The computer-based system of claim 18, wherein the User Interface is a command line Interface, wherein each time the user types text that can be matched against discovered and/or filtered services, upon the user pressing a pre-assigned key, it inserts the matched service or, if multiple matches exist it provides a listing of them for the user to select by means of further typing text that matches one of the many services.

37. The computer-based system of claim 36, wherein the sequence of user selected service comprise a sequence that corresponds to a task and can subsequently be executed by means of pressing a pre-assigned key.

38. The computer-based system of claim 14, wherein the Discovery module, Filtering Module, Composition Module and Execution module and User Interface execute on the same computing device.

39. The computer-based system of claim 38, wherein the Composition Module comprises an Inference Engine that can identify all possible pair wise combinations of services that can be successfully executed in a composition given semantic descriptions of the constituent services.

40. The computer-based system of claim 38, wherein the Composition Module comprises an Inference Engine that can identify all possible compositions of services that can be successfully executed in their entirety given the semantic descriptions of the constituent services.

41. The computer-based system according to any one of claims 39 and 40, wherein the number of possible pair-wise matches or compositions of services that are returned by the Inference engine might be restricted by criteria that include an upper bound on computational resources, an upper bound on computing time, the number of services in a composition, or by a pre-determined total number of pair-wise matches or compositions to be computed

42. The computer-based system of claim 41, wherein the Inference Engine includes a composition logic wherein the composition logic is a rule, or a set of rules, or a set of logic statements.

43. The computer-based system of claim 38, wherein the discovery module uses local semantic service description (SSD) databases or accesses remote SSD databases and retrieves the SSDs of the found service from those databases, using the service'"'"'s ID specific to the discovery mechanism through which the service is found.

44. The computer-based system of claim 43, wherein the discovery mechanisms is UPnP and the ID is Unique Device Name (UDN) of UPnP.

45. The computer-based system of claim 38, wherein mechanism known as UPnP action is used to retrieve one or more pointers to the semantic description of the service.

46. The computer-based system of claim 14, wherein the User Interface is running on a computing device that is different than the computing devices Discovery Module, Filtering Module, Composition Module and Execution module execute.

47. The computer-based system of claim 46, wherein the User Interface is a web client (browser) that communicates with the Discovery, Filtering, Composition and Execution Modules using http.

48. The computer-based system of claim 46, wherein the User Interface is an application that executes on the end-user'"'"'s computing device and communicates with each of the Discovery, Filtering, Composition and Execution modules via the programmatic interface of these modules.

49. The computer-based system of claim 14, wherein the end-user is using any of a desktop, laptop, pen computer, PDA or mobile phone.

50. The computer-based system of claim 12, wherein a service may have one or more interfaces for the user to interact with the service prior and during its execution.

51. The computer-based system of claim 14, wherein a service comprises any one of:

“

View on Display”

which lets the user to view a user-specified document on a display and optionally control how it is shown;

“

View on Multiple Displays”

which lets the user to view a user-specified document on more than one display and optionally control how it is shown;

“

Map Viewer”

which lets the user to view a map of a user-specified location and optionally control how it is shown;

“

Map Router”

which lets the user to view a map or textual description of the route from/to a user-specified location from/to a predefined or a user-specified location optionally control how it is shown;

“

Instance Providing Service”

which provides, when executed, a semantic instance where a semantic instance is an instance of a class described in an ontology;

“

Print”

which lets the user to print a user-specified document or an semantic instance in a specific format on a printer in a pre-defined way or optionally control how it is printed;

“

Fax”

which lets the user to send a fax of a user-specified document to a user-specified fax number and optionally control how it is faxed;

“

Play Video”

which lets the user to play a user-specified video content and optionally control how it is played;

“

Play Audio”

which lets the user to play the user-specified audio content and optionally control how it is played;

“

Digital Photo Frame”

which lets the user to show a user-specified image content and optionally control how it is shown;

“

Location Determination”

which automatically determines the location of a user-operated device;

“

Directory Publisher”

which lets the user to choose one instance out of set of instances of the same type;

“

Telephone Dialer”

which lets the user to dial a user-specified phone number;

“

TC Storage”

which lets the user to store or retrieve files;

“

Copy to Removable Storage”

which lets the user to store one or a set of files into predefined removable storage media and optionally control how they are saved;

“

Emailer”

which lets the user to send email to a user-specified email address optionally with attached user-specified documents and optionally control how it is sent;

“

List chooser”

which lets the user to choose an semantic instance from the list of semantic instances of the same type;

“

Property chooser”

which lets the user to choose one of the properties from a semantic instance;

“

Your input”

which lets the user to input the values for the attributes of the semantic object;

“

Instance viewer”

which lets the user to check and modify a semantic instance;

“

Local Instance Selector”

which lets the user to select one or more semantic instances local to the computing device;

“

Semantic Instance Copier”

which lets the user to copy the semantic instance into the local computing device;

“

L-Note”

which lets the user to read and leave notes pertinent to a location;

“

Database publisher”

which publishes a whole of or a part of a table from a database;

“

Place information”

which provides information related to the place; and

“

Hosted Services”

which uses the client-side browser functionalities to accomplish their tasks, including “

Upload File”

to let the user to upload a file from the client device, “

URL to input”

to let the user to input an URL, and “

View on Browser”

to let the user view the file on the client device.

52. The computer-based system of claim 12, wherein the means for creating, removing, and managing services includes any one or combination of the following:

means for making SDSs available through one or more of the discovery mechanisms,

means for changing the discovery mechanism to use,

means for holding the SDS,

means for recovering the SDS to one of the discovery mechanisms,

means for removing the SDSs from discovery mechanisms, and

means for changing its publishing parameters including expiration, invocation limit, and access control.

53. The computer-based system of claim 52, wherein the means for creating, removing, and managing services comprise any one or a combination of:

“

PIPE”

which provides local or remote APIs to publish and optionally manage services, and optionally provides the user with user interfaces to publish and manage the services;

“

Semantic Instance Scrapers for Applications”

which publishes the instances as they become available for the application and optionally lets the user manage the service provisions;

“

Instance Selector with Instance Providing Service Creation”

which lets the user publish the instance the user selected for the instance selector service and optionally lets the user manage the service provisions;

“

White hole”

which lets the user publish objects from a local computing device as semantic instances through its user interface and optionally lets the user manage the service provisions;

“

Public Directory”

which lets the user publish instances on a device and optionally lets the user manage the service provisions;

“

Bank”

which lets the user publish instances through its service invocation and optionally lets the user manage the service provisions;

“

Image/AudioNideo Service Control”

which lets the user publish instances created by or from the devices and optionally lets the user manage the service provisions;

“

Removable Media Publisher”

which lets the user publish instances on the removable media inserted into a computing device and optionally lets the user manage the service provisions; and

“

Fax”

which publishes the documents received and optionally lets the user manage the service provisions.

54. A computer-based method comprising:

performing task computing, thus enabling users to define tasks by combining available functionality and to execute such tasks.

55. The computer-based method of claim 54, wherein available functionality originates in devices, computing applications and electronic services available through remote procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously defined tasks.

56. The computer-based method of claim 55, wherein all available functionality is abstracted to the user as a service and each service is expressed in a service description language.

57. The computer-based method of claim 56 wherein:

each service has at least one semantic description associated with it,

each semantic description can be provided by any combination of the creator of the service, the owner of the service or some other third party, and

the service can be made available or unavailable by making available or unavailable one or more of its semantic descriptions.

58. The computer-based method of claim 57 further comprising:

discovering available services;

filtering the services based upon relevance of the discovered services to the user and her context;

specifying the task possibly in collaboration with the user;

executing the specified task; and

optionally creating, removing, and managing services.

59. The computer-based method of claim 58 wherein the specified tasks are saved, automatically, or as instructed by the user, as new available services.

60. The computer-based method of claim 59, wherein the previously specified tasks saved as services are available to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

61. The computer-based method of claim 60, wherein the means for discovering services utilize a service discovery protocol, including any one or any combination of the following:

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR).

62. The computer-based method of claim 61, wherein the filtering discovered services include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of services when considered as a whole, individual service features.

63. The computer-based method of claim 62, wherein the specifying a task include any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile

64. The computer-based method of claim 63, wherein the means for executing a specified task utilizes a service execution mechanism including any one or any combination of the following:

Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed (IR), Local function call, and Local object call.

65. The computer-based method of claim 64, wherein the creating, removing, and managing services include any one or any combination of the following:

Graphical user interface, service execution, device interaction, events from the operating system of the user-operated device and applications executing on the user-operated device.

66. A computer-readable medium storing a program which when executed by a computer causes the computer to execute the functions comprising:

performing task computing, thus enabling users to define tasks by combining available functionality and to execute such tasks.

67. The computer-readable medium of claim 66, wherein available functionality originates in devices, computing applications and electronic services available through remote procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously defined tasks.

68. The computer-readable medium of claim 67, wherein all available functionality is abstracted to the user as a service and each service is expressed in a service description language.

69. The computer-readable medium of claim 68, wherein:

each service has at least one semantic description associated with it,

each semantic description can be provided by any combination of the creator of the service, the owner of the service or some other third party, and

the service can be made available or unavailable by making available or unavailable one or more of its semantic descriptions.

70. The computer-readable medium of claim 69, further comprising:

discovering available services;

filtering the services based upon relevance of the discovered services to the user and her context;

specifying the task possibly in collaboration with the user;

executing the specified task; and

optionally creating, removing, and managing services.

71. The computer-readable medium of claim 70, wherein the specified tasks are saved, automatically, or as instructed by the user, as new available services.

72. The computer-readable medium of claim 71, wherein the previously specified tasks saved as services are available to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

73. The computer-readable medium of claim 72, wherein the discovering services utilize a service discovery protocol, including any one or any combination of the following:

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR).

74. The computer-readable medium of claim 73, wherein the filtering discovered services include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of services when considered as a whole, individual service features.

75. The computer-readable medium of claim 74, wherein the specifying a task include any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile

76. The computer-readable medium of claim 75, wherein the executing a specified task utilizes a service execution mechanism including any one or any combination of the following:

Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed (IR), Local function call, and Local object call.

77. The computer-readable medium of claim 76, wherein the creating, removing, and managing services include any one or any combination of the following:

Graphical user interface, service execution, device interaction, events from the operating system of the user-operated device and applications executing on the user-operated device.