Task computing

• US 8,561,069 B2
• Filed: 12/12/2003
• Issued: 10/15/2013
• Est. Priority Date: 12/19/2002
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Masuoka, Ryusuke, Labrou, Yannis, Song, Zhexuan
• Primary Examiner(s): ZHE, MENG YAO
• Application Number:
  US10/733,328
• Publication Number:
  US 20040230636A1
• Time in Patent Office: 3,595 Days
• Field of Search: 718/100
• US Class Current: 718/100
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A computer-based system includes task computing enabling users to define tasks by combining available functionality and to execute such tasks. The computer-based system of includes available functionality which originates in devices, computing applications and electronic services available through local and remote procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously defined tasks. All available functionality is abstracted to the user as a service and each service is expressed in a service description language, and the services have a semantic description associated with them.

Claims

1. A computer-based system capable of communication with a plurality of different types of computing sources of functionality, each computing source of functionality being a service to a user, comprising:

non-transitory computer readable medium that stores a semantic service description (SSD) that is associated with a service and, each SSD based upon one or more ontologies for filtering, composing and executing the service, and is discoverable as an available service according to one or more discovery protocols, wherein an SSD further comprises;

a semantic description of the service, including a semantic description of input/output parameters of the service as semantic input/output parameters, based upon the one or more ontologies,

a semantic filter parameter specifying relevance of the service, based upon the one or more ontologies, and

a grounding including;

a service invocation interface to the service; and

an input/output parameter mapping between the semantic input/output parameters and syntactic input/output interface parameters of the service and/or an input/output parameter transformation function between the semantic input/output parameters and syntactic input/output interface parameters of the service;

a computer processor capable of executing;

dynamically discovering any available SSDs as available services through the one or more discovery protocols to discover the SSDs;

supporting real-time composition by a user of a current executable task that is a combination of two or more of the available services by dynamically presenting to the user possible services by filtering the discovered available services according to a context of the user in a computing environment for a service selection from the available services and/or according to the current composed executable task, based upon the semantic filter parameters in the SSDs of the discovered available services; and

executing a task by invoking the two or more available services that comprise the task, including enabling the user to interact with an invoked available service, based upon the grounding in the associated SSDs including the service invocation interface, and the input/output mapping and/or the transformation function between the semantic input/output parameters and the syntactic input/output parameters.

2. The computer-based system of claim 1, wherein the computing sources of functionality originate in devices, computing applications, electronic services and/or previously defined tasks.

3. The computer-based system of claim 1, wherein the SSD is expressed in a service description language.

4. The computer-based system of claim 1, wherein

each SSD is provided by any combination of a creator or owner of the computing source of functionality or some other third party and

the computer processor further executes making available or unavailable a service by making available or unavailable discovery of one or more of SSDs associated with the service.

5. The computer-based system of claim 1 wherein the processor further executes saving composed executable tasks automatically, or as instructed by the user, as new available services.

6. The computer-based system of claim 5, wherein the composed executable tasks saved as services are available to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

7. The computer-based system of claim 1, wherein the one or more discovery protocols include one or any combination of the following:

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR).

8. The computer-based system of claim 1, wherein the filtering of the discovered available services according to the context of the user include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of the available services when considered as a whole, individual available service features.

9. The computer-based system of claim 1, wherein the real-time composing by the user of the executable task includes any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile.

10. The computer-based system of claim 1, wherein the SSD service invocation interface is according to one or more of a remote procedure call including Web Services, SOAP, UPnP actions, JINI, CORBA, RMI, RPC, DCE, DCOM, KQML, FIPA-ACL, or InfraRed (IR), or local function call, or local object call, or code described directly in the SSD.

11. The computer-based system of claim 1, wherein the computer rocessor further executes managing a computing source of functionality as an available service through the associated SSD by any one or any combination of the following:

a graphical user interface controlling creation, provision, holding, and/or removal of the associated SSD, executing or interacting with the computing source of functionality, or processing events from an operating system of a user-operated device and applications executing on the user-operated device.

12. The computer-based system of claim 1, wherein the computer processor executes:

a module to publish the SSD as the available service by making the SSD discoverable through one of the plurality of discovery protocols;

a module to discover the SSDs as the available services as a Service Discoverer;

a module to filter the SSDs as filtered available services as a Service Filter;

a module to automatically specify a task of two or more available services or to assist the user in specifying the task of two or more available services to comprise the task, as a Task Specifier;

a module to assist the user with executing the task by invoking the available services that comprise the task, including enabling the user to interact with an invoked service as the available services are executed, as a Task Executer, and

a User Interface to the Service Discoverer, the Service Filter, the Task Specifier, and the Task Executer.

13. The computer-based system of claim 12, further comprising a single computing device or a distributed networked computer system, wherein the modules execute on the single computing device, or wherein the modules, or subcomponents of the modules, execute as distributed across multiple networked computing devices in the distributed networked computer system, and the modules are accessible by programmatic interfaces accessible to the User Interface invoked at a relevant computing device operated by the user.

14. The computer-based system of claim 12, wherein the User Interface enables an executable workflow composition of the task according to a sequence of invoking the Service Discovery, followed by the Service Filter, followed by the Task Specifier, followed by the Task Executer, or any combinations thereof.

15. The computer-based system of claim 14, wherein the User Interface is graphical and comprises any combination of:

a Discovery Pane for displaying selectable discovered and filtered services;

a Details Pane for displaying additional information of a selected service from the Discovery Pane;

a Composition Pane for displaying listings of compatible services, according to the filtering, from the Discovery Pane;

a Construction Pane for interactively specifying a workflow composition of selected compatible services as the task;

an Information Pane for displaying status information;

a Save Pane for saving a composition of services;

wherein the User Interface displays all or a subset of the panes simultaneously to the user, and

wherein the User Interface real-time updates information in each pane according to a result of an action in another pane.

16. The computer-based system of claim 15, wherein the User Interface enables navigating to the Construction pane by either selecting a service from the Discovery Pane and selecting a displayed construct function, or by selecting a pair of compatible services and selecting the displayed construct function.

17. The computer-based system of claim 15, wherein the User Interface enables adding and/or removing services from a currently composed executable task in the Construction Pane by choosing a plus or minus function for each possible workflow position that an insertion or deletion is possible, according to the discovering and the filtering.

18. The computer-based system of claim 15, wherein the User Interface enables the user to execute a specified task by choosing an execution function when the task becomes executable, as automatically determined according to the executing, while either at the Construction Pane or the Composition pane.

19. The computer-based system of claim 15, wherein the executing further comprises automatically completing a composition of services as a task by appending any, pre-defined or user-defined, compatible composition of available services before or after a partially specified composition of services by the user, and executing the complete composition in response to a user input.

20. The computer-based system of claim 15, wherein a saved composition of services as the task is extendable with other available discovered and filtered services.

21. The computer-based system of claim 12, wherein the User Interface is graphical and comprises selectable tab window panes including any combination of:

a Discovery Pane for displaying selectable discovered and filtered services;

a Details Pane for displaying additional information of a selected service from the Discovery Pane;

a Composition Pane for Displaying listings of compatible services, according to the filtering, from the Discovery Pane;

a Construction Pane for interactively specifying a workflow composition of services as the task;

an Information Pane for displaying status information; and

a Save Pane for saving a composition of services,

wherein the User Interface visibly displays at any given time only one of window pane tabs to the user, and

wherein the User Interface real-time updates information in each pane according to a result of an action in another pane.

22. The computer-based system of claim 14, wherein the User Interface is graphical and comprises a first window pane to display discovered and filtered services, a second window pane to display a current task composition by the user, controls to navigate a displayed history of the task composition, and a control execute the task composition.

23. The computer-based system of claim 22, wherein the User Interface automatically places a selected service to a proper place in the workflow composition of the task.

24. The computer-based system of claim 23, wherein the User Interface automatically completes the task composition upon an available filtered compatible service.

25. The computer-based system of claim 14, wherein the User Interface is a web client.

26. The computer-based system of claim 25, wherein the web client comprise a browser to pop-up a new window or re-direct the browser to a new link enabling the user to enter information related to the Task Execution or enter feedback regarding the executed task.

27. The computer-based system of claim 14, wherein the User Interface is a command line interface, wherein each time the user types text compatible with a discovered and/or a filtered service, the User Interface inserts in the workflow composition of the task the compatible service or, if multiple compatible services exist the User Interface provides a selectable listing thereof.

28. The computer-based system of claim 27, wherein a displayed sequence of user selected services corresponds to the task, and a single user input executes the task.

29. The computer-based system of claim 12, wherein the Service Discovery, the Service Filter, the Task Specifier, the Task Executer, and the User Interface execute on same computing device.

30. The computer-based system of claim 12, wherein the Task Specifier comprises an Inference Engine presenting to the user the feasible possible executable tasks by identifying all possible pair wise combinations of the available services for an executable workflow composition of the task.

31. The computer-based system of claim 29, wherein the Task Specifier comprises an Inference Engine presenting to the user the feasible possible executable tasks by identifying all possible compositions of the available services as possible executable tasks.

32. The computer-based system according to any one of claims 30 and 31, wherein the Inference Engine returns a number of the possible pair-wise combinations of the available services or executable task compositions according to criteria that include an upper bound on computational resources, an upper bound on computing time, a number of the available services in a composition, or by a pre-determined total number of the possible pair-wise combination of services or task compositions to be computed.

33. The computer-based system of claim 32, wherein the Inference Engine includes a rule, or a set of rules, or a set of logic statements, as a task composition logic.

34. The computer-based system of claim 12, further comprising one or more local and/or remote databases to store one or more SSDs, wherein the Service Discoverer uses a local SSD database or accesses a remote SSD to retrieve a corresponding associated SSD of a source of functionality presenting a service, according to an identifier (ID) specific to a discovery mechanism used to discover the source of functionality presenting the service.

35. The computer-based system of claim 34, wherein one of the plurality of the discovery mechanisms is UPnP and the ID is Unique Device Name (UDN) of UPnP.

36. The computer-based system of claim 29, wherein one of the discovery mechanisms is UPnP discovery.

37. The computer-based system of claim 13, wherein the User Interface executes on a computing device different than computing devices executing the Service Discoverer, the Service Filter, the Service Specifier, or the Task Executer.

38. The computer-based system of claim 37, wherein the User Interface is a web client (browser) that communicates by HTTP with the Service Discoverer, the Service Filter, the Task Specifier, or the Task Executer.

39. The computer-based system of claim 37, wherein the User Interface is an application that executes on a user computing device and communicates by an application programming interface with Service Discoverer, the Service Filter, the Task Specifier, or the Task Executer.

40. The computer-based system of claim 13, wherein the single computing device is any of a desktop, laptop, pen computer, PDA or a mobile phone.

41. The computer-based system of claim 12, wherein a service may have one or more interfaces for the user to interact with the service prior and during execution of the service.

42. The computer-based system of claim 12, wherein the SSDs as the available services comprise any one of:

“

View on Display”

to view a user-specified document on a display and optionally control how the document is displayed;

“

View on Multiple Displays”

to view a user-specified document on more than one display and optionally control how the document is displayed;

“

Map Viewer”

to view a map of a user-specified location and optionally control how the map is displayed;

“

Map Router”

to view a map or textual description of a route from/to a user-specified location from/to a predefined or by a user-specified location optionally control how the map or textual description is displayed;

“

Instance Providing Service”

to provide a semantic instance of a class described in an ontology;

“

Print”

to print a user-specified document or a semantic instance in a specific format on a printer in a pre-defined way or optionally control how the document or the semantic instance is printed;

“

Fax”

to send a fax of a user-specified document to a user-specified fax number and optionally control how the document is faxed;

“

Play Video”

to play a user-specified video content and optionally control how the video is played;

“

Play Audio”

to play a user-specified audio content and optionally control how the audio is played;

“

Digital Photo Frame”

to display a user-specified image content and optionally control how the image is displayed;

“

Location Determination”

to automatically determine a location of a user-operated device;

“

Directory Publisher”

to choose one instance out of a set of instances of the same type;

“

Telephone Dialer”

to dial a user-specified phone number;

“

Storage”

to store or retrieve files;

“

Copy to Removable Storage”

to store one or a set of files into predefined removable storage media and optionally control how the files are saved;

“

Emailer”

to send email to a user-specified email address optionally with attached user-specified documents and optionally control how the email is sent;

“

List chooser”

to choose an semantic instance from a list of semantic instances of the same type;

“

Property chooser”

to choose one of properties from a semantic instance;

“

Your input”

to input values for attributes of a semantic object;

“

Instance viewer”

to check and modify a semantic instance;

“

Local Instance Selector”

to select one or more local semantic instances;

“

Semantic Instance Copier”

to copy a semantic instance;

“

L-Note”

to read and leave notes pertinent to a location;

“

Database publisher”

to publish a whole of or a part of a table from a database;

“

Place information”

to provide information related to a place; and

“

Hosted Services”

to use a browser to execute functions, including “

Upload File”

to let the user to upload a file, “

URL to input”

to let the user to input an URL, and “

View on Browser”

to let the user view a file.

43. The computer-based system of claim 11, wherein the graphical user interface for managing the computing source of functionality as an available service through the associated SSD is capable of any one or combination of the following:

making the SSD available through one or more of the discovery mechanisms,

changing the SSD discovery mechanism to use,

holding the SSD,

recovering the SSD to one of the discovery mechanisms,

removing the SSD from discovery mechanisms, and

changing the SSD publishing parameters including expiration, invocation limit, and access control.

44. The computer-based system of claim 43, wherein the graphical user interface comprises any one or a combination of:

a Pervasive Instance Provision Environment (“

PIPE”

) to provide local or remote APIs to publish and optionally manage services, and optionally provide the user with user interfaces to publish and manage the services;

a “

Semantic Instance Scrapers for Applications”

to publish available instances and to optionally enable the user to manage the service provisions based upon the publication;

an “

Instance Selector with Instance Providing Service Creation”

to enable the user publish an instance the user selects for the instance selector service and optionally lets the user manage the service provisions;

a publisher (“

White hole”

) to publish objects as semantic instances through its user interface and optionally lets the user manage the service provisions;

a “

Public Directory”

to publish instances on a device and optionally lets the user manage the service provisions;

a “

Bank”

to publish instances through its service invocation and optionally lets the user manage the service provisions;

an “

Image/Audio/Video Service Control”

to publish instances created by or from the devices and optionally lets the user manage the service provisions;

a “

Removable Media Publisher”

to publish instances on the removable media inserted into a computing device and optionally lets the user manage the service provisions; and

a “

Fax”

to publish the documents received and optionally lets the user manage the service provisions.

45. A computer implemented method comprising:

using a computer enabling a user to compose a task that comprises a plurality of different types of computing sources of functionality, each computing source of functionality being a service to the user, by;

associating each service with a semantic service description (SSD), which is based upon one or more ontologies for filtering, composing and executing the service, and is discoverable as an available service according to one or more discovery protocols, wherein an SSD further comprises;

a semantic description of the service, including a semantic description of input/output parameters of the service as semantic input/output parameters, based upon the one or more ontologies,

a semantic filter parameter specifying relevance of the service, based upon the one or more ontologies, and

a grounding including;

a service invocation interface to the service; and

an input/output parameter mapping between the semantic input/output parameters and syntactic input/output interface parameters of the service and/or an input/output parameter transformation function between the semantic input/output parameters and syntactic input/output interface parameters of the service;

dynamically discovering any available SSDs as available services through the one or more discovery protocols to discover the SSDs;

supporting interfacing with the user for composition of a current executable task that is a combination of two or more of the available services by dynamically presenting to the user possible services by filtering the discovered available services according to a context of the user in a computing environment for a service selection from the available services and/or according to the current composed executable task, based upon the semantic filter parameters in the SSDs of the discovered available services; and

executing a task by invoking the two or more available services that comprise the task, including enabling the user to interact with an invoked available service, based upon the grounding in the associated SSDs including the service invocation interface, and the input/output mapping and/or the transformation function between the semantic input/output parameters and the syntactic input/output parameters.

46. The computer-based method of claim 45, wherein the computing sources of functionality originate in devices, computing applications, electronic services available and/or previously defined tasks.

47. The computer-based method of claim 45, wherein the SSD is expressed in a service description language.

48. The computer-based method of claim 45 wherein:

each SSD is provided by any combination of a creator or owner of the computing

sources of functionality or some other third party, and

making available or unavailable the service by making available or unavailable discovery of one or more of SSDs associated with the service.

49. The computer-based method of claim 45, further comprising saving composed executable tasks automatically, or as instructed by the user, as new available services.

50. The computer-based method of claim 49, further comprising making available the saved composed executable tasks to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

51. The computer-based method of claim 45, wherein the one or more discovery protocols include any one or any combination of the following:

UPnP discovery, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR) discovery.

52. The computer-based method of claim 45, wherein the filtering of the discovered available services according to a context of the user include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of the available services when considered as a whole, or individual available service features.

53. The computer-based method of claim 45, wherein the real-time composing by the user of the executable task includes any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile.

54. The computer-based method of claim 45, wherein the SSD service invocation interface is according to one or more of a remote procedure call including Web Services, SOAP, UPnP actions, JINI, CORBA, RMI, RPC, DCE, DCOM, KQML, FIPA-ACL, or InfraRed (IR), or local function call, or local object call, or code described directly in the SSD.

55. The computer-based method of claim 54, further comprising managing a computing source of functionality as an available service through the associated SSD by any one or any combination of the following:

a graphical user interface controlling creation, provision, holding and/or removal of the associated SSD, executing or interacting with the computing source of functionality, or processing events from an operating system of a user-operated device, and applications executing on the user-operated device.

65. The computer-readable medium of claim 45, wherein the SSD service invocation interface is according to one or more of a remote procedure call including Web Services , SOAP, UPnP actions, JINI, CORBA, RMI, RPC, DCE, DCOM, KQML, FIPA-ACL, or InfraRed (IR), or local function call, or local object call, or code described directly in the SSD.

56. A non-transitory computer-readable medium storing a program which when executed by a computer causes the computer to execute functions comprising:

enabling a user to compose a task that comprises a plurality of different types of computing sources of functionality, each computing source of functionality being a service to the user, by;

associating each service with a semantic service description (SSD), which is based upon one or more ontologies for filtering, composing and executing the service, and is discoverable as an available service according to one or more discovery protocols, wherein an SSD further comprises;

a semantic description of the service, including a semantic description of input/output parameters of the service as semantic input/output parameters, based upon the one or more ontologies,

a filter parameter specifying relevance of the service, based upon the one or more ontologies, and

a grounding including;

a service invocation interface to the service; and

an input/output parameter mapping between the semantic input/output parameters and syntactic input/output interface parameters of the service and/or an input/output parameter transformation function between the semantic input/output parameters and syntactic input/output interface parameters of the service;

dynamically discovering any available SSDs as available services through the one or more discovery protocols to discover the SSDs;

interfacing with the user for composing a current executable task that is a combination of two or more of the available services by dynamically presenting to the user possible services by filtering the discovered available services according to a context of the user in a computing environment for a service selection from the available services and/or according to the current composed executable task, based upon the semantic filter parameters in the SSDs of the discovered available services; and

executing a task by invoking the two or more available services that comprise the task, including enabling the user to interact with an invoked available service, based upon the grounding in the associated SSDs including the service invocation interface, and the input/output mapping and/or the transformation function between the semantic input/output parameters and the syntactic input/output parameters.

57. The computer-readable medium of claim 56, wherein the computing sources of functionality originate in devices, computing applications, electronic services available and/or previously defined tasks.

58. The computer-readable medium of claim 56, wherein the SSD is expressed in a service description language.

59. The computer-readable medium of claim 56, wherein:

each SSD is provided by any combination of a creator or owner of the computing

sources of functionality or some other third party, and

making available or unavailable the service by making available or unavailable discovery of one or more of SSDs associated with the service.

60. The computer-readable medium of claim 56, further comprising saving composed executable tasks automatically, or as instructed by the user, as new available services.

61. The computer-readable medium of claim 60, further comprising making available the saved composed executable tasks to the user who created them, to all other users, or to any user-defined, or pre-defined group of users.

62. The computer-readable medium of claim 56, wherein the one or more discovery protocols include any one or any combination of the following:

UPnP discovery, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed (IR) discovery.

63. The computer-readable medium of claim 56, wherein the filtering of the discovered available services according to a context of the user include any one or any combination of the following:

User profile, Task at hand, User device characteristics, User location, User motion status, User network connectivity, User specified keywords, features of a set of the available services when considered as a whole, or individual available service features.

64. The computer-readable medium of claim 56, wherein the real-time composing by the user of the executable task includes any one or any combination of the following:

a planning-based, automated system, or

an interactive user interface that supports any one or any combination of the following elements;

visual, voice, text, Braille, tactile.

66. The computer-readable medium of claim 56, further comprising managing a computing source of functionality as an available service through the associated SSD by any one or any combination of the following:

a graphical user interface controlling creation, provision, holding and/or removal of the associated SSD, executing or interacting with the computing source of functionality, or processing events from an operating system of a user-operated device, and applications executing on the user-operated device.

67. A method of providing a computer system capable of communication with a plurality of different types of computing sources of functionality, each computing source of functionality being a service to a user, comprising:

configuring one or more computing devices to;

have each service associated with a semantic service description (SSD), which is based upon one or more ontologies and discoverable as an available service according to one or more discovery protocols, each SSD including;

a semantic description of the service, based upon the one or more ontologies,

a semantic filter parameter specifying relevance of the service, based upon the one or more ontologies; and

a grounding including;

a service invocation interface to the service; and

an input/output parameter mapping between the semantic input/output parameters and syntactic input/output interface parameters of the service and/or an input/output parameter transformation function between the semantic input/output parameters and syntactic input/output interface parameters of the service;

dynamically discover any available SSDs as available services through the one or more discovery protocols to discover the SSDs; and

support composition by a user of a current executable task that is a combination of two or more of the available services by dynamically presenting to the user possible services by filtering of the discovered available services according to a context of the user in a computing environment for a service selection from the available services and/or according to the current composed executable task, based upon the semantic filter parameters in the SSDs of the discovered available services, the task is executable by invoking the two or more available services that comprise the task, including enabling the user to interact with an invoked available service, based upon the grounding in the associated SSDs including the service invocation interface, and the input/output mapping and/or the transformation function between the semantic input/output parameters and syntactic input/output interface parameters of the service.