VISUAL TOOLKIT FOR A VIRTUAL USER INTERFACE

• US 20070220437A1
• Filed: 03/07/2007
• Published: 09/20/2007
• Est. Priority Date: 03/15/2006
• Status: Active Grant

• Current Assignee: Navisense
• Original Assignee: Navisense
• Inventors: Boillot, Marc
• Granted Patent: US 8,578,282 B2
• Time in Patent Office: Days
• US Class Current: 715/762
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

An Integrated Development Environment (IDE) (100) for creating a touchless Virtual User Interface (VUI) 120 is provided. The IDE can include a development window (152) for graphically presenting a visual layout of user interface (UI) components (161) that respond to touchless sensory events in a virtual layout of virtual components (261), and at least one descriptor (121) for modifying a touchless sensory attribute of a user component. The touchless sensory attribute describes how a user component responds to a touchless touchless sensory event on a virtual component.

Claims

1. An Integrated Development Environment (IDE), executing computer instructions in a computer-readable storage medium of a computer system, suitable for use in developing a Virtual User Interface (VUI) application, the IDE comprising:

a development window for graphically constructing a visual layout of a User Inteface (UI) to correspond to a virtual layout of a Virtual User Interface (VUI); and

at least one descriptor that identifies a response of a user interface component in the UI to touchless sensory events applied to a corresponding virtual component in the VUI.

2. The IDE of claim 1, further comprising a component palette having user components that are dragged and dropped in the UI to create Virtual components in the VUI.

3. The IDE of claim 1, further comprising a dialog window that presents at least one modifiable touchless sensory attribute of a touchless sensory event associated with a virtual component.

4. The IDE of claim 1, wherein a touchless sensory attribute comprises at least one of an absolute location of a finger, a duration of time of a finger at a location, a relative displacement of a finger, a velocity of a finger, and an acceleration of a finger in the VUI.

5. The IDE of claim 1, wherein a touchless sensory event comprises at least one of a touchless finger positioning, a touchless finger push, a touchless finger release, a touchless finger pause, a touchless finger sign, a touchless scroll, and a touchless select.

6. The IDE of claim 1, wherein the computer system receives touchless sensory events from a touchless sensing unit producing a touchless sensing field, wherein the touchless sensing unit generates the touchless sensory events in response to touchless finger actions in the VUI.

7. The IDE of claim 1, wherein a touchless sensory event on a virtual component produces one of audio or visual feedback on a corresponding UI component in the display window.

8. The IDE of claim 2, further comprising a form sheet, wherein said touchless sensing unit visually projects the virtual layout containing the Virtual components on the form sheet.

9. The IDE of claim 1, further comprising a compiler for compiling a source code of the VUI application into at least one code object;

a linker for converting said code object into relocatable code; and

a code builder for building said relocatable code into an executable code object.

10. The IDE of claim 9, further comprising a

a flashing module for loading the executable code object into a touchless sensing unit.

11. The IDE of claim 2, further comprising at least one of:

a cable for connecting said IDE to a touchless sensing unit; and

a wireless device for connecting said IDE to a touchless sensing unit.

12. A visual toolkit, executing computer instructions in a computer-readable storage medium of a computer system, suitable for creating a VUI application comprising:

a development window that presents a visual layout of re-locatable user components in a User Interface (UI);

a touchless sensing unit, operatively coupled to the development window, that generates a touchless sensing field; and

at least one descriptor in the development window that describes how a user interface component of the UI responds to touchless sensory events applied to a corresponding virtual component in the VUI,

wherein an arranging of the re-locatable user components in the UI creates a virtual layout of virtual components in the touchless sensing field for producing the Virtual User Interface (VUI) application.

13. The visual toolkit of claim 12, wherein the visual toolkit generates an executable object of the VUI application that represents the visual layout in the UI.

15. The visual toolkit of claim 12, wherein the visual toolkit provides programming through C, C++, Java, .NET, or Visual Basic.

17. The visual toolkit of claim 17, wherein the visual toolkit implements a sensory Application Programming Interface (API) for providing portability across computing platforms, wherein the sensory API exposes methods, fields, event listeners, and event handlers for processing touchless finger actions in the VUI.

14. The visual toolkit of claim 17, wherein the executable object is one of an HTML object, an XML object, a Java Object, a C++ class structure, a .NET object, or a Java Servlet.

16. The visual toolkit of claim 17, further comprising a low-level driver that converts a touches sensory event on a virtual component in the VUI to an action on a user component in the UI.

18. A computer readable storage medium, having stored thereon a computer program having a plurality of code sections executable by a portable computing device for causing the portable computing device to perform the steps of:

configuring a virtual layout of virtual components;

modifying touchless sensory attributes of the virtual components, and creating a touchless Virtual User Interface (VUI) application in accordance with the configuring of the virtual components and the touchless sensory attributes.

19. The computer readable storage medium of claim 18, further comprising computer instructions for receiving a directive to arrange the virtual components in the touchless sensing field;

and

updating the configuration of the virtual layout in accordance with the directive to arrange.

20. The computer readable storage medium of claim 18, further comprising computer instructions for receiving a coordinate object from a touchless sensing device;

identifying a virtual component from the coordinate object; and

performing a user action in the Virtual User Interface (VUI) application according to the virtual component identified.