Virtual input tools

• US 8,427,438 B2
• Filed: 03/26/2009
• Issued: 04/23/2013
• Est. Priority Date: 03/26/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Louch, John O.
• Primary Examiner(s): Haley, Joseph
• Assistant Examiner(s): LEE, NICHOLAS J
• Application Number:
  US12/412,218
• Publication Number:
  US 20100245260A1
• Time in Patent Office: 1,489 Days
• Field of Search: 345/173
• US Class Current: 345/173
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A virtual input device, e.g., a virtual representation of a physical input device, is disclosed. In one aspect, virtual coordinates of the virtual input device correlate to real coordinates on the physical input device. Dimensions of the physical input device are proportional to dimensions of the virtual input device, and interactive objects are presented in the virtual input device.

Claims

1. A computer-implemented method comprising:

identifying an input device that is coupled to a processing device;

identifying real coordinates of positions on the input device that receive user input;

generating a virtual representation of the input device;

correlating the real coordinates to corresponding virtual coordinates in the virtual representation of the input device using anamorphic scaling such that regions closer to edges of the virtual representation of the input device are configured to be less sensitive to physical input;

presenting the virtual representation of the input device in an interface that is coupled to the processing device;

identifying one or more applications that are currently under execution in the processing device;

generating virtual representations of each of the one or more applications, wherein the virtual representations of the one or more applications include visual representations of corresponding interface windows associated with the one or more applications that are displayed in the interface coupled to the processing device; and

automatically presenting the virtual representations of each of the one or more applications in the virtual representation of the input device that is displayed in the interface coupled to the processing device, wherein the virtual representations of each of the one or more applications are presented simultaneously in the virtual representation of the input device.

2. The method of claim 1, comprising:

identifying one or more objects for display in the interface;

generating virtual representations of each of the one or more objects; and

presenting the virtual representations of each of the one or more objects in the virtual representation of the input device.

3. The method of claim 2, comprising:

receiving a first input through the input device;

identifying first real coordinates on the input device that identify a position on the input device where the first input was received;

mapping the first real coordinates to corresponding first virtual coordinates in the virtual representation of the input device; and

modifying the virtual representation of the input device or a virtual representation of a first object, based on the first virtual coordinates.

4. The method of claim 3, where the modifying includes indicating a selection of a virtual representation of a second object by emphasizing the virtual representation of the second object in the virtual representation of the input device.

5. The method of claim 3, where the modifying includes emphasizing the virtual representation of the first object in the virtual representation of the input device, wherein the emphasizing indicates selection of the virtual representation of the first object.

6. The method of claim 2, comprising:

for each of the one or more objects, generating text related to each object; and

combining the text and the virtual representations of each of the one or more objects in the virtual representation of the input device, for display in the interface.

7. The method of claim 2, comprising:

receiving a user input selecting a first virtual representation that is presented in a two-dimensional representation; and

responsive to receiving the user input, animating the first virtual representation such that the first virtual representation is presented as a three-dimensional representation.

8. The method of claim 1, where the virtual representation of the input device is presented in the interface in response to a first predetermined user input.

9. The method of claim 1, where the virtual representation of the input device is removed from display in the interface in response to a second predetermined user input.

10. The method of claim 1, where the virtual representation of the input device includes a control object, and wherein the virtual representation of the input device is removed from display in the interface in response to a user input selecting the control object.

29. The method of claim 1, wherein the virtual representations of the one or more applications include different types of visual representations of different types of content.

31. The method of claim 1, wherein automatically presenting the virtual representations of each of the one or more applications in the virtual representation of the input device comprises:

displaying, in the interface coupled to the processing device, the virtual representations of each of the one or more applications included in the virtual representation of the input device independent of receiving a user input.

11. A system comprising:

an input device;

a data processing apparatus; and

a data store, in which is stored virtual representation of the input device, wherein dimensions of the input device are proportional to dimensions of the virtual representation of the input device, and

real coordinates of the input device are correlated to virtual coordinates in the virtual representation of the input device using anamorphic scaling such that regions closer to edges of the virtual representation of the input device are configured to be less sensitive to physical input, and

wherein the virtual representation of the input device is displayed in an interface coupled to the data processing apparatus presenting simultaneously virtual representations of one or more applications that are currently under execution in the data processing apparatus, the virtual representations of the one or more applications including visual representations of corresponding interface windows associated with the one or more applications that are displayed in the interface coupled to the data processing apparatus.

12. The system of claim 11, wherein the virtual representation of the input device includes virtual representations of one or more interactive objects.

13. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the processor to perform operations comprising:

identifying an input device that is coupled to the processor;

identifying real coordinates of positions on the input device that receive user input;

generating a virtual representation of the input device;

correlating the real coordinates to corresponding virtual coordinates in the virtual representation of the input device using anamorphic scaling such that regions closer to edges of the virtual representation of the input device are configured to be less sensitive to physical input;

presenting the virtual representation of the input device in an interface that is coupled to the processor;

identifying one or more applications that are currently under execution in the processor;

generating virtual representations of each of the one or more applications, wherein the virtual representations of the one or more applications include visual representations of corresponding interface windows associated with the one or more applications that are displayed in a background of the interface coupled to the processing device; and

automatically presenting the virtual representations of each of the one or more applications in the virtual representation of the input device that is displayed in the interface coupled to the processor, wherein the virtual representations of each of the one or more applications are presented simultaneously in the virtual representation of the input device.

14. The non-transitory computer-readable medium of claim 13, comprising instructions to cause the processor to perform operations including:

identifying one or more objects for display in the interface;

generating virtual representations of each of the one or more objects; and

presenting the virtual representations of each of the one or more objects in the virtual representation of the input device.

15. The non-transitory computer-readable medium of claim 14, comprising instructions to cause the processor to perform operations including:

receiving a first input through the input device;

identifying first real coordinates on the input device that identify a position on the input device where the first input was received;

mapping the first real coordinates to corresponding first virtual coordinates in the virtual representation of the input device; and

modifying the virtual representation of the input device or a virtual representation of a first object, based on the first virtual coordinates.

16. The non-transitory computer-readable medium of claim 15, where the modifying includes indicating a selection of a virtual representation of a second object by emphasizing the virtual representation of the second object in the virtual representation of the input device.

17. The non-transitory computer-readable medium of claim 15, where the modifying includes emphasizing the virtual representation of the first object in the virtual representation of the input device, wherein the emphasizing indicates selection of the virtual representation of the first object.

18. The non-transitory computer-readable medium of claim 14, comprising instructions to cause the processor to perform operations including:

for each of the one or more objects, generating text related to each object; and

combining the text and the virtual representations of each of the one or more objects in the virtual representation of the input device, for display in the interface.

19. The non-transitory computer-readable medium of claim 14, comprising instructions to cause the processor to perform operations including:

receiving a user input selecting a first virtual representation that is presented in a two-dimensional representation; and

responsive to receiving the user input, animating the first virtual representation such that the first virtual representation is presented as a three-dimensional representation.

20. The non-transitory computer-readable medium of claim 13, where the virtual representation of the input device is presented in the interface in response to a first predetermined user input.

21. The non-transitory computer-readable medium of claim 13, where the virtual representation of the input device is removed from display in the interface in response to a second predetermined user input.

22. The non-transitory computer-readable medium of claim 13, where the virtual representation of the input device includes a control object, and wherein the instructions cause the processor to perform operations including removing the virtual representation of the input device from display in the interface in response to a user input selecting the control object.

30. The non-transitory computer-readable medium of claim 13, wherein the virtual representations of the one or more applications include different types of visual representations of different types of content.

32. The non-transitory computer readable medium of claim 13, wherein the instructions that cause the processor to perform operations comprising automatically presenting the virtual representations of each of the one or more applications in the virtual representation of the input device comprises instructions that cause the processor to perform operations comprising:

displaying, in the interface coupled to the processor, the virtual representations of each of the one or more applications included in the virtual representation of the input device independent of receiving a user input.

23. A computer-implemented method comprising:

generating a virtual representation of an input device that is coupled to a processing device, the virtual representation of the input device having dimensions that are proportional to dimensions of the input device according to a predetermined ratio;

identifying real coordinates of positions on the input device that receive user input;

identifying virtual coordinates of corresponding positions on the virtual representation of the input device based on the predetermined ratio;

correlating the real coordinates to the virtual coordinates in the virtual representation of the input device using anamorphic scaling such that regions closer to edges of the virtual representation of the input device are configured to be less sensitive to physical input;

presenting the virtual representation of the input device in an interface that is coupled to the processing device;

identifying one or more applications that are currently under execution in the processing device;

generating virtual representations of each of the one or more applications, wherein the virtual representations of the one or more applications include visual representations of corresponding interface windows associated with the one or more applications that are displayed in a background of the interface coupled to the processing device; and

presenting simultaneously, independent of receiving a user input, the virtual representations of each of the one or more applications in the virtual representation of the input device that is displayed in the interface coupled to the processing device.

24. A computer-implemented method comprising:

receiving input through an input device that is coupled to a processing device;

identifying real coordinates of positions on the input device that received the input;

mapping the real coordinates to virtual coordinates of corresponding positions on a virtual representation of the input device displayed in an interface that is coupled to the processing device, wherein the mapping includes using anamorphic scaling such that regions closer to edges of the virtual representation of the input device are configured to be less sensitive to physical input;

identifying a virtual object displayed in the virtual representation of the input device that corresponds to the virtual coordinates; and

activating an object that corresponds to the virtual object for display in the interface, wherein the virtual object includes a visual representation of a corresponding interface window associated with the object that is simultaneously displayed in the interface that is coupled to the processing device independent of receiving user input.

25. The method of claim 24, wherein the virtual object includes a portion of a document and activating the object for display includes displaying the portion of the document in the interface.

26. The method of claim 24, wherein the object is a file directory and activating the object for display includes displaying contents of the file directory in the interface.

27. The method of claim 24, wherein the object is a software application and activating the object for display includes displaying one or more application windows of the software application in the interface.

28. The method of claim 27, wherein virtual representations of the one or more application windows are displayed in the virtual representation of the input device.