Method and arrangement for providing an expanded desktop

• US 7,103,841 B2
• Filed: 05/07/2002
• Issued: 09/05/2006
• Est. Priority Date: 05/08/2001
• Status: Expired due to Term

• Current Assignee: RPX Corporation
• Original Assignee: Nokia Corporation
• Inventors: Paihonen, Sami, Ronkainen, Sami
• Primary Examiner(s): Bayerl, Raymond J.
• Application Number:
  US10/140,599
• Publication Number:
  US 20020175933A1
• Time in Patent Office: 1,582 Days
• Field of Search: 715/762, 715/716, 715/727, 715/728, 715/729, 715/978, 715/781, 715/803, 715/864, 715/865, 715/778, 715/700, 715/717, 715/767, 715/764, 715/802, 715/848, 715/849, 715/863, 381/1, 381/17, 381/24, 340/825.19, 340/692, 340/825.24, 340/825.25, 340/384.1, 340/384.5, 340/384.7
• US Class Current: 715/727
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: Y10S 715/978 Audio interaction as part of an operator interface

Abstract

The invention relates to a method and arrangement for expanding a desktop. Especially the invention relates to a method and arrangement for expanding a desktop virtually. The objects of the invention are fulfilled by relating an audible sound to the application, window or other user interface element as it is shifted out of the display. The sound related to the user interface element may also be characteristic for the element it is related to. In addition, the element can be located by only listening to the sound related to the element.

Claims

1. A method for providing an expanded virtual desktop that includes at least one user interface element, comprising:

providing at least one area on the expanded virtual desktop that is not visually discernable by the user and that expands outside of all areas visible to a user in a screen;

locating the at least one user interface element outside all visible display areas visible to the user in the screen and within the expanded virtual desktop;

relating a determined sound to said at least one user interface element, and

locating said at least one user interface element in the expanded virtual expanded desktop on the basis of said determined sound related to said at least one user interface element located outside all visible display areas visible to the user in the screen and within the expanded virtual desktop.

2. A method according to claim 1, comprising that said sound related to the at least one user interface element located outside the visible display area is characteristic for the direction of the user interface element in the expanded virtual desktop.

3. A method according to claim 1, comprising that said sound related to the at least one user interface element located outside of the visible display area in a determined part of the expanded virtual desktop is characteristic for this part of the expanded virtual desktop.

4. A method according to claim 1, comprising that said sound related to the at least one user interface element located outside of the visible display area is characteristic for said user interface element.

5. A method according to claim 1, comprising that said sound is continuous.

6. A method according to claim 1, comprising that the determined sound related to the at least one user interface element located outside of the visible display area is provided when there is an announcement in said at least one user interface element.

7. A method according to claim 1, comprising that determined sounds related to at least two of the at least one user interface elements located outside of the visible display area are provided alternately.

8. A method according to claim 1, further comprising providing the determined sound related to the at least one user interface element located outside of the visible display area is started/stopped when the at least one user interface element is pointed to by using a pointing device.

9. A method according to claim 8, comprising that said pointing device is at least one of the following:

a mouse, 3D-mouse, arrow key, user'"'"'s arm, user'"'"'s finger and joystick.

10. A method according to claim 1, comprising that the determined sound related to the at least one user interface element located outside of the visible display area is activated and deactivated using a command.

11. A method according to claim 10, comprising that said command is at least one of the following:

a gesture, pointing and spoken command, mouse click and a pressing of a key and soft created button.

12. A method according to claim 1, comprising that the determined sound is provided relating to the at least one user interface element located outside of the visible display area, wherein said sound is provided as coming from the point in space in the expanded virtual desktop where the at least one user interface element is located.

13. A method according to claim 1, comprising that the at least one user interface element located outside of the visible display area is activated and brought back into the visible display area of the desktop and shifted outside of the visible display area of the desktop by a pointing device.

14. A method according to claim 13, comprising that said pointing device is at least one of the following:

a mouse, 3D-mouse, arrow key, user'"'"'s arm, user'"'"'s finger and joystick.

15. A method according to claim 1, further comprising that a desired user interface element of the at least one user interface element located outside the visible display area is activated by a determined command if a sound is provided relating to the desired user interface element.

16. A method according to claim 15, comprising that said command is at least one of the following:

a gesture, pointing and spoken command, mouse click and a pressing of a key and soft created button.

17. A method according to claim 1, comprising that the at least one user interface element located outside of the visible display area is activated and brought back into the visible display area of the desktop and shifted outside of the visible display area of the desktop by a command.

18. A method according to claim 17, comprising that said command is at least one of the following:

a gesture, pointing and spoken command, mouse click and a pressing of a key and soft created button.

19. An arrangement for providing an expanded virtual desktop that includes at least one user interface element, wherein the arrangement comprises:

means for providing at least one area on the expanded virtual desktop that is not visually discernable by the user and expands outside of all areas that are visible to the user in a screen;

means for providing the at least one user interface element outside all visible display areas that are visible to the user in the screen and within the expanded virtual desktop area that is not visually discernable by the user and expanding outside of all areas being visible to the user in the screen; and

means for providing a determined sound related to the at least one user interface element so that a localization of the at least one user interface element in the virtual expanned desktop is based on the determined sound related to the at least one user interface element located outside of all visible display areas that are visible to the user in the screen.

20. An arrangement according to claim 19, comprising that, wherein the at least one user interface element is located in a determined location in the expanded virtual desktop, a sound is characteristic for a direction of the location.

21. An arrangement according to claim 19, comprising that the at least one user interface element is located outside the visible display area and in a determined part of the expanded virtual desktop wherein, a sound is characteristic for the determined part of the expanded virtual desktop where the at least one user interface element is located.

22. An arrangement according to claim 19, comprising means for providing a characteristic sound for a specific user interface element of the at least one user interface element located outside of the visible display area.

23. An arrangement according to claim 19, comprising that said arrangement comprises means for providing the determined sound related to the at least one user interface element located outside the visible display area when there is an announcement in the at least one user interface element.

24. An arrangement according to claim 19, comprising that said arrangement comprises means for detecting a pointing device, and means for providing the determined sound related to the at least one user interface element located outside of the visible display area when the at least one user interface element is pointed to by a pointing device.

25. An arrangement according to claim 19, comprising that said arrangement comprises means for detecting a user'"'"'s arm or finger, and means for providing the determined sound related to the at least one user interface element located outside of the visible display area when the at least one user interface element is pointed by a user'"'"'s arm or finger.

26. An arrangement according to claim 19, comprising that said arrangement comprises means for detecting a spoken command, and means for providing the determined sound related to the at least one user interface element located outside of the visible display area when the user says a spoken command related to said the at least one user interface element.

27. An arrangement according to claim 19, comprising that said arrangement comprises at least one pointing device.

28. An arrangement according to claim 19, comprising that said arrangement comprises at least one of the following:

button, arrow key, software created button, mouse, 3D-mouse, speech recognition means, gesture recognition means and recognition means for observation of motions or location of user'"'"'s arm or finger to do at least one of the following;

activate the at least one user interface element, shift the at least one user interface element, activate a sound of the at least one user interface element, deactivate a sound of the at least one user interface element and point the at least one user interface element wherein, the at least one user interface element is located outside of the visible display area.

29. An arrangement according to claim 19, comprising that said arrangement comprises means for providing a 3D-sound related to the at least one user interface element located outside of the visible display area as coming from a point in virtual space within the expanded virtual desktop where the at least one user interface element is located.

30. An arrangement according to claim 29, comprising that said means for providing the 3D-sound comprise a sound card and stereo headphones or at least two speakers.

31. An arrangement according to claim 19, comprising that said arrangement comprises at least one microphone.

32. An arrangement according to claim 19, comprising that said arrangement comprises a display or screen wherein there is at least one touch sensitive area on the display or screen.

33. A data processing system comprising:

means for providing an expanded virtual desktop that includes at least one user interface element, the expanded virtual desktop comprising at least one area that is not visually discernable by the user and expands outside of all areas visible to the user in a screen;

means for providing the at least one user interface element located outside of all visible display areas that are visible to the user in the screen and within the expanded virtual desktop area that is not visually discernable by the user and expands outside of all areas that are visible to the user in the screen; and

means for providing a determined sound related to the at least one user interface element for localization of the at least one user interface element in the virtual expanded desktop based on the determined sound related to the at least one user interface element located outside all visible display areas visible to the user in the screen.

34. A computer-readable medium encoded with a computer program code for providing an expanded virtual desktop expanding outside a visible display area that includes at least one user interface element, the expanded virtual desktop comprising at least one area not being visually discernable by the user and expanding outside of all areas being visible to the user in a screen, the program code causing a computer to execute a method comprising:

locating the at least one user interface element outside all visible display areas being visible to the user in the screen and within the expanded virtual desktop area not being visually discernable by the user and expanding outside of all areas visible to the user in the screen;

relating a determined sound to the at least one user interface element;

wherein

the at least one user interface element is located in the virtual expanded desktop on the basis of the determined sound related to the at least one user interface element located outside all visible display areas visible to the user in the screen.