Intuitive control of portable data displays
First Claim
1. A computer implemented method for assisting a user in the control and operation of a computer system having a display screen coupled with the computer system, the computer implemented method comprising the steps of:
- equipping the user with a portable display device coupled to the computer system;
mapping visual feedback generated by the computer system and intended for display upon a display screen into a virtual desktop suitable for display by the portable display device;
continually displaying a certain portion of the virtual desktop within the portable display device such that the user can view the certain portion of the virtual desktop;
tracking motion of the portable display device including discrete motion gestures initiated by the user; and
when the tracked motion corresponds to a request for a special discrete command, performing the special discrete command.
8 Assignments
0 Petitions
Accused Products
Abstract
A virtual computer monitor is described which enables instantaneous and intuitive visual access to large amounts of visual data by providing the user with a large display projected virtually in front of the user. The user wears a head-mounted display or holds a portable display containing a head-tracker or other motion tracker, which together allow the user to position an instantaneous viewport provided by the display at any position within the large virtual display by turning to look in the desired direction. The instantaneous viewport further includes a mouse pointer, which may be positioned by turning the user'"'"'s head or moving the portable display, and which may be further positioned using a mouse or analogous control device. A particular advantage of the virtual computer monitor is intuitive access to enlarged computer output for visually-impaired individuals.
520 Citations
29 Claims
-
1. A computer implemented method for assisting a user in the control and operation of a computer system having a display screen coupled with the computer system, the computer implemented method comprising the steps of:
-
equipping the user with a portable display device coupled to the computer system;
mapping visual feedback generated by the computer system and intended for display upon a display screen into a virtual desktop suitable for display by the portable display device;
continually displaying a certain portion of the virtual desktop within the portable display device such that the user can view the certain portion of the virtual desktop;
tracking motion of the portable display device including discrete motion gestures initiated by the user; and
when the tracked motion corresponds to a request for a special discrete command, performing the special discrete command. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A computer implemented method for assisting a vision-impaired user in the control and operation of a computer system, the computer system being display driven in that a primary source of feedback for interacting with the computer system is visual feedback generated by the computer system, the visual feedback intended for display upon a display screen coupled with the computer system, the computer implemented method comprising the steps of:
-
equipping the vision-impaired user with a portable display device coupled to the computer system;
mapping the visual feedback generated by the computer system and intended for display upon a display screen into a virtual desktop suitable for display by the portable display device;
continually displaying a certain portion of the virtual desktop within the portable display device such that the vision-impaired user can view the certain portion of the virtual desktop, the certain portion of the virtual desktop corresponding to that portion of the virtual desktop which is located within the line of sight of the vision-impaired user;
adjusting a magnification of the virtual desktop in accordance with input from the visually-impaired user, whereby the certain portion is correspondingly adjusted such that the certain portion displayed within the portable device is a larger or a smaller percentage of the virtual desktop depending upon the vision-impaired user'"'"'s magnification adjustment, the adjust magnification input being a function of the portable display;
tracking changes in the vision-impaired user'"'"'s line of sight; and
updating the portable display device such that the certain portion displayed maintains the correspondence with that portion of the virtual desktop which is located within the line of sight of the vision-impaired user, whereby the visually-impaired user can adjust the magnification of the visual feedback provided by the computer system to a magnification level improving comprehension of the visual feedback, and, even when the magnification level precludes simultaneous display of all the visual feedback within the portable display device, the visually-impaired user may still access all the visual feedback by adjusting his or her line of sight. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
receiving a request to adjust the fixed relation between the pointer and the vision-impaired user'"'"'s line of sight; and
adjusting the fixed relation in accordance with the received request.
-
-
14. A computer implemented method as recited in claim 12 further comprising the step of providing the vision-impaired user a pointer control input device for controlling operation of the pointer, the pointer control input device coupled to the computer system.
-
15. A computer implemented method as recited in claim 14 further comprising the steps of:
-
receiving a create selection box command via the pointer control input device;
anchoring a first comer of a selection box at a first point where the pointer is disposed within the certain portion displayed upon receipt of the create selection box command;
sizing the selection box in conjunction with movement of the vision-impaired user'"'"'s line of sight;
receiving a close selection box command via the pointer control input device;
defining the selection box by anchoring a second comer of the selection box, the second comer opposing the first comer, the second comer anchored at a second point where the pointer is disposed within the certain portion displayed upon receipt of the close selection box command; and
marking as selected that portion of the virtual desktop, which is bounded by the selection box.
-
-
16. A computer implemented method as recited in claim 15 wherein the create selection box command corresponds to the depression of a button located on the pointer control input device and the close selection box command corresponds to the release of the button located on the pointer control input device.
-
17. A computer implemented method as recited in claim 11 further comprising the step of redefining the orientation of the certain portion displayed within the virtual desktop such that, without adjusting the vision-impaired user'"'"'s line of sight, the certain portion displayed within the head-mounted display device changes.
-
18. A computer implemented method as recited in claim 17 wherein the orientation of the certain portion displayed is redefined in response to a request by the vision-impaired user.
-
19. A computer implemented method as recited in claim 11 wherein the step of mapping the visual feedback into the virtual desktop includes the substep of performing spatial field compression to reduce the total display orientation change required for redirecting the vision-impaired user'"'"'s line of sight from one location within the virtual desktop to another location within the virtual desktop.
-
20. A computer implemented method as recited in claim 11 wherein the visual feedback generated by the computer system and intended for display upon a display screen includes multiple application windows.
-
21. A computer implemented method as recited in claim 20 wherein a first window of the multiple application windows corresponds to a first application executing upon the computer system.
-
22. A computer implemented method as recited in claim 20 further comprising the steps of:
-
monitoring a real scene in real space and time;
capturing images of the real scene; and
displaying within a first window of the multiple application windows the captured images of the real scene.
-
-
23. A computer implemented method as recited in claim 22 wherein a second window of the multiple application windows corresponds to an application program executing upon the computer system.
-
24. A computer implemented method for assisting a user in the control and operation of a computer system having a display screen coupled with the computer system, the computer implemented method comprising the steps of:
-
equipping the user with a portable display device coupled to the computer system;
equipping the user with a wearable control device operable to monitor movement and gestures made by the user with the portable control device, the computer system storing a map corresponding certain gestures and movements to certain computer operating instructions;
mapping visual feedback generated by the computer system and intended for display upon a display screen into a virtual desktop suitable for display by the portable display device;
continually displaying a certain portion of the virtual desktop within the portable display device such that the user can view the certain portion of the virtual desktop;
tracking motion of the portable control device;
determining that a user implemented gesture with the portable control device corresponds to a specific computer operating instruction; and
executing the specific computer operating instruction. - View Dependent Claims (25, 26, 27, 28, 29)
-
Specification