Gesture controlled adaptive projected information handling system input and output devices

• US 10,228,848 B2
• Filed: 02/26/2016
• Issued: 03/12/2019
• Est. Priority Date: 03/21/2014
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Zagorin Cave, LLP
• Inventors: Walline, Erin K., Krishnakumar, Karthikeyan, Ligameri, Mark R., Ancona, Rocco, Knoppert, Michiel Sebastiaan Emanuel Petrus
• Primary Examiner(s): Haley, Joseph R
• Application Number:
  US15/054,793
• Publication Number:
  US 20160196060A1
• Time in Patent Office: 1,110 Days
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Projected input and output devices adapt to a desktop environment by sensing objects at the desktop environment and altering projected light in response to the sensed objects. For instance, projection of input and output devices is altered to limit illumination against and end user'"'"'s hands or other objects disposed at a projection surface. End user hand positions and motions are detected to provide gesture support for adapting a projection work space, and configurations of projected devices are stored so that an end user can rapidly recreate a projected desktop. A projector scan adjusts to limit traces across inactive portions of the display surface and to increase traces at predetermined areas, such as video windows.

Claims

1. A method for adapting a projected input device in response to gestures, the method comprising:

projecting a user interface at a projection surface with a second illumination;

projecting an image of an input device at the projection surface in response to an end user gesture, the input device presented with a first illumination and configured to accept end user touches at the projection surface as inputs;

detecting a predetermined gesture by an end user hand movement, the gesture proximate the projected input device, the gesture including touches at the input device as inputs to the input device; and

in response to the detecting, adapting the input device image according to a definition of the predetermined gesture including at least enabling the input device presented with the second illumination, the first illumination subdued relative to the second illumination;

wherein the input device comprises a keyboard that accepts typed inputs and the predetermined gesture comprises typed inputs at the keyboard.

2. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises a placement of fingers on the input device and dragging of the fingers along the projection surface; and

adapting the input device image further comprises moving the input device in the direction of the dragging.

3. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises placing fingers on the projection surface in a typing position; and

adapting the input device image further comprises moving the input device to align predetermined typing keys of the input device to predetermined of the fingers.

4. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises pointing a finger at an object disposed proximate the projection surface; and

adapting the input device image further comprises populating the input device with information associated with the object.

5. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises swiping the hand across the input device image; and

adapting the input device image further comprises changing the input device from a first configuration to a second configuration.

6. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises detecting identification information input by the end user by the gesture; and

adapting the input device image further comprises presenting plural output devices enabled to interact with the input device based upon the identification information.

7. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises detecting a rotational motion in a plane unaligned with a plane of the projection surface; and

adapting the input device image further comprises rotating the input device image in the projection surface plane.

8. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises a touch by an end user on the input device image, the input device image having subdued illumination, the input device image providing access to secured information; and

adapting the input device further comprises providing a password interface to access the secured information, the input device having full illumination at input of a password.

9. The method of claim 1 wherein:

detecting a predetermined gesture by an end user hand movement further comprises detecting a unique feature associated with the end user hand; and

adapting the input device further comprises projecting a tool bar associated with the unique feature.

10. A system adapting a projected input device in response to gestures, the system comprising:

a projector operable to present input devices as images projected at a projection surface;

an input sensor interfaced with the projector and operable to detect inputs made at the input devices; and

a gesture module stored in non-transitory memory and interfaced with the projector and the input sensor, the gesture module operable to adapt a projected input device in response to an end user gesture detected by the input sensor, the gesture module presenting the projected input device with a first illumination in response to the gesture and presenting the projected input device with a second illumination upon detection of an end user input interaction with the projected input device, the first illumination subdued relative to the second illumination;

wherein the input device comprises a keyboard, the first gesture comprises hands placed on the projection surface in a typing position, the keyboard is presented against a user interface having the second illumination, and the detection of end user input interaction results in keyboard presentation at the second illumination of the user interface.

11. The system of claim 10 wherein the keyboard in response to the end user interaction further presents a tool bar having tools selectable by the end user with a touch to the selected tool.

12. The system of claim 10 wherein the keyboard in response to the end user interaction further presents a typing window that presents inputs made at keys of the keyboard.

13. A method for presenting a projected keyboard by a projector at a projection surface, the method comprising:

projecting a user interface at a projection surface with a projector at a second illumination;

sensing with a camera an end user placement of hands on the projection surface in a typing configuration;

in response to the sensing the end user placement of hands, presenting the projected keyboard on the projection surface with a first illumination, the first illumination subdued relative to the second illumination;

sensing with the camera an end user typed input at the keyboard; and

in response to sensing the end user typed input at the keyboard, presenting the keyboard with the second illumination.