Continuous Calibration of an Information Handling System Projected User Interface

• US 20160316186A1
• Filed: 04/21/2015
• Published: 10/27/2016
• Est. Priority Date: 04/21/2015
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Krishnakumar, Karthik, Knoppert, Michiel Sebastiaan Emanuel Petrus, Ancona, Rocco, Sanaullah, Abu S., Ligameri, Mark R.
• Granted Patent: US 11,106,314 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/3215 Monitoring of peripheral devices
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0395 Mouse pads
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 9/3179 : Video signal processing therefor
• CPC Class Codes: H04N 9/3194 : including sensor feedback

Abstract

Visual images projected on a projection surface by a projector provide an interactive user interface having end user inputs detected by a detection device, such as a depth camera. The detection device monitors projected images initiated in response to user inputs to determine calibration deviations, such as by comparing the distance between where a user makes an input and where the input is projected. Calibration is performed to align the projected outputs and detected inputs. The calibration may include a coordinate system anchored by its origin to a physical reference point of the projection surface, such as a display mat or desktop edge.

Claims

1. An information handling system having a user interface presented at a projection surface, the information handling system comprising:

a processor operable to process information for presentation to a user;

memory interfaced with the processor and operable to store the information;

a graphics system interfaced with the processor and memory, the graphics system operable to generate pixel information to create visual images at one or more display devices;

a projector interfaced with the graphics system and operable to project the pixel information as the visual images at the projection surface;

a detection device interfaced with the graphics system and operable to detect end user inputs made at the projection surface; and

a calibration engine interfaced with the graphics system and detection device, the calibration engine operable to compare a location of an end user input detected by the detection device with a location of a visual image created by the end user input to determine a calibration deviation and to initiate calibration of the projector relative to the projection surface if the calibration deviation exceeds a threshold.

2. The system of claim 1 wherein the calibration engine initiates calibration with an infrared emitter and infrared camera to compare alignment of an infrared image with the projected visual images.

3. The system of claim 1 wherein the calibration engine initiates calibration by moving the projected visual images presented by the projector by the calibration deviation to the end user input location.

4. The system of claim 1 further comprising:

a display mat disposed on the desktop, the display mat interfaced with the graphics system and operable to display the visual images, the display mat including capacitive sensors to detect end user input locations at the display mat;

wherein the calibration engine is further operable to compare end user input locations detected by the display mat and end user input locations detected by the detection device to determine the calibration deviation.

5. The system of claim 4 further comprising:

a user interface engine interfaced with the detection device and operable to establish a coordinate system anchored to a physical reference detected by the detection device at the projection surface;

wherein the calibration engine initiates calibration of the projector relative to the projection surface to align projection with the coordinate system.

6. The system of claim 5 wherein the projection surface comprises a desktop and the physical reference comprises an increase in distance detected by the detection device along an edge of the desktop.

7. The system of claim 5 wherein the physical reference comprises a perimeter of the display mat.

8. The system of claim 5 wherein the physical reference comprises a predetermined location of the display mat indicated by illumination of a predetermined image at the display mat.

9. A method for calibrating an information handling system projected user interface and end user input locations, the method comprising:

capturing an image of a projection surface with a detection device;

analyzing the image to determine an end user input location;

projecting an output at an output location of the projection surface in response to the end user input;

capturing an image of the output with the detection device;

analyzing the image of the output to determine the output location; and

comparing the output location with the input location to determine calibration deviation.

10. The method of claim 9 wherein:

the end user input comprises writing on the projection surface;

the output comprises a projected image of the writing proximate the input location; and

the calibration deviation comprises difference between the location of the projected image of the writing and an intended location of the projected image of the writing.

11. The method of claim 10 further comprising:

determining that the calibration deviation exceeds a threshold; and

in response to the determining, initiating a calibration to align the input and output locations.

12. The method of claim 11 wherein initiating a calibration further comprises:

presenting an infrared image at the projection surface; and

capturing the infrared image with a camera to align the input and output locations.

13. The method of claim 9 further comprising:

capturing with the detection device a physical reference associated with the projection surface;

defining a projection area with Cartesian coordinates anchored at the physical reference; and

in response to the detecting the calibration deviation, calibrating the detection device and the projector to the Cartesian coordinates.

14. The method of claim 13 wherein the physical reference comprises a desktop edge and the detection device comprises a depth camera, the method further comprising:

detecting with the depth camera an increase in measured distance to define a position of the desktop edge;

defining the Cartesian coordinates from an origin located along the desktop edge; and

periodically monitoring for calibration deviation of projected images relative to desktop edge by reference to the Cartesian coordinates.

15. The method of claim 14 further comprising:

detecting from the image of the desktop surface a position of a display mat disposed on the desktop surface;

defining the position of the display mat relative to the projection area coordinates; and

calibrating the depth camera and projector by reference to the detected position of the display mat.

16. The method of claim 13 wherein the physical reference comprises a display mat disposed on the projection surface, the display mat operable to present images from the information handling system and accept touch inputs to provide the touch inputs to the information handling system, the method further comprising:

presenting a calibration image at a position on the display mat; and

in response to detection of the calibration image through a depth camera image, calibrating the depth camera and projector to calibration image position.

17. The method of claim 9 wherein the detection device comprises a depth camera, the end user input comprises writing motions by a pen at the projection surface, and the projected output comprises an image projected at the locations of the writing motions.

18. A system for calibrating an information handling system projected user interface and end user input locations comprising:

a projector operable to project images at a projection surface;

a depth camera operable to capture an image of the projection surface and to detect end user inputs made at the projection surface; and

a calibration engine interfaced with the projector and depth camera, the calibration engine operable to compare a location of an end user input detected by the depth camera with a location of a visual image projected by the projector in response to the end user input to determine a calibration deviation and to initiate calibration of the projector relative to the projection surface if the calibration deviation exceeds a threshold.

19. The system of claim 18 wherein the calibration engine calibrates the projector to Cartesian coordinates having an origin anchored to a display mat resting on the projection surface.

20. The system of claim 18 wherein the projection surface comprises a desktop and the calibration engine calibrates the projector to Cartesian coordinates having an origin anchored to a desktop edge detected by the depth camera.