Information handling system projected work space calibration

• US 9,720,446 B2
• Filed: 04/21/2015
• Issued: 08/01/2017
• Est. Priority Date: 04/21/2015
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Bui, Vinh X., Morrison, John T., Krishnakumar, Karthik, Sanaullah, Abu S.
• Primary Examiner(s): LAMB, CHRISTOPHER RAY
• Application Number:
  US14/691,766
• Publication Number:
  US 20160314727A1
• Time in Patent Office: 833 Days
• Field of Search: None
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Desktop surface references are selected and applied to define a coordinate system for calibrating projected visual images and end user inputs at the projected visual images. For example, a desktop edge is detected with the depth camera by the increase in detected distance along the axis from the depth camera to the desktop edge, and then the desktop edge is used as an origin for a coordinate system that defines a projection area for presenting a user interface. Monitoring end user inputs and projected outputs relative to the desktop edge aids in coordinating interactions by a user through the projected user interface in the event the camera or projector move relative to the desktop surface.

Claims

1. An information handling system having a user interface presented at a desktop surface, the information handling system comprising:

a processor operable to process information for presentation to a user;

memory interfaced with the processor and operable to store the information;

a graphics system interfaced with the processor and memory, the graphics system operable to generate pixel information to create visual images at one or more display devices;

a projector interfaced with the graphics system and operable to project the pixel information as the visual images at the desktop surface;

a display mat interfaced with the graphics system and operable to present the pixel information as visual images at the desktop surface; and

a user interface engine interfaced with the graphics system and detection device, the user interface engine operable to define a projection area as an arc passing through the display mat, the projector presenting the visual images in the projection area, the user interface engine rotating the user interface along the arc in response to a user input, the rotating transferring at least one visual image from presentation by the projector to presentation by the display mat.

2. The information handling system of claim 1 wherein the projection area is defined with Cartesian coordinates relative to an origin located at a desktop edge.

3. The information handling system of claim 2 wherein the defined projection area includes at least a projection size, aspect ratio, focus and keystone correction.

4. The information handling system of claim 2 wherein further comprising:

the display mat presents one or more windows of visual images by reference to Cartesian coordinates of the arc; and

the projector presents one or more windows of visual images along the arc projection area defined with polar coordinates relative to an end user position, the polar coordinates translated to the projection area Cartesian coordinates for presentation by the projector.

5. The information handling system of claim 1 further comprising:

a camera aligned to capture images of the projection area;

an input engine interfaced with the camera and operable to detect inputs made at the projection area and captured by an image of the camera, the input engine providing the inputs to the graphics system for presentation of the inputs with the visual images; and

a calibration engine interfaced with the input engine and the graphics system, the calibration engine operable to compare a location of the detected inputs and the location of projection of the detected inputs to correct the projection area for motion of the projector relative to the desktop.

6. The information handling system of claim 5 wherein the display mat further comprises a capacitive sensor surface operable to detect touches and determine touch positions, the calibration engine further operable to compare the location of the detected inputs with touch positions at the capacitive sensor surface to correct the projection area for motion of the projector relative to the desktop.

7. The information handling system of claim 1 wherein the detection device detects desktop constraints by detecting a change in distance between first and second vectors of greater than a predetermined amount.

8. The information handling system of claim 1 wherein the detection device determines a desk width by resolving the distance to the desktop edge and a height of the detection device over the desktop.

9. A method for calibrating an information handling system projected user interface to a desktop surface, the method comprising:

capturing an image of the desktop surface with a camera;

analyzing the image to identify a desktop edge;

defining projection area coordinates by reference to an origin located at the desktop edge;

presenting the projected user interface in the projection area with a projector aligned to present images at the desktop surface;

detecting from the image of the desktop surface a position of a display mat disposed on the desktop surface;

defining the position of the display mat relative to the projection area coordinates;

presenting the user interface with the display mat and with the projector, the projector presenting the user interface along an arc distal defined through the display mat according to the polar coordinates;

detecting a gesture proximate the desktop surface; and

in response to the gesture, rotating the user interface along the arc, the rotating transferring at least one visual image from presentation by the projector to presentation by the display mat.

10. The method of claim 9 further comprising applying the projection area to determine projection size, projection aspect ratio and projection keystone correction.

11. The method of claim 10 wherein defining projection area coordinates by reference to the desktop edge further comprises:

defining the projection area with Cartesian coordinates having positions relative to the desktop edge;

monitoring projected images relative to the Cartesian coordinates to detect projector movement relative to the desktop edge; and

in response to detecting desktop movement, adjusting the Cartesian coordinates to align with the projection area.

12. The method of claim 11 further comprising:

detecting a user input at the projection area with the camera at detected Cartesian coordinates; and

presenting a projected output associated with the user input by reference to the detected Cartesian coordinates.

13. The method of claim 12 further comprising:

detecting with the camera a positional deviation between the detected Cartesian coordinates and the presented projected output associated with the user input; and

in response to the detecting the positional deviation, calibrating the camera and the projector to the Cartesian coordinates.

14. The method of claim 9 further comprising:

detecting with the camera a user proximate the desktop edge;

determining user dimensions from an image of the user captured by the camera; and

including at least polar coordinates to define the projection area based upon the user dimensions.

15. The method of claim 9 further comprising:

orienting images presented at the display mat according to the projection area coordinates defined as Cartesian coordinates; and

orienting images presented by the projector according to the polar coordinates.

16. An information handling system comprising:

a processor operable to generate visual information for presentation to an end user;

memory interfaced with the processor and operable to store the visual information;

a depth camera interfaced with the process and operable to measure distances;

a graphics system interfaced with the processor and memory, the graphics system operable to process the visual information to generate visual images at one or more display devices;

a projector interfaced with the graphics system and operable to project visual images at a desktop surface; and

a user interface engine interfaced with the graphics system and depth camera, the user interface engine operable to detect one or more references at the desktop surface from an image taken by the depth camera and to apply the one or more references to define a coordinate system on the desktop surface relative to the one or more references, the one or more references including at least a display mat;

wherein the depth camera and projector reference the coordinate system for presenting user interface images at the desktop surface and detecting end user inputs at the desktop surface, the user interface engine presenting a user interface with the display mat and the projector, the projector presenting the user interface along an arc defined through the display mat, the user interface engine rotating the user interface along the arc in response to a user input, the rotating transferring at least one visual image from presentation by the projector to presentation by the display mat.

17. The information handling system of claim 16 wherein the one or more references comprise an edge of the desktop.