3D manipulation using applied pressure

• US 8,493,384 B1
• Filed: 04/01/2010
• Issued: 07/23/2013
• Est. Priority Date: 04/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Reisman, Jason L., Davidson, Philip L., Han, Jefferson Y.
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US12/753,095
• Time in Patent Office: 1,209 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Placement by one or more input mechanisms of a touch point on a multi-touch display device that is displaying a three-dimensional object is detected. A two-dimensional location of the touch point on the multi-touch display device is determined, and the touch point is matched with a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point. A change in applied pressure at the touch point is detected, and a target depth value for the contact point is determined based on the change in applied pressure. A solver is used to calculate a three-dimensional transformation of the three-dimensional object using an algorithm that reduces a difference between a depth value of the contact point after object transformation and the target depth value.

Claims

1. A computer-implemented method of manipulating a three-dimensional object displayed in a multi-touch display device, the method comprising:

displaying a three-dimensional object in two dimensions on the multi-touch display device by projecting the three-dimensional object onto an image plane of a camera, the three-dimensional object having an initial three-dimensional location and an initial three-dimensional rotational orientation;

detecting touching by one or more input mechanisms of one or more touch points on the multi-touch display device;

determining, for each touch point, a two-dimensional location of the touch point on the multi-touch display device;

determining, for each touch point, a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point;

detecting a change in applied pressure at a particular one of the touch points based on an application of pressure by the one or more input mechanisms at the particular touch point;

accessing a current depth value of a particular one of the three-dimensional contact points on the surface of the object, the particular contact point being projected for display onto the image plane of the camera at the location of the particular touch point and the current depth value of the particular contact point representing a depth of the particular contact point along a Z-axis of the camera away from the image plane;

determining a target depth value for the particular contact point based on the current depth value and the detected change in applied pressure at the particular touch point;

using a solver to calculate a three-dimensional transformation of the three-dimensional object that specifies at least one of a new three-dimensional rotation and a new three-dimensional location for the three-dimensional object, the three-dimensional transformation being calculated by the solver using an algorithm that reduces deviation between projected two-dimensional locations of the three-dimensional contact points after object transformation and two dimensional locations of their respective touch points while additionally orienting the three-dimensional object to minimize a difference between a depth value of the particular contact point after object transformation and the target depth value;

transforming the three-dimensional object using the three-dimensional transformation such that the transformed three-dimensional object is positioned and rotated in accordance with the at least one of the new three-dimensional location and the new three-dimensional rotation; and

displaying the transformed three-dimensional object on the multi-touch display device by projecting the transformed three-dimensional object onto the image plane of the camera.

2. The method of claim 1, wherein determining a target depth value for the contact point comprises:

accessing a current depth value of the contact point, the current depth value representing a depth of the contact point along a Z-axis of the camera away from the image plane; and

determining a target depth value for the contact point based on the current depth value and the detected change in applied pressure at the touch point.

3. A computer-implemented method of manipulating a three-dimensional object displayed in a multi-touch display device, the method comprising:

displaying a three-dimensional object in two dimensions on the multi-touch display device by projecting the three-dimensional object onto an image plane of a camera, the three-dimensional object having an initial three-dimensional location and an initial three-dimensional rotational orientation;

detecting touching by one or more input mechanisms of a touch point on the multi-touch display device;

determining a two-dimensional location of the touch point on the multi-touch display device;

determining a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point;

detecting a change in applied pressure at the touch point based on an application of pressure by the one or more input mechanisms at the touch point;

determining a target depth value for the contact point based on the detected change in applied pressure at the touch point;

using a solver to calculate a three-dimensional transformation of the three-dimensional object that specifies at least one of a new three-dimensional rotation and a new three-dimensional location for the three-dimensional object, the three-dimensional transformation being calculated by the solver using an algorithm that reduces a difference between a depth value of the contact point after object transformation and the target depth value;

transforming the three-dimensional object using the three-dimensional transformation such that the transformed three-dimensional object is positioned and rotated in accordance with the at least one of the new three-dimensional location and the new three-dimensional rotation; and

displaying the transformed three-dimensional object on the multi-touch display device by projecting the transformed three-dimensional object onto the image plane of the camera.