3D manipulation using applied pressure

• US 9,041,679 B2
• Filed: 02/18/2014
• Issued: 05/26/2015
• Est. Priority Date: 04/01/2009
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Reisman, Jason L., Davidson, Philip L., Han, Jefferson Y.
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US14/183,430
• Publication Number:
  US 20140168128A1
• Time in Patent Office: 462 Days
• Field of Search: 345/473, 345/420, 345/173
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Placement by one or more input mechanisms of a touch point on a multi-touch display device that is displaying a three-dimensional object is detected. A two-dimensional location of the touch point on the multi-touch display device is determined, and the touch point is matched with a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point. A change in applied pressure at the touch point is detected, and a target depth value for the contact point is determined based on the change in applied pressure. A solver is used to calculate a three-dimensional transformation of the three-dimensional object using an algorithm that reduces a difference between a depth value of the contact point after object transformation and the target depth value.

Claims

1. A computer-implemented method of manipulating a three-dimensional object displayed in a multi-touch display device, the method comprising:

displaying an initial view of a three-dimensional object on a two-dimensional touch display;

detecting a first touch, second touch, and third touch at respective touch points on the touch display that correspond to virtual contact points on the three-dimensional object displayed on the touch display;

detecting movement of the touch point for the first touch from an initial position to a final position on the touch display, while each of the second touch and third touch remain at respective initial positions; and

rendering, on the multi-touch display device, a new three-dimensional view of the three-dimensional object based on the movement of the first touch, wherein each contact point remains displayed by the multi-touch display device substantially underneath its corresponding touch point, the new three-dimensional view being scaled, rotated, and/or translated from the initial view of the three-dimensional object;

wherein a screen-space projection of each respective contact point is determined by projecting each contact point onto the touch display; and

wherein rendering the new three-dimensional view of the three-dimensional object comprises applying an algorithm to reduce distances between the final position of the first touch point and a first screen-space projection, the initial position of the second touch point and a second screen-space projection, and the initial position of the third touch point and a third screen-space projection.

2. The method of claim 1, wherein the second touch point and third touch point define an axis about which the new view of the three-dimensional object is rotated compared to the initial view of the three-dimensional object.

3. The method of claim 1, wherein detecting the first touch, second touch, and third touch at respective touch points comprises detecting only three touches at only three touch points;

and

wherein applying the algorithm to reduce the distances results in zero distances between each touch point and a respective screen-space projection when movement of the three-dimensional object is unconstrained.

4. The method of claim 1, wherein the algorithm comprises a least squares minimization algorithm.

5. The method of claim 4, wherein the least squares minimization algorithm comprises a Levenberg-Marquardt algorithm.

6. The method of claim 5, wherein applying the algorithm to reduce the Euclidean distances results in zero distances between each touch point and respective screen-space projection when the three-dimensional object is able to move with six degrees of freedom.

7. A computer-implemented method of manipulating a three-dimensional object displayed in a multi-touch display device, the method comprising:

displaying an initial view of a three-dimensional object on a two-dimensional touch display;

detecting multiple touches at respective touch points on the touch display that correspond to virtual contact points on the three-dimensional object displayed on the touch display;

detecting movement of one or more touch points from a respective initial position to a respective final position on the touch display, while any touch point remaining stationary remains at the respective initial position; and

rendering, on the multi-touch display device, a new three-dimensional view of the three-dimensional object based on the movement of the one or more touch points, wherein each contact point remains displayed by the multi-touch display device substantially underneath its corresponding touch point, the new three-dimensional view being scaled, rotated, and/or translated from the initial view of the three-dimensional object;

wherein a screen-space projection of each respective contact point is determined by projecting each contact point onto the touch display; and

wherein rendering the new three-dimensional view of the three-dimensional object comprises applying an algorithm to reduce distances between the one or more touch points and the respective screen-space projections.

8. The method of claim 7, wherein the algorithm comprises a least squares minimization algorithm.

9. The method of claim 8, wherein the least squares minimization algorithm comprises a Levenberg-Marquardt algorithm.

10. The method of claim 7, wherein detecting the multiple touches at respective touch points comprises detecting more than three touch points;

and

wherein applying the algorithm to reduce the distances results in distances greater than zero between each touch point and a respective screen-space projection, and the distances are distributed among the touch points in a uniform manner, when movement of the three-dimensional object is unconstrained.

11. The method of claim 7, wherein detecting the multiple touches at respective touch points comprises detecting more than three touch points;

and

wherein applying the algorithm to reduce the distances results in distances greater than zero between each touch point and a respective screen-space projection, and the distances are distributed among the touch points in a non-uniform manner, when movement of the three-dimensional object is unconstrained.

12. A multi-touch display device, comprising:

a processor;

a two-dimensional touch display; and

a program of instructions executable by the processor to manipulate a three-dimensional object displayed by the device, the program of instructions configured to;

display an initial view of the three-dimensional object on the touch display;

detect a first touch, second touch, and third touch at respective touch points on the touch display that correspond to virtual contact points on the three-dimensional object displayed on the touch display;

detect movement of the touch point for the first touch from an initial position to a final position on the touch display, while each of the second touch and third touch remain at respective initial positions; and

render a new three-dimensional view of the three-dimensional object based on the movement of the first touch, wherein each contact point remains displayed by the multi-touch display device substantially underneath its corresponding touch point, the new three-dimensional view being scaled, rotated, and/or translated from the initial view of the three-dimensional object;

wherein a screen-space projection of each respective contact point is determined by projecting each contact point onto the touch display; and

wherein rendering the new three-dimensional view of the three-dimensional object comprises applying an algorithm to reduce distances between the final position of the first touch point and a first screen-space projection, the initial position of the second touch point and a second screen-space projection, and the initial position of the third touch point and a third screen-space projection.

13. The multi-touch display device of claim 12, wherein the second touch point and third touch point define an axis about which the new view of the three-dimensional object is rotated compared to the initial view of the three-dimensional object.

14. The multi-touch display device of claim 12, wherein detecting the first touch, second touch, and third touch at respective touch points comprises detecting only three touches at only three touch points;

and

wherein applying the algorithm to reduce the distances results in zero distances between each touch point and a respective screen-space projection when movement of the three-dimensional object is unconstrained.

15. The multi-touch display device of claim 12, wherein the algorithm comprises a least squares minimization algorithm.

16. The multi-touch display device of claim 15, wherein the least squares minimization algorithm comprises a Levenberg-Marquardt algorithm.

17. The multi-touch display device of claim 16, wherein applying the algorithm to reduce the distances results in zero distances between each touch point and respective screen-space projection when the three-dimensional object is able to move with six degrees of freedom.