Physics simulation-based interaction for surface computing

• US 8,154,524 B2
• Filed: 09/03/2008
• Issued: 04/10/2012
• Est. Priority Date: 06/24/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew David, Izadi, Shahram, Garcia-Mendoza, Armando, Kirk, David, Hilliges, Otmar
• Primary Examiner(s): SHERMAN, STEPHEN G
• Application Number:
  US12/203,350
• Publication Number:
  US 20090315839A1
• Time in Patent Office: 1,315 Days
• Field of Search: 345173-178, 178 1801- 1809, 178/18.11, 340/4.12, 340/407.1, 340/407.2, 463/30
• US Class Current: 345/173
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: G06F 2111/10 Numerical modelling
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

The claimed subject matter provides a system and/or a method that facilitates enhancing interactive surface technologies for data manipulation. A surface detection component can employ a multiple contact surfacing technology to detect a surface input, wherein the detected surface input enables a physical interaction with a portion of displayed data that represents a corporeal object. A physics engine can integrate a portion of Newtonian physics into the interaction with the portion of displayed data in order to model at least one quantity related associated with the corporeal object, the quantity is at least one of a force, a mass, a velocity, or a friction.

Claims

1. A system that facilitates enhancing interactive surface technologies for data manipulation, comprising:

one or more processors;

memory communicatively coupled to the one or more processors;

a surface detection component stored in the memory and executable by the one or more processors that detects a surface input on a surface, and enables a physical interaction between the detected surface input and a portion of displayed data on the surface;

a proxy object component stored in the memory and executable by the one or more processors that generates a proxy object for the detected surface input, the surface input being incorporated into a physics simulation;

a particle proxy component stored in the memory and executable by the one or more processors that models a contour shape of the surface input with a plurality of proxy objects, a contour of the plurality of proxy objects comprising a particle proxy; and

a physics engine stored in the memory and executable on the one or more processors that integrates a portion of Newtonian physics into the physical interaction between the detected surface input and the portion of displayed data to model a quantity associated with the portion of displayed data, the quantity comprising a force, a mass, a velocity, or a friction.

2. The system of claim 1, further comprising a display that renders the portion of displayed data.

3. The system of claim 2, wherein the display comprises a monitor, a television, a liquid crystal display (LCD), a plasma, a rear-projection display, a front-projection display, a cathode ray tube (CRT) monitor, a flat-panel, a display for a computing device, a portable digital assistance (PDA) screen, a laptop display, a computer monitor, a smartphone display, a cellular device screen, a mobile communication device display, or a portable gaming device display.

4. The system of claim 1, the physics engine employs a physic update related to the portion of Newtonian physics to the particle proxy between a first simulation frame and a second simulation frame.

5. The system of claim 4, the physics engine enacts at least one of a collision force or a friction force with the physic update.

6. The system of claim 4, the physics engine enacts at least one of a gathering of two or more objects or a piling of two or more objects with the physic update.

7. The system of claim 4, the physics engine enables support for at least one of a soft body simulation, a cloth body simulation, or a fluid simulation.

8. The system of claim 1, the physics engine applies an amount of a friction force proportional to a number of proxies applied to an object which is related to a surface area of touch.

9. The system of claim 1, further comprising a flow component that calculates a motion of the particle proxy by tracking a motion of the particle proxy between a first frame and a second frame, the motion of the particle proxy being independent of tracked contact information of the surface input.

10. The system of claim 9, the flow component calculates a velocity utilizing a block matching related to optical flow.

11. The system of claim 10, wherein the flow component computes a surface motion for a point related to the particle proxy by comparing each successive edge image.

12. The system of claim 10, the flow component computes a tangential motion by ray casting an image point and projecting the image point onto a tangent plane formed by a 3D point and an object normal.

13. The system of claim 1, further comprising a manipulator that provides at least one of a manipulation fidelity, a gathering, a manipulation of a 3-dimensional (3D) object, or a rigidity simulation.

14. A computer-implemented method that facilitates simulating physics in connection with surface computing, comprising:

under control of one or more processors configured with executable instructions;

detecting a surface input on a surface;

generating a proxy object for the surface input;

identifying a contour of the surface input;

dynamically modeling the contour of the surface input with a plurality of proxy particle objects; and

employing a force, a velocity, and/or a friction for each proxy particle object between a first frame and a second frame of surface input detection.

15. The method of claim 14, further comprising providing at least one of a soft body simulation, a cloth body simulation, or a fluid simulation.

16. The method of claim 14, further comprising:

applying an amount of a friction force proportional to a number of proxies applied to a displayed object on the surface, the number of proxies being related to a surface area of the surface input; and

sensing surface-like pressure which is transduced to force on the number of proxies.

17. The method of claim 14, further comprising calculating a motion of a particle proxy by tracking a motion of the particle proxy between the first frame and the second frame, the motion of the particle proxy being independent of tracked contact information of the surface input.

18. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

detecting a surface input, the detected surface input enabling a physical interaction with a portion of displayed data on a surface;

integrating a portion of Newtonian physics into the physical interaction with the portion of displayed data to model a quantity related to the portion of displayed data, the quantity comprising a force, a mass, a velocity, or a friction;

generating a proxy object for the detected surface input, the detected surface input being incorporated into a physics simulation;

modeling a contour shape of the detected surface input with a plurality of proxy objects, a contour of the plurality of proxy objects comprising a particle proxy; and

employing a physic update related to the portion of Newtonian physics to the particle proxy between a first simulation frame and a second simulation frame.