Methods and systems using three-dimensional sensing for user interaction with applications

• US 20110292181A1
• Filed: 04/16/2009
• Published: 12/01/2011
• Est. Priority Date: 04/16/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Acharya, Sunil, Ackroyd, Steve
• Application Number:
  US12/386,457
• Publication Number:
  US 20110292181A1
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

User interaction with a device is sensed using a three-dimensional imaging system. The system preferably includes a library of user profiles and upon acquiring a three-dimensional images of a user can uniquely identify the user, and activate appliances according to user preferences in the user profile. The system can also use data from the acquired image of the user'"'"'s face to confirm identity of the user, for purposes of creating a robust biometric password. Acquired three dimensional data can measure objects to provide automated, rapid and accurate measurement data, can provide image stabilization data for cameras and the like, and can create virtual three-dimensional avatars that mimic a user'"'"'s movements and expressions and can participate in virtual world activities. Three-dimensional imaging enables a user to directly manipulate a modeled object in three-dimensional space.

Claims

1. A method for a user to interface with at least one appliance, the method comprising the following steps:

(a) storing in a system a library of user profile data representing at least one potential user;

(b) capturing three-dimensional image data of a user in a space within which said appliance is desired to be operative;

(c) comparing data captured at step (b) with data stored in step (a) to identify said user and a profile for said user; and

(d) causing said device to activate in a manner according to said profile for said user.

2. The method of claim 1, wherein step (b) is carried out using a time-of-flight imaging system.

3. The method of claim 1, wherein at step (b), said appliance includes at least one appliance selected from a group consisting of (i) an entertainment appliance, (ii) a message-capturing appliance, (iii) a security appliance, (iv) an air-conditioning appliance, and (v) a space heating appliance.

4. The method of claim 1, wherein step (d) activates said appliance as a function of at least one of current date and current time.

5. The method of claim 1, wherein said appliance is a television, and:

step (a) includes storing a database of television programming data representing programs viewable on said television as a function of time; and

step (d) includes said system commanding said television to turn-on to a specific channel in accordance with said user profile.

6. The method of claim 5, further including:

using said system to capture three-dimensional data identifying each user watching said television, as a function of date and time; and

generating data representing a log of which users view what programming on said television at what dates and at what times.

7. The method of claim 6, further including communicating generated said data representing a log to at least one of a producer of television programming, a sponsor who has commercials viewable on said television, and a producer of film making.

8. The method of claim 1, wherein data captured at step (b) for a user is used as biometric identification limiting access to at least one appliance selected from a group consisting of (i) an answering machine, (ii) a computer account, (iii) a computer file, and (iv) financial data.

9. A method to enhance performance of an RGB image captured by a user appliance that includes a camera, the method comprising:

(a) providing said appliance with a system that captures three-dimensional image data of at least one object within a relevant field of view for said appliance;

(b) using three-dimensional image data captured at step (a) to reduce effects from any jitter in at least one RGB image acquired by said appliance;

(c) causing said appliance to output at least one RGB image corrected at step (b);

wherein effects of jitter in an RGB image output by said appliance is reduced.

10. The method of claim 9, wherein said appliance is at least one of (i) a camera within a mobile phone, (ii) a stand-alone still camera, and (iii) a video camera.

11. The method of claim 9, wherein step (a) includes providing a time-of-flight system.

12. The method of claim 9, wherein said appliance includes a user visible display of an image acquired by said appliance, and:

step (b) uses three-dimensional image data captured at step to determine orientation of a plane of said camera;

said system further displays a video game on said display including a displayed virtual object that moves virtually as a function of changes in orientation of said plane of said camera;

wherein said camera is caused to act quasi-haptically by allowing a user to control position of said displayed virtual object as said user alters orientation of said camera such that a video game can be played using said camera.

13. A method enabling movement of a displayable virtual object as a function of movement of at least part of a first user, the method comprising the following steps:

(a) providing a first system to capture three-dimensional image data of at least a portion of said first user;

(b) providing a first display whereon is viewable at least one of (i) a display of a virtual object, and (ii) a display of second user;

(c) using data captured at step (a) to allow said first user to directly manipulate said virtual object displayed on said first display.

14. The method of claim 13, wherein at step (b) said virtual object includes at least one of (i) a molecule, and (ii) a DNA strand.

15. The method of claim 13, wherein step (a) includes providing a time-of-flight system.

16. The method of claim 13, wherein data captured at step (a) is used to create a dynamic avatar representation of said first user, said avatar transmittable for viewing on at least a second display.

17. The method of claim 13, further including at least a second system to capture three-dimensional image data of at least a portion of a second user, said second system creating a dynamic avatar representation of said second user, said dynamic avatar created by said second system being transmittable for viewing on at least said first display.

18. The method of claim 17, wherein each avatar is transmittable via at least one of a network and the Internet.

19. The method of claim 17, wherein said first system and said second system enable said first user and said second user to interact with each other.

20. The method of claim 17, wherein said first system enables said first user to interact with a virtual reality world.