Method to control perspective for a camera-controlled computer

• US 8,649,554 B2
• Filed: 05/29/2009
• Issued: 02/11/2014
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Snook, Gregory N., Latta, Stephen, Geisner, Kevin, Lee, Johnny, Markovic, Relia, Lanygridge, Adam Jethro
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US12/475,245
• Publication Number:
  US 20100281439A1
• Time in Patent Office: 1,719 Days
• Field of Search: 382/100, 382/103, 382/104, 382/209, 382/218, 382/107, 382/168, 382/170, 382/180, 382/190, 382/224, 345/156, 345173-474, 345/419, 715/863, 704/243, 348/77, 348/E7.085, 348/40, 348/169, 375/240.28, 434/258, 463/8, 463 30- 31, 463/33, 463/36, 463/258, 473/269, 473/447, 700/138
• US Class Current: 382/103
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Systems, methods and computer readable media are disclosed for controlling perspective of a camera-controlled computer. A capture device captures user gestures and sends corresponding data to a recognizer engine. The recognizer engine analyzes the data with a plurality of filters, each filter corresponding to a gesture. Based on the output of those filters, a perspective control is determined, and a display device displays a new perspective corresponding to the perspective control.

Claims

1. A method for changing a perspective displayed on a display device, comprising:

receiving data captured by a capture device, the capture device capturing movement or position of at least part of a user or an object controlled by the user;

analyzing the data to determine it is likely that the user performed a system-recognized gesture that is indicative of modifying a perspective of a camera view into a virtual scene displayed on the display device, the system-recognized gesture comprising the user moving his or her head relative to the display device; and

displaying the modified perspective of the camera view into the virtual scene on the display device based on a position of the user'"'"'s head relative to the display device, the modified perspective being determined by a viewing area formed by the planes from the user'"'"'s eyes or head extending to a border of the display device, such that movement by the user results in a parallax view perspective.

2. The method of claim 1, wherein the perspective is at least one from a set, the set comprising:

a change from a first preset to a second preset, a dolly, a crane, a zoom, a pan, a tilt, a truck, and a cant.

3. The method of claim 1, further comprising:

receiving second data captured by a second capture device, the second capture device capturing at least part of a second user or an object controlled by the second user;

analyzing the second data to determine it is likely that the second user performed a second system-recognized gesture, the second system-recognized gesture corresponding to a second perspective; and

wherein displaying the perspective on the display device comprises;

displaying a combination of the perspective and the second perspective on the display device.

4. The method of claim 3, wherein the perspective is displayed on a first portion of the display device, and the second perspective is displayed on a second portion of the display device.

5. The method of claim 3, wherein the system-recognized gesture corresponds to a physical location of the user, the second system-recognized gesture corresponds to a physical location of the second user, and a combination of the first perspective and the second perspective comprises:

a perspective corresponding to an average physical location of the user and the second user.

6. The method of claim 3, wherein displaying a combination of the first perspective and the second perspective comprises:

displaying the perspective overlaid on the second perspective or the second perspective overlaid on the perspective.

7. The method of claim 6, wherein each perspective is tinted with a unique color.

8. The method of claim 6, wherein one of the perspective and the second perspective comprises an infrared image one of the perspective and the second perspective comprises a color image.

9. The method of claim 1, wherein a perspective comprises a plurality of attributes, and the perspective comprises a first attribute of the plurality of attributes and the second perspective comprises a second attribute of the plurality of attributes.

10. A system for changing a perspective displayed on a display device, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system at least to;

receive data captured by a capture device, the capture device capturing movement or position of at least part of a user or an object controlled by the user;

analyze the data to determine it is likely that the user performed a system-recognized gesture that is indicative of modifying a perspective of a camera view into a virtual scene displayed on the display device, the system-recognized gesture comprising the user moving his or her head relative to the display device; and

display the modified perspective of the camera view into the virtual scene on the display device based on a position of the user'"'"'s head relative to the display device, wherein the modified perspective is determined by a viewing area formed by the planes from the user'"'"'s eyes or head extending to a border of the display device, such that movement by the user results in a parallax view perspective.

11. The system of claim 10, wherein the display device displays a scene comprising an unaffected portion and an affected portion, and the instructions that, when executed on the processor cause the system to at least display the modified perspective of the camera view into the virtual scene on the display device further cause the system at least to:

display the perspective on the affected portion of the scene of the display device.

12. The system of claim 10, wherein the instructions that, when executed on the processor cause the system to at least display the modified perspective of the camera view into the virtual scene on the display device further cause the system at least to:

move the camera view into the virtual scene displayed on the display device in a direction corresponding to a vector of movement of the user, the vector of movement being coplanar with a plane of the display device.

13. The system of claim 10, wherein the gesture comprises a distance from the display device or the capture device, and the modified perspective comprises a magnification of a portion of the virtual scene.

14. The system of claim 13, wherein the magnification comprises magnification of text or magnification of the virtual scene on the display device.

15. The system of claim 10, wherein a virtual object in the viewing area is determined to be a focus point, and the parallax viewing perspective keeps the focus point in a noticeable position in the viewing area.

16. The system of claim 10, wherein the data corresponds to a position or movement of the head of the user, a position or movement of a hand of the user, a position or movement of a body part of the user, a position or movement of the user, a position or movement of a prop controlled by the user, a location of the user in a scene in which the user is being captured, or a location of the user relative to the capture device or the display device.

17. The system of claim 10, wherein the memory further bears instructions that, when executed on the processor, cause the system at least to:

receive second data captured by a second capture device, the second capture device capturing at least part of a second user or an object controlled by the second user;

analyze the second data to determine it is likely that the second user performed a second system-recognized gesture, the second system-recognized gesture corresponding to a second perspective; and

display a combination of the perspective and the second perspective on the display device.

18. The system of claim 17, wherein the system-recognized gesture corresponds to a physical location of the user, the second system-recognized gesture corresponds to a physical location of the second user, and a combination of the first perspective and the second perspective comprises:

a perspective corresponding to an average physical location of the user and the second user.

19. A computer-readable storage medium, comprising computer-readable instructions that when executed on a processor, cause the processor to perform the operations of:

receiving data captured by a capture device, the capture device capturing movement or position of at least part of a user or an object controlled by the user;

analyzing the data to determine it is likely that the user performed a system-recognized gesture, the system-recognized gesture corresponding to a perspective, the system-recognized gesture comprising the user moving his or her head relative to the display device; and

displaying the perspective on the display device based on a position of the user'"'"'s head relative to the display device, the perspective being determined by a viewing area formed by the planes from the user'"'"'s eyes or head extending to a border of the display device, such that movement by the user results in a parallax view perspective.