Generating position information using a video camera

• US 8,013,838 B2
• Filed: 06/30/2006
• Issued: 09/06/2011
• Est. Priority Date: 06/30/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gao, Rui, Hsu, Feng-Hsiung
• Primary Examiner(s): Nguyen; Kevin
• Assistant Examiner(s): Lee, Jr.; Kenneth B
• Application Number:
  US11/428,279
• Publication Number:
  US 20080001918A1
• Time in Patent Office: 1,894 Days
• Field of Search: 345157-163
• US Class Current: 345/157
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A position determination module is described which receives image information from a video camera. The position determination module identifies at least one reference field in the image information to provide identified reference information. The position determination module then generates position information based on the identified reference information. The position information is associated with a position of the reference field with respect the video camera. In one implementation, the video camera is stationary and the reference field is coupled to a user-manipulated device. In another implementation, the reference field is coupled to a stationary display device and the video camera is coupled to a movable user-manipulated device.

Claims

1. A method comprising:

receiving image information from a video camera, the image information capturing a scene which includes at least part of a display device, wherein;

the video camera is affixed to a user-manipulated device configured for carrying by a user; and

the user-manipulated device comprises a remote control device or a controller;

identifying a plurality of reference fields in the image information to provide identified reference information for controlling an application displayed on the display device, wherein the plurality of reference fields are displayed as part of graphical information displayed on the screen of the display device, the user-manipulated device capable of identifying the plurality of reference fields based on colors of the plurality of reference fields, based on shapes of the plurality of reference fields, and based on patterns of elements that comprise each reference field of the plurality of reference fields;

generating position information based on the identified reference information, the position information expressing a position of the user-manipulated device relative to at least one reference field associated with the display device, wherein the position of the user-manipulated device is calculated based on a position of the video camera associated with the user-manipulated device; and

controlling the application based on the generated position information.

2. The method of claim 1, wherein the receiving of the image information comprises receiving image information in response to the user pointing at a particular object being presented by the display device with the user-manipulated device.

3. The method of claim 1, wherein the plurality of reference fields comprise at least two reference fields arranged diagonally opposed to one another at opposite corners of the screen of the display device.

4. The method of claim 3, wherein the plurality of reference fields each include a plurality of reference elements.

5. The method of claim 4, wherein the plurality of reference elements of a first reference field are arranged in a different pattern from the plurality of reference elements of a second reference field.

6. The method of claim 1, wherein the controlling of the application involves determining an object displayed on the screen of the display device that the user is pointing at with the user-manipulated device.

7. The method of claim 1, further comprising:

receiving inertial sensor information from an inertial sensor, and

using the inertial sensor information to supplement the identified reference information provided by the plurality of reference fields in the image information to generate the position information.

8. One or more computer readable media containing machine-executable instructions for implementing the receiving, identifying, generating, and controlling of claim 1.

9. An apparatus including logic configured to implement the receiving, identifying, generating, and controlling of claim 1.

10. A system comprising:

a user-manipulated device, the user-manipulated device including a video camera associated therewith;

a display device for presenting objects generated by an application;

at least one reference field incorporated as graphical information displayed on a screen of the display device to provide reference information for controlling an application displayed on the display device, the user-manipulated device capable of identifying the at least one reference field based on one or more colors of the at least one reference field, based on a shape of the at least one reference field, and based on a pattern of elements that comprise the at least one reference field; and

a position determination module configured to;

receive image information from the video camera, the image information capturing a scene which includes at least part of the display device, wherein the user-manipulated device is carried by a user and the video camera moves with the user-manipulated device;

identify said at least one reference field in the image information to provide identified reference information; and

generate position information based on the identified reference information, the position information expressing a position of the video camera relative to said at least one reference field.

11. The system of claim 10, wherein said at least one reference field comprises plural reference fields arranged at respective positions with respect to the display device.

12. The system of claim 10, wherein said at least one reference field comprises a defined pattern of plural reference elements.

13. The system of claim 12, wherein:

the at least one reference field comprises at least a first reference field and a second reference field; and

the plural reference elements of a first reference field are arranged in a different pattern from the plural reference elements of a second reference field.

14. The system of claim 10, further comprising an inertial sensor for providing inertial information, wherein the position determination module is configured to use the inertial sensor information to supplement the identified reference information provided by identifying the at least one reference field in the image information to generate the position information.

15. The system of claim 10, wherein the user-manipulated device is a game controller and the application is a game.

16. The system of claim 10, wherein the user-manipulated device is a mobile telephone.

17. A system comprising:

a user-manipulated device, the user-manipulated device including a video camera associated therewith;

a display device for presenting objects generated by an application;

a plurality of reference fields placed at defined positions relative to the display device, each reference field comprising a plurality reference elements arranged in a pattern, wherein a first pattern of a first plurality of reference elements differs from a second pattern of a second plurality of reference elements; and

a position determination module configured to;

receive image information from the video camera, the image information capturing a scene which includes at least part of the display device, wherein the user-manipulated device is carried by a user and the video camera moves with the user-manipulated device;

identify said at least one reference field in the image information to provide identified reference information; and

generate position information based on the identified reference information, the position information expressing a position of the video camera relative to said at least one reference field.

18. The system of claim 17, wherein at least some of the plurality of reference elements comprise discrete mechanisms coupled to the display device.

19. The system of claim 17, wherein at least some of the plurality of reference elements comprise graphical information displayed on the display screen.

20. The system of claim 17, further comprising an inertial sensor for providing inertial information, wherein the position determination module is configured to use the inertial sensor information to supplement the identified reference information provided by identifying the at least one reference field in the image information for in generating the position information.