Generating position information using a video camera

• US 8,587,520 B2
• Filed: 08/11/2011
• Issued: 11/19/2013
• Est. Priority Date: 06/30/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hsu, Feng-Hsiung, Gao, Rui
• Primary Examiner(s): Nguyen, Kevin M
• Assistant Examiner(s): Lee, Jr., Kenneth B
• Application Number:
  US13/208,003
• Publication Number:
  US 20110300940A1
• Time in Patent Office: 831 Days
• Field of Search: 345157-163, 348/135, 463/36
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Some implementations provide techniques and arrangements to receive image information. A plurality of reference fields of a user-manipulated device may be identified. Each reference field of the plurality of reference fields may include reference elements. The plurality of reference fields may be identified based on colors of the reference elements, shapes of the reference elements, and/or a pattern of the reference elements. Some implementations may generate position information based on the plurality of reference fields. The position information may identify a position of the user-manipulated device relative to the video camera. Some implementations may provide the position information to an application.

Claims

1. A method comprising:

receiving image information from a video camera;

identifying at least one reference field of a user-manipulated device based on colors and shapes the comprise the at least one reference field, the at least one passive reference field reflecting light;

determining identified reference information based on the at least one passive reference field;

receiving supplemental positioning information from an inertial sensor, the supplemental positioning information used to compensate for blind spots in the image information received from the video camera;

generating position information based at least in part on the identified reference information and the supplemental positioning information, the position information expressing a position of the user-manipulated device relative to the video camera, the position information generated at least in part by mapping the position of the at least one passive reference field to one or more fixed reference points; and

controlling an application based on the generated position information.

2. The method of claim 1, wherein the user-manipulated device is pointed toward a display device.

3. The method of claim 1, further comprising:

receiving inertial sensor information from an inertial sensor, the user-manipulated device including the inertial sensor; and

generating the position information based at least in part on the identified reference information and the inertial sensor information.

4. The method of claim 1, further comprising receiving the image information from the video camera in response to the user-manipulated device being positioned to point at a particular object presented by the display device.

5. The method of claim 1, wherein:

the pattern of elements is associated with the user-manipulated device; and

another pattern of elements is associated with another user-manipulated device.

6. The method of claim 1, wherein:

the at least one reference field includes a first reference field and a second reference field;

a first plurality of reference elements of the first reference field are arranged in a first pattern; and

a second plurality of reference elements of the second reference field are arranged in a second pattern that is different from the first pattern.

7. The method of claim 1, wherein the user-manipulated device comprises a game controller.

8. The method of claim 1, wherein the game controller comprises a data glove device, a wristband device, a headband device, a shoe-borne device, or any combination thereof.

9. The method of claim 1, wherein:

the video camera is stationary; and

a field of view of the video camera encompasses an area in which a user is expected to operate the user-manipulated device.

10. The method of claim 1, wherein the video camera sweeps back and forth to encompass an area in which a user is operate the user-manipulated device.

11. An electronic device comprising:

a processor; and

storage media including instructions executable by the processor to perform acts comprising;

receiving image information from a video camera;

identifying a plurality of reference fields of a user-manipulated device based on the image information, each reference field of the plurality of reference fields including reference elements, the plurality of reference fields identified based on colors of the reference elements and a pattern of the reference elements;

receiving supplemental positioning information from an internal sensor, the supplemental positioning information used to compensate for blind spots detected in the image information received from the video camera;

generating position information based on the plurality of reference fields and the supplemental positioning information, the position information identifying a position of the user-manipulated device relative to the video camera; and

providing the position information to an application.

12. The electronic device of claim 11, wherein the user-manipulated device includes one or more control actuators.

13. The electronic device of claim 12, wherein the one or more actuators comprise a button actuator, a joystick actuator, a knob actuator, a steering mechanism actuator, or any combination thereof.

14. The electronic device of claim 11, wherein the image information is received from two or more video cameras.

15. The electronic device of claim 11, wherein the position information is generated via one or more geometrical mapping equations based on a position of the reference elements with reference to one or more fixed reference points.

16. The electronic device of claim 11, wherein the position information is generated by one or more geometrical mapping equations based on a position of the reference elements with reference to each other.

18. The system of claim 16, wherein the reference elements comprise passive elements that do not generate electromagnetic radiation.

19. The system of claim 16, wherein the reference elements reflect at least one of infrared radiation or visible-spectrum light.

17. A system comprising:

a gaming console comprising a memory including instructions and a processor to execute the instructions to perform acts comprising;

receiving image information from a plurality of video cameras, the plurality of video cameras capturing the image information from a plurality of angles;

identifying at least one reference field of a user-manipulated device based on a color of the at least one reference field, based on a shape of the at least one reference field, and based on a pattern of reference elements that comprise the at least one reference field;

identifying reference information based on the at least one reference field;

receiving supplemental positioning information from an inertial sensor of the user-manipulated device;

based on the reference information and the supplemental positioning information, generating position information that identifies a position of the user-manipulated device relative to the two or more video cameras; and

controlling one or more applications based on the position information.

20. The system of claim 17, wherein the reference elements are placed on a back plate of the user-manipulated device.