Computer interface employing a manipulated object with absolute pose detection component and a display

• US 7,961,909 B2
• Filed: 09/18/2009
• Issued: 06/14/2011
• Est. Priority Date: 03/08/2006
• Status: Active Grant

• Current Assignee: Electronic Scripting Products, Inc.
• Original Assignee: Electronic Scripting Products, Inc.
• Inventors: Mandella, Michael J., Gonzalez-Banos, Hector H., Alboszta, Marek
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US12/586,226
• Publication Number:
  US 20100013860A1
• Time in Patent Office: 634 Days
• Field of Search: 382/100, 382/103, 382/107, 382/122, 382/123, 382/312, 382/313, 382/314, 382/321, 178/18.01, 178/18.09, 235/472.01, 345163-166, 345179-183
• US Class Current: 382/103
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G01B 21/04 by measuring coordinates of points
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 10/17 : using hand-held instruments

Abstract

A system that has a remote control, e.g., a wand, equipped with a relative motion sensor that outputs data indicative of a change in position of the wand. The system also has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. The system uses one or more controllers to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The data enables determination of the absolute pose of the wand, which includes the absolute position of a reference point chosen on the wand and the absolute orientation of the wand. To properly express the absolute parameters of position and/or orientation of the wand a reference location is chosen with respect to which the calculations are performed. The system is coupled to a display that shows an image defined by a first and second orthogonal axes such as two axes belonging to world coordinates (X_o,Y_o,Z_o). The one or more controllers are configured to generate signals that are a function of the absolute position of the wand in or along a third axis for rendering the display. To simplify the mapping of a real three-dimensional environment in which the wand is operated to the cyberspace of the application that the system is running, the third axis is preferably the third Cartesian coordinate axis of world coordinates (X_o,Y_o,Z_o).

Claims

1. A system comprising:

a) a first plurality of predetermined light sources disposed in an asymmetric substantially linear pattern;

b) a photodetector configured to detect said first plurality of predetermined light sources and generate photodetector data representative of the positions of said first plurality of predetermined light sources;

c) a controller configured to identify a derivative pattern of said first plurality of predetermined light sources from said photodetector data, wherein said derivative pattern is indicative of the position of said photodetector;

d) a remote control, said photodetector being in said remote control;

wherein said controller is further configured to determine an absolute position of said remote control with respect to a reference location, and wherein said system is coupled to a display that shows an image substantially defined by a first and a second orthogonal axes, and wherein said controller is configured to generate signals for rendering said display, said signals being a function of said absolute position of said remote control in a third axis, said third axis being orthogonal to said first and second orthogonal axes.

2. A system comprising:

a) a first plurality of predetermined light sources disposed in an asymmetric substantially linear pattern;

b) a photodetector configured to detect said first plurality of predetermined light sources and generate photodetector data representative of the positions of said first plurality of predetermined light sources;

c) a controller configured to identify a derivative pattern of said first plurality of predetermined light sources from said photodetector data, wherein said derivative pattern is indicative of the position of said photodetector;

d) a remote control, said photodetector being in said remote control;

wherein said controller is further configured to determine an absolute position of said remote control with respect to a reference location, and wherein said system is coupled to a display that shows an image substantially defined by a first and a second orthogonal axes, and wherein said controller is configured to determine an initial absolute position of said remote control in a third axis, wherein said third axis is orthogonal to said first and second orthogonal axes.

3. The system of claim 2, wherein said controller is further configured to determine a change in position of said remote control in said third axis, and to combine said initial absolute position of said remote control with said change in position of said remote control.

4. A method for use with a system having a remote control, the method comprising:

a) accepting light data indicative of light detected by a photodetector from a first plurality of predetermined light sources;

b) accepting relative motion data from a relative motion sensor indicative of a change in a position of said remote control; and

c) determining an absolute position of said remote control based on said light data and said relative motion data, wherein said absolute position is determined with respect to a reference location.

5. The method of claim 4, wherein said system is coupled to a display that shows an image substantially defined by a first and a second orthogonal axes, and wherein determining said absolute position of said remote control comprises determining an absolute position of said remote control in a third axis that is orthogonal to said first and said second orthogonal axes.

6. The method of claim 5, further comprising generating signals for rendering said display based on said absolute position of said remote control in said third axis.

7. The method of claim 4, wherein said remote control cooperated with an electronic device that performs the steps of:

a) displaying a plurality of selectable characters; and

b) navigating a cursor to a particular selectable character based on the output of said relative motion sensor.

8. The method of claim 7, further comprising receiving a selection of a particular one of said selectable characters.

9. The method of claim 4, wherein said remote control cooperates with an electronic device that performs the steps of:

a) displaying an image on a screen of said electronic device;

b) detecting a rotation of said remote control; and

c) rotating at least a portion of said image in response to said rotation of said remote control.

10. The method of claim 9, further comprising receiving a selection of said portion of said image.

11. The method of claim 9, wherein said electronic device is associated with a media system.

12. The method of claim 4, wherein said relative motion sensor comprises at least one of an accelerometer and a gyroscope.