Computer interface employing a manipulated object with absolute pose detection component and a display

• US 8,553,935 B2
• Filed: 05/25/2011
• Issued: 10/08/2013
• Est. Priority Date: 03/08/2006
• Status: Active Grant

• Current Assignee: Electronic Scripting Products, Inc.
• Original Assignee: Electronic Scripting Products, Inc.
• Inventors: Mandella, Michael J., Gonzalez-Banos, Hector H., Alboszta, Marek
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/134,006
• Publication Number:
  US 20110227915A1
• Time in Patent Office: 867 Days
• Field of Search: 382/100, 382/103, 382/107, 382/122, 382/123, 382/312, 382/313, 382/314, 382/321, 178/18.01, 178/18.09, 235/472.01, 345163-166, 345179-183
• US Class Current: 382/103
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G01B 21/04 by measuring coordinates of points
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 10/17 : using hand-held instruments

Abstract

A system that has a remote control, e.g., a wand, equipped with a relative motion sensor that outputs data indicative of a change in position of the wand. The system also has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. The system uses one or more controllers to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The data enables determination of the absolute pose of the wand, which includes the absolute position of a reference point chosen on the wand and the absolute orientation of the wand. To properly express the absolute parameters of position and/or orientation of the wand a reference location is chosen with respect to which the calculations are performed. The system is coupled to a display that shows an image defined by a first and second orthogonal axes such as two axes belonging to world coordinates (X_o,Y_o,Z_o). The one or more controllers are configured to generate signals that are a function of the absolute position of the wand in or along a third axis for rendering the display. To simplify the mapping of a real three-dimensional environment in which the wand is operated to the cyberspace of the application that the system is running, the third axis is preferably the third Cartesian coordinate axis of world coordinates (X_o,Y_o,Z_o).

Claims

1. A method for use with a system having a manipulated object, the method comprising:

a) accepting light data indicative of light detected by a photodetector mounted on-board said manipulated object from a first plurality of predetermined light sources having known locations in world coordinates;

b) accepting relative motion data from a relative motion sensor mounted on-board said manipulated object indicative of a change in an orientation of said manipulated object; and

c), determining the pose of said manipulated object based on said light data and said relative motion data, wherein said pose is determined with respect to said world coordinates.

2. The method of claim 1, wherein said first plurality of light sources is arranged in a predetermined pattern.

3. The method of claim 2, wherein said predetermined pattern comprises at least one member of the group consisting of linear patterns, non-linear patterns and asymmetric patterns.

4. The method of claim 2, wherein said first plurality of predetermined light sources comprises IR LEDs.

5. The method of claim 1, wherein said system is coupled to a display that shows an image substantially defined by a first and second orthogonal axes.

6. The method of claim 5, wherein said pose is defined by Euler angles (φ

, θ

, ψ

) in rotated object coordinates or their mathematical equivalents.

7. The method of claim 5, wherein said manipulated object is configured to generate signals for rendering said display.

8. The method of claim 7, wherein said rendering comprises rearranging of a visual element.

9. The method of claim 5, wherein at least a subset of said pose is used for rotating at least a portion of said image.

10. The method of claim 5, wherein at least a subset of said pose is used for rotating a visual element of said image.

11. The method of claim 1, wherein said manipulated object is selected from the group consisting of wands, remote controls, three-dimensional mice, game controls, gaming objects, jotting implements, surgical implements, three-dimensional digitizers, digitizing styli, hand-held tools and utensils.

12. A system comprising a manipulated object, said system comprising:

a) a first plurality of predetermined light sources disposed at known positions in world coordinates;

b) a photodetector mounted on-board said manipulated object for generating light data indicative of light detected from said first plurality of light sources;

c) a relative motion sensor mounted on-board said manipulated object for generating relative motion data indicative of a change in an orientation of said manipulated object; and

d) a processor for determining the pose of said manipulated object based on said light data and said relative motion data, wherein said pose is determined with respect to said world coordinates.

13. The system of claim 12, wherein said known positions are fixed positions.

14. The system of claim 13, wherein said fixed positions define a predetermined pattern.

15. The system of claim 14, wherein said predetermined pattern comprises at least one member of the group consisting of linear patterns, non-linear patterns and asymmetric patterns.

16. The system of claim 12, wherein said first plurality of predetermined light sources comprises IR LEDs.

17. The system of claim 12, further comprising a display for showing an image substantially defined by a first and second orthogonal axes.

18. The system of claim 17, wherein said pose is defined by Euler angles (φ

, θ

, ψ

) in rotated object coordinates or their mathematical equivalents.

19. The system of claim 17, wherein said processor is further configured to generate signals for rendering said display in response to at least a subset of said pose.

20. The system of claim 17, wherein said image further comprises a visual element and said signals for rendering comprise signals for rotating said visual element.

21. The system of claim 12, wherein said manipulated object is selected from the group consisting of wands, remote controls, three-dimensional mice, game controls, gaming objects, jotting implements, surgical implements, three-dimensional digitizers, digitizing styli, hand-held tools and utensils.