Novel man machine interfaces and applications

• US 20040046736A1
• Filed: 07/21/2003
• Published: 03/11/2004
• Est. Priority Date: 08/22/1997
• Status: Active Grant

• Current Assignee: Peter Smith, Timothy R. Pryor
• Original Assignee: Peter Smith, Timothy R. Pryor
• Inventors: Pryor, Timothy R., Smith, Peter
• Granted Patent: US 7,042,440 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/042 : by opto-electronic means
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 : by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06T 19/20 : Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 : Shape modification
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G10H 1/0008 : Associated control or indicating means
• CPC Class Codes: G10H 2220/321 : Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders

Abstract

Affordable methods and apparatus are disclosed for inputting position, attitude (orientation) or other object characteristic data to computers for the purpose of Computer Aided Design, Painting, Medicine, Teaching, Gaming, Toys, Simulations, Aids to the disabled, and internet or other experiences. Preferred embodiments of the invention utilize electro-optical sensors, and particularly TV Cameras, providing optically inputted data from specialized datum'"'"'s on objects and/or natural features of objects. Objects can be both static and in motion, from which individual datum positions and movements can be derived, also with respect to other objects both fixed and moving. Real-time photogrammetry is preferably used to determine relationships of portions of one or more datums with respect to a plurality of cameras or a single camera processed by a conventional PC.

Claims

1. A method for designing an object comprising the steps of:

a) providing a computer having a display;

b) providing a physical surrogate of said object;

c) providing in said computer a data base of said object;

d) using said data base, displaying a 3D representation of said object;

e) determining the position of a member positioned by a person designing said object;

f) determining a position of said surrogate; and

g) from said determined position, determining changes to said object data base to be made in said computer.

2. A method according to claim 1, wherein said surrogate is a model of an object.

3. A method according to claim 1, wherein said determining the position of said member is made electro-optically.

4. A method according to claim 1, wherein said electro-optical determination is made using at least one TV camera.

5. A method according to claim 4, wherein a stereo pair of TV cameras are employed.

6. A method according to claim 1, wherein datums on said member are sensed.

7. A method according to claim 1, wherein datums on said surrogate are sensed.

8. A method according to claim 1, wherein datums are sensed in relation to the position of said surrogate.

9. A method according to claim 1, wherein said member is positioned based on determination of information on said 3D display.

10. A method according to claim 1, wherein changes in the data base are reflected in changes to said 3D display.

11. A method according to claim 1, wherein said steps f and g are repeated to effect a continued change in said data base and the display thereof.

12. A method according to claim 1, wherein said steps e to g are repeated to effect a continued change in said data base and the display thereof.

13. A method according to claim 1, including the further step of creating a new surrogate using said changed data base.

14. A method according to claim 1, including the further step of sensing orientation as well as position of at least one of said surrogate or said member.

15. A method according to claim 1, wherein the relative position of said member with respect to said surrogate is sensed.

16. A method according to claim 1, wherein said changes to said data base effect a change in a surface shape of said 3D representation of said object.

17. A method according to claim 1, wherein said surrogate is stationary.

18. A method according to claim 1, wherein said surrogate is moved in executing said design.

19. A method according to claim 1, wherein said surrogate is rotated in executing said design.

20. A method according to claim 1, wherein said surrogate is held in a hand of said person.

21. A method according to claim 1, wherein said member is said person'"'"'s finger.

22. A method according to claim 1, wherein said member is said person'"'"'s hand.

23. A method according to claim 1, wherein said member is held in a hand of said person.

24. A method according to claim 1, wherein said member has a known shape which is used to change said data base when moved by said person.

25. A method according to claim 1, wherein information concerning said member is electro-optically communicated to said computer.

26. A method according to claim 1, wherein said data base is downloaded from a file.

27. A method according to claim 1, wherein said data base is generated electro-optically from a physical object.

28. A method according to claim 1, where said surrogate is an arbitrary object.

29. A method according to claim 2 wherein said surrogate is a vehicle model

30. A method for designing an object comprising the steps of:

providing a computer having a display;

providing a physical surrogate of said object;

providing in said computer a data base of said object;

using said data base, displaying a 3D representation of said object;

determining the orientation of a member positioned by a person designing said object;

determining the orientation of said surrogate; and

from said determined orientation, determining changes to said object data base to be made in said computer.

31. A method according to claim 30, including the further step of determining the position of said surrogate and/or said member.

32. A method according to claim 30, wherein said orientation of said surrogate is determined electro-optically.

33. A method according to claim 31, wherein said position is determined electro-optically.

34. A method according to claim 30, wherein said electro-optical determination employs at least one TV camera.

35. A method for designing an object comprising the steps of:

providing a computer having a display;

providing a physical surrogate of said object;

providing in said computer a data base of said object;

using said data base, displaying a 3D representation of said object;

determining the relative position of a member positioned by a person designing said object with respect to said surrogate; and

from said determined relative position, determining changes to said object data base to be made in said computer.

36. A method according to claim 35, including the further step of determining the relative orientation of said surrogate and/or said member.

37. A method according to claim 35, wherein said relative position is determined electro-optically.

38. A method according to claim 36, wherein said relative orientation is determined electro-optically.

39. A method according to claim 35, wherein said member is said person'"'"'s finger.

40. A method according to claim 35, wherein said member is said person'"'"'s hand.

41. A method according to claim 35, wherein said member is held in a hand of said person.