Method and apparatus for user interface of input devices

• US 8,730,165 B2
• Filed: 05/29/2013
• Issued: 05/20/2014
• Est. Priority Date: 10/31/2007
• Status: Expired due to Fees

• Current Assignee: Genedics, LLC
• Original Assignee: Genedics, LLC
• Inventors: Fein, Gene S., Fein, Jackson T., Merritt, Edward
• Primary Examiner(s): Hicks, Charles V
• Application Number:
  US13/904,684
• Publication Number:
  US 20130257724A1
• Time in Patent Office: 356 Days
• Field of Search: 345/158
• US Class Current: 345/158
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G03H 1/0005 Adaptation of holography to specific applications holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042
• CPC Class Codes: G03H 1/2294 Addressing the hologram to an active spatial light modulator
• CPC Class Codes: G03H 2001/0061 in haptic applications when the observer interacts with the holobject
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A 3 dimensional (3-D) user interface system employs: one or more 3-D projectors configured to display an image at a first location in 3-D space; one or more sensors configured to sense user interaction with the image and to provide user interaction information; and a processor configured (i) to receive the user interaction information from the one or more sensors; (ii) to correlate the user interaction with the image; and (iii) to provide one or more indications responsive to a correlation of the user interaction with the image, including displaying the image at a second location in 3-D space.

Claims

1. A system for a 3 dimensional (3-D) user interface, the system comprising:

one or more 3-D projectors configured to display an image at a first location in a 3-D coordinate system;

one or more sensors configured based on quadrilateral angle navigation to sense user interaction with the image and to provide user interaction information; and

a processor configured (i) to receive the user interaction information from the one or more sensors;

(ii) to correlate the user interaction with the image; and

(iii) to provide one or more indications responsive to a correlation of the user interaction with the image,

wherein the one or more indications comprise displaying the image at a second location in the 3-D coordinate system.

2. The system of claim 1, further comprising a communications port coupled to the processor and configured to provide communications interface with a computer system.

3. The system of claim 1, wherein the image is a holograph.

4. The system of claim 1, wherein the user interaction with the image comprises movement of the user relative to the image, or any sound produced by the user, or both.

5. The system of claim 1, wherein the one or more sensors comprise any one or combination of location sensors, motion sensors, light sensors and sound sensors.

6. The system of claim 5, wherein the location sensors comprise laser sensors configured to geometrically identify a position within the 3-D coordinate system.

7. The system of claim 1, wherein the image at the first location ceases to be displayed before, when or after the image at the second location starts to be displayed.

8. The system of claim 1, wherein the distance between the first location and the second location are within a range such that a sequential display of the image at the first location and the second location can be thought of as a movement of the image from the first location to the second location.

9. The system of claim 8, wherein the user interaction comprises movement of the whole body or one or more body parts of the user from the first location to the second location, thereby creating an effect of the image moving from the first location to the second location responsive to the movement of the whole body or the one or more body parts of the user from the first location to the second location.

10. The system of claim 9, wherein the image is displayed at the second location immediately after the whole body or the one or more body parts of the user moves to the second location, thereby creating an effect of the image moving with the whole body or the one or more body parts of the user.

11. The system of claim 10, wherein the image is that of an input device for a computer-based system, and the one or more indications further comprise inputting data to the computer-based system responsive to the correlation of the user interaction with the image.

12. The system of claim 11, wherein the data input to the computer-based system comprise data that are related or not related to the spatial difference between the first location and the second location in the 3-D coordinate system, or both.

13. The system of claim 11, wherein the input device is any one or combination of:

a touch screen, a touch pad, a digitizing tablet, a haptic device, a mouse, track balls, light guns, steering wheels, yokes, joysticks, directional pads, wired gloves, game controllers, game pads, game paddles and Wii remotes.

14. The system of claim 11, wherein the image of the input device is any one or combination of a keyboard and a mouse, and the one or more indications further comprise input to the computer through one of a keyboard and a mouse.

15. A method for providing a 3 dimensional (3-D) user interface, the method comprising:

generating an image at a first location in a 3-D coordinate system;

sensing user interaction with the image, said sensing using quadrilateral angle navigation;

correlating the user interaction with the image; and

providing one or more indications responsive to a correlation of the user interaction with the image,

wherein the one or more indications comprise displaying the image at a second location in the 3-D coordinate system.

16. The method of claim 15, wherein the image is a holograph.

17. The method of claim 15, wherein the user interaction with the image comprises movement of the user relative to the image, or any sound produced by the user, or both.

18. The method of claim 15, wherein sensing comprises using one or more laser sensors to geometrically identify a position within the 3-D coordinate system, or using a sound sensor to identify an audio input from the user.

19. The method of claim 15, wherein sensing comprises using a sound sensor to identify an audio input from the user.

20. The method of claim 15, wherein the image at the first location ceases to be displayed before, when or after the image at the second location starts to be displayed.

21. The method of claim 15, wherein the distance between the first location and the second location are within a range such that a sequential display of the image at the first location and the second location can be thought of as a movement of the image from the first location to the second location;

and

wherein the user interaction comprises movement of the whole body or one or more body parts of the user from the first location to the second location, thereby creating an effect of the image moving from the first location to the second location responsive to the movement of the whole body or the one or more body parts of the user from the first location to the second location.

22. The method of claim 21, wherein the image is displayed at the second location immediately after the whole body or the one or more body parts of the user moves to the second location, thereby creating an effect of the image moving with the whole body or one or more body parts of the user.

23. The method of claim 15, wherein the image is that of an input device for a computer system, and the one or more indications further comprise inputting data to the computer system responsive to the correlation of the user interaction with the image, and

the input device including any of a touch screen, a touch pad, a digitizing tablet, a haptic device, a mouse, track balls, light guns, steering wheels, yokes, joysticks, directional pads, wired gloves, game controllers, game pads, game paddles and remote control devices.

24. A non-transitory computer readable medium having computer readable program codes embodied therein for causing a computer to provide a 3 dimensional (3-D) user interface, the computer readable program codes causing the computer to:

generate an image at a first location in a 3-D coordinate system, the image being of any one of;

an input device, a touch screen, a touch pad, a digitizing tablet, a haptic device, a mouse, track balls, light guns, steering wheels, yokes, joysticks, directional pads, wired gloves, game controllers, game pads, game paddles and remote control devices;

apply quadrilateral angle navigation and sense user interaction with the image;

correlate the user interaction with the image; and

provide one or more indications responsive to a correlation of the user interaction with the image,

wherein the one or more indications comprise generating the image at a second location in the 3-D coordinate system.