Gestural input for navigation and manipulation in virtual space

• US 20070222746A1
• Filed: 03/23/2006
• Published: 09/27/2007
• Est. Priority Date: 03/23/2006
• Status: Abandoned Application

• Current Assignee: Accenture Global Services GmbH (Accenture PLC)
• Original Assignee: Accenture Global Services GmbH (Accenture PLC)
• Inventors: LeVine, Richard
• Application Number:
  US11/387,404
• Publication Number:
  US 20070222746A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A gestural input system provides an intuitive man-machine interface that converts human locomotion in free space, independent of fiducials and references surfaces to data that may be processed by a controller. A gestural input translates movement of a human and/or manipulation of a device in free space to data or signals for navigating digital content such as a 2D or 3D virtual or augmented reality.

Claims

1. A handheld electronic device having user interface, the handheld device comprising:

(a) a sensor that generates information associated with an environmental condition for the handheld electronic device, changes in the environmental condition resulting, at least in part, based on movement of the handheld device relative to the environment;

(b) a controller that processes the data generated by the sensor

(c) a computer program executable by the controller for;

tracking movement of the handheld device in three-dimensional free space according to the data generated by the sensor;

translating the tracked movement of the device to a corresponding movement of digital content; and

displaying the corresponding movement in digital content via the user interface.

2. The handheld electronic device of claim 1 where the sensor is a digital camera.

3. The handheld electronic device of claim 2 where the digital camera generates data representative of a sequence of picture frames of the environment.

4. The handheld electronic device of claim 3 where the program executable by the controller includes detecting movement of the handheld electronic device according to detecting changes of objects between picture frames in the sequence of picture frames.

5. The handheld electronic device of claim 4 where the program executable by the controller includes detecting movement of the handheld electronic device according to visual edge detection.

6. The handheld electronic device of claim 4 where the program executable by the controller includes detecting movement of the handheld electronic device according to visual image interpolation.

7. The handheld electronic device of claim 1 where the changes in the environmental condition result, at least in part, on any one of changes in motion, acceleration, distance, velocity, and direction of motion of the handheld electronic device.

8. The handheld electronic device of claim 7 where the information generated by the sensor associated with an environmental condition for the handheld detects changes in the environmental conditions based on any one of inertial tracking, dead reckoning, infra-red tracking, radio frequency triangulation, reflective radio frequency technology (RADAR), and reflective sound technology (SONAR).

9. The handheld electronic device of claim 1 where the program executable by the controller includes identifying a sequence of movements of the handheld device and translating the sequence of movements to a programmed instruction to be performed by the controller.

10. The handheld electronic device of claim 1 where the program executable by the controller includes identifying a sequence of movements of the handheld device and translating the sequence of movements to a programmed instruction to be performed by the controller.

11. The handheld electronic device of claim 1 where the movement in digital content is scaled to the tracked movement of the handheld device.

12. The handheld electronic device of claim 11 where the movement in digital content is scaled at a ratio of substantially 1:

1 to the tracked movement of the handheld device.

13. The handheld electronic device of claim 11 where the movement in digital content is scaled at a ratio of less than about 1:

1 to the tracked movement of the handheld device.

14. The handheld electronic device of claim 11 where the movement in digital content is scaled at a ratio of greater than 1:

1 to the tracked movement of the handheld device.

15. The handheld electronic device of claim 1 where the digital content is a virtual reality application.

16. The handheld electronic device of claim 15 where the virtual reality application is a gaming application.

17. The handheld electronic device of claim 15 where the handheld device comprises a telephone.

18. The handheld electronic device of claim 15 where the handheld device comprises a personal digital assistant.

19. The handheld electronic device of claim 15 further comprising at least one data input selected from the group consisting of a joystick, stylus, trackball, wheel, roller, touch screen, mouse, and a touchpad.

20. A gestural input device, comprising:

a camera configured to capture a sequence of picture frames of a field of view of the camera;

means for tracking features captured in the picture frames;

means for converting tracked features captured in the picture frames to movement of the input device; and

means for translating movement of the input device to movement in digital content.

21. The device of claim 20, wherein the means for tracking features captured in the picture frames comprises visual edge detection of objects captured in the sequence of picture frames.

22. The device of claim 21, where the means for tracking features captured in the picture frames comprises identifying coordinate differences between picture frames of the sequence of picture frames of edges of objects captured in the picture frames.

23. The device of claim 20, where the means for tracking features captured in the picture frames comprises visual image interpolation of objects captured in the sequence of picture frames.

24. The apparatus of claim 20, further comprising:

means for identifying a sequence of movements of the device; and

means for executing a programmed function according to identifying the sequence of movements.

25. The apparatus of claim 24, where the sequence of movements represents a user'"'"'s personal identifier.

26. The apparatus of claim 20, further comprising a means for wirelessly communicating the translated movement of the input device to movement in digital content.

27. The apparatus of claim 20, further comprising means for displaying the translated movement of the input device in digital content.

28. A computer-readable medium having computer-executable instructions stored therein, the computer-executable instructions causing a controller to execute:

receiving data representative of changes in environmental conditions affected by a motion of the controller in free space in the environment;

identifying the motion of the controller according to the data; and

translating the motion of the controller to an instruction to be executed by the controller.

29. The computer-readable medium of claim 28 further causing a controller to execute:

recognizing a sequence of movements of the controller; and

identifying a programmed sequence of instructions to be executed by the controller.

30. The computer-readable medium of claim 28 further causing a controller to wirelessly transmitting translating the instruction.