Spatial, multi-modal control device for use with spatial operating system

• US 8,669,939 B2
• Filed: 05/27/2010
• Issued: 03/11/2014
• Est. Priority Date: 02/08/2006
• Status: Active Grant

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Underkoffler, John S., Sparrell, Carlton, Belker, Harald, Kramer, Kwindla Hultman
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US12/789,262
• Publication Number:
  US 20110025598A1
• Time in Patent Office: 1,384 Days
• Field of Search: 345156-169, 345173-179, 345204-215, 178 1801- 1809, 178 1901- 1907, 178 2001- 2004
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A system comprising an input device includes a detector coupled to a processor. The detector detects an orientation of the input device. The input device has multiple modal orientations corresponding to the orientation. The modal orientations correspond to multiple input modes of a gestural control system. The detector is coupled to the gestural control system and automatically controls selection of an input mode in response to the orientation.

Claims

1. An input device comprising:

a processor coupled to a detector and contained in a housing having a hand-held form factor, wherein the detector detects and translates into input signals at least one of an orientation in which the housing is currently operated and a position of the housing, wherein the detector detects the at least one of the orientation and the position using signals induced in coils; and

a transmitter coupled to the processor, wherein the transmitter communicates the input signals to a gestural control system, wherein the input signals automatically control a plurality of input modes of the gestural control system and control translation and rotation of a three-space object coupled to the gestural control system;

wherein a first side of a plurality of sides corresponds to a first input mode and a second side of a plurality of sides corresponds to a second input mode, wherein a transition comprising rotating the input device around a longitudinal axis more than 120 degrees relative to a center of the first side causes a change from the first input mode to the second input mode.

2. An input device comprising a hand-held housing that includes a processor, wherein the processor is coupled to a detector that detects and translates into input signals an orientation in which the input device is currently operated and transitions between a plurality of orientations of the input device, wherein the input signals automatically control a plurality of input modes of a gestural control system and control a three-space object coupled to the gestural control system, wherein the input signals control the three-space object through three translational degrees of freedom and three rotational degrees of freedom;

wherein a first side of a plurality of sides corresponds to a first input mode and a second side of a plurality of sides corresponds to a second input mode, wherein a transition comprising rotating the input device around a longitudinal axis more than 120 degrees relative to a center of the first side causes a change from the first input mode to the second input mode.

3. An input device comprising a hand-held housing that includes a processor, wherein the processor is coupled to a detector that detects and translates into input signals an orientation in which the input device is currently operated and transitions between a plurality of orientations of the input device, wherein the input signals automatically control a plurality of input modes of a gestural control system;

wherein a first side of a plurality of sides corresponds to a first input mode and a second side of a plurality of sides corresponds to a second input mode, wherein a transition comprising rotating the input device around a longitudinal axis more than 120 degrees relative to a center of the first side causes a change from the first input mode to the second input mode.

4. The device of claim 3, wherein the detector comprises circuitry including a plurality of coils.

5. The device of claim 4, comprising a remote electromagnetic field (EMF) generator that generates an EMF that induces signals in the plurality of coils.

6. The device of claim 5, comprising a plurality of remote EMF generators, wherein each EMF generator induces signals in the plurality of coils when the housing is proximate to the EMF generator that induces the signals.

7. The device of claim 5, wherein the detector detects the orientation using the EMF signals induced in the plurality of coils.

8. The device of claim 7, wherein the detector detects a position of the input device using the signals induced in the plurality of coils.

9. The device of claim 4, comprising a transmitter coupled to the processor, wherein the transmitter is a wireless transmitter.

10. The device of claim 9, wherein the transmitter communicates the orientation to the gestural control system.

11. The device of claim 9, wherein the transmitter communicates a position of the input device to the gestural control system.

12. The device of claim 9, comprising at least one input sensor coupled to the processor, wherein the transmitter communicates state of the at least one input sensor to the gestural control system.

13. The device of claim 12, wherein the detector translates raw input sensor position data into input sensor state.

14. The device of claim 13, wherein position of the input sensor controls the gestural control system.

15. The device of claim 3, wherein the plurality of orientations corresponds to the plurality of input modes of a gestural control system.

16. The device of claim 15, wherein the plurality of input modes comprise a direct manipulation mode in which the input device directly manipulates of an application element of a component coupled to the gestural control system.

17. The device of claim 15, wherein the plurality of input modes comprise a meta-manipulation mode in which the input device directly manipulates a set of application elements of a component coupled to the gestural control system.

18. The device of claim 15, wherein the plurality of input modes comprise a three-dimensional manipulation mode in which the input device controls three-dimensional manipulation of an application element of a component coupled to the gestural control system.

19. The device of claim 3, wherein the input signals automatically control selection of an input mode of the plurality of input modes in response to the orientation.

20. The device of claim 19, comprising applying hysteresis to selection of the input mode during the transitions.

21. The device of claim 3, wherein the transitions are rotational transitions about at least one axis of the input device.

22. The device of claim 3, wherein the detector tracks in real-time instantaneous three-space orientation at a point in time and space.

23. The device of claim 22, wherein the detector translates raw tracking data of the input device into six degrees of spatial orientation.

24. The device of claim 3, wherein the detector tracks in real-time instantaneous three-space position at a point in time and space.

25. The device of claim 24, wherein the detector translates raw tracking data of the input device into six degrees of spatial position.

26. The device of claim 3, wherein the housing comprises a plurality of sides, wherein each side of the plurality of sides corresponds to an orientation of the plurality of orientations.

27. The device of claim 26, wherein each side of the plurality of sides corresponds to an input mode of the plurality of input modes.

28. The device of claim 27, wherein each side of the plurality of sides is assigned an input mode to which the side corresponds.

29. The device of claim 27, wherein each side of the plurality of sides is dynamically assigned an input mode based on a context.

30. The device of claim 26, wherein at least one side of the plurality of sides comprises an input sensor.

31. The device of claim 26, wherein each side of the plurality of sides comprises an input sensor.

32. The device of claim 31, wherein each input sensor of each side is dynamically assigned a function based on the orientation.

33. The device of claim 31, wherein each input sensor of each side is dynamically assigned a function based on a context.

34. The device of claim 26, wherein the plurality of sides includes three sides, wherein the housing has a triangular cross-section, wherein each side of the plurality of sides corresponds to an orientation of the plurality of orientations.

35. The device of claim 20, wherein the detector applies a hysteresis band to the selection of the input mode during the transition.

36. The device of claim 35, wherein the hysteresis band is at least one of equal to and greater than approximately 30 degrees.

37. The device of claim 35, wherein the hysteresis band is programmable.

38. The device of claim 3, wherein the detector detects an event of the input device, wherein the event corresponds to at least one of a three-space location and a three-space orientation of the input device, wherein the event corresponds to an application of a first type.

39. The device of claim 38, wherein the processor generates data sequences comprising input device event data specifying the event and state information of the event, wherein the input device event data and state information are type-specific data having a type corresponding to a first application of the gestural control system.

40. The device of claim 39, wherein the processor forms a data capsule to include the data sequences, wherein the data capsule has a data structure comprising an application-independent representation of the data sequences.

41. The device of claim 40, wherein the processor places the data capsule in a repository.

42. The device of claim 41, wherein a second event running under an application of a second type searches the repository and identifies a correspondence between the data capsule and the second event, wherein the application of the second type executes an operation corresponding to the second event using contents of the data sequences of the data capsule.

43. The device of claim 3, wherein the gestural control system controls a three-space object coupled to the gestural control system.

44. The device of claim 43, comprising controlling via the device the three-space object through three translational degrees of freedom and three rotational degrees of freedom.

45. The device of claim 44, wherein the controlling comprises a direct coupling between motion of the input device and the three-space object.

46. The device of claim 44, wherein the controlling includes an indirect coupling between motion of the input device and the three-space object.

47. The device of claim 44, wherein the three-space object is presented on a display device coupled to the processor.

48. The device of claim 44, comprising controlling movement of the three-space object by mapping the plurality of input modes of the input device to a plurality of object translations of the three-space object.

49. The device of claim 48, wherein the mapping includes a direct mapping between the plurality of input modes and the plurality of object translations.

50. The device of claim 48, wherein the mapping includes an indirect mapping between the plurality of input modes and the plurality of object translations.

51. The device of claim 48, wherein the mapping includes correlating positional offsets of the plurality of input modes to positional offsets of the object translations of the three-space object.

52. The device of claim 48, wherein the mapping includes correlating positional offsets of the input device to translational velocity of the object translations of the three-space object.

53. The device of claim 44, comprising controlling movement of the three-space object by mapping a linear gesture of the input device to a linear translation of the three-space object.

54. The device of claim 44, comprising controlling movement of the three-space object by mapping a rotational gesture of the input device to a rotational translation of the three-space object.

55. The device of claim 44, comprising controlling movement of the three-space object by mapping a linear gesture of the input device to a rotational translation of the three-space object.

56. The device of claim 44, comprising controlling movement of the three-space object by mapping a rotational gesture of the input device to a linear translation of the three-space object.

57. The device of claim 44, wherein the detecting comprises detecting when an extrapolated position of the input device intersects virtual space, wherein the virtual space comprises space depicted on a display device coupled to the gestural control system.