Control system for navigating a principal dimension of a data space

• US 8,537,112 B2
• Filed: 09/03/2009
• Issued: 09/17/2013
• Est. Priority Date: 02/08/2006
• Status: Expired due to Fees

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Underkoffler, John S., Kramer, Kwindla H.
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US12/553,929
• Publication Number:
  US 20100060576A1
• Time in Patent Office: 1,475 Days
• Field of Search: 345156-178, 178 1801- 1809, 178 2001- 2004
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Systems and methods are described for navigating through a data space. The navigating comprises detecting a gesture of a body from gesture data received via a detector. The gesture data is absolute three-space location data of an instantaneous state of the body at a point in time and physical space. The detecting comprises identifying the gesture using the gesture data. The navigating comprises translating the gesture to a gesture signal, and navigating through the data space in response to the gesture signal. The data space is a data-representational space comprising a dataset represented in the physical space.

Claims

1. A method for navigating through a data space, the method comprising:

detecting a gesture of a body from gesture data received via a detector, wherein the gesture data is absolute three-space location data of an instantaneous state of the body at a point in time and physical space, the detecting comprising identifying the gesture using only the gesture data;

translating the gesture to a gesture signal;

navigating through the data space in response to the gesture signal, wherein the data space is a data-representational space comprising a dataset represented in the physical space; and

rendering the dataset in a plurality of data frames that are graphical depictions of a plurality of regions of the data space and displaying each data frame as a visible frame on a display, wherein the plurality of data frames comprises a stack of discrete data frames arranged in a plurality of planes of the data space, wherein each data frame lies in a separate plane of the plurality of planes.

2. The method of claim 1, comprising:

detecting a first pose of the body;

activating pushback interaction in response to detecting the first pose.

3. The method of claim 2, comprising:

recording a first position at which the first pose is entered, wherein the first position is a three-space hand position;

setting the first position as an origin, wherein subsequent detected body positions are reported as relative offsets to the origin.

4. The method of claim 3, wherein the detecting comprises detecting an upward movement of the body, wherein the upward movement is movement along a y-axis outward from the display, wherein the y-axis is defined as an axis normal to a view surface of the display.

5. The method of claim 4, wherein, in response to the upward movement of the body, the navigating comprises displacing the plurality of data frames along the y-axis from a current data frame, wherein the discrete data frames of the stack translate upward, wherein the current data frame is the discrete data frame currently rendered on the display.

6. The method of claim 5, wherein an appearance of each discrete data frame varies in response to the displacing, wherein discrete data frames of the stack positioned above the current data frame take on a relatively high degree of transparency and blur, wherein the current data frame transitions from visible and begins taking on a degree of transparency and blur, wherein discrete data frames of the stack positioned below the current data frame take on a relatively lower degree of transparency and blur and start to become visible.

7. The method of claim 6, comprising a mapping between a physical distance of displacement of the body during a gesture and logical distance between successive discrete data frames of the stack, wherein the mapping controls the displacing of the plurality of data frames along the y-axis and the appearance of each discrete data frame.

8. The method of claim 7, wherein the detecting comprises detecting a downward movement of the body, wherein the downward movement is movement along the y-axis toward the display.

9. The method of claim 8, wherein, in response to the downward movement of the body, the navigating comprises displacing the plurality of data frames along the y-axis from the current data frame, wherein the discrete data frames of the stack translate downward.

10. The method of claim 9, wherein an appearance of each discrete data frame varies in response to the displacing, wherein discrete data frames of the stack positioned below the current data frame take on a relatively high degree of transparency and blur, wherein the current data frame transitions from visible and begins taking on a degree of transparency and blur, wherein discrete data frames of the stack positioned above the current data frame take on a relatively lower degree of transparency and blur and become visible.

11. The method of claim 10, comprising a mapping between a physical distance of displacement of the body during a gesture and logical distance between successive discrete data frames of the stack, wherein the mapping controls the displacing of the plurality of data frames along the y-axis and the appearance of each discrete data frame.

12. The method of claim 11, comprising continuously updating a displacement along the y-axis of the plurality of data frames in direct response to movement of the body along the y-axis.

13. The method of claim 12, comprising:

detecting a second pose of the body;

terminating pushback interaction in response to detecting the second pose, wherein the terminating comprises displaying a data frame of the plurality of data frames as coplanar with the display.

14. The method of claim 1, wherein the data space comprises a plurality of virtual detents.

15. The method of claim 14, wherein each virtual detent corresponds to each plane of the plurality of planes.

16. The method of claim 1, comprising forming a gestural interaction space comprising a first active zone, a second active zone, and a dead zone, wherein the first active zone is adjacent the display, the dead zone is adjacent the first active zone, and the second active zone is adjacent the dead zone.

17. The method of claim 16, wherein the navigating through the data space in response to the gesture signal is activated in response to the gesture when the gesture is detected in one of the first active zone and the second active zone.

18. The method of claim 16, comprising a feedback indicator rendered on the display.

19. The method of claim 18, wherein the feedback indicator displays feedback indicating the body is in one of the first active zone, the second active zone, and the dead zone.

20. The method of claim 18, wherein the feedback indicator displays feedback indicating a physical offset of the body from the origin.

21. The method of claim 1, comprising aligning a parameter-control axis of the dataset with a dimension of the physical space.

22. The method of claim 21, wherein the dimension is a depth dimension.

23. The method of claim 21, wherein the dimension is a horizontal dimension.

24. The method of claim 21, wherein the dimension is a vertical dimension.

25. The method of claim 21, wherein the dimension is a lateral dimension.

26. The method of claim 21, wherein the navigating comprises motion along the dimension to effect a data-space translation along the parameter-control axis.

27. The method of claim 21, wherein the navigating comprises navigating to quantized parameter spaces of the data space.

28. The method of claim 1, wherein the detecting includes detecting an evolving position of the body.

29. The method of claim 1, wherein the detecting includes detecting an evolving orientation of the body.

30. The method of claim 1, wherein the detecting includes detecting an evolving pose of the body, wherein the pose is a geometric disposition of a part of the body relative to at least one other part of the body.

31. The method of claim 1, wherein the detecting includes detecting evolving motion of the body.

32. The method of claim 1, wherein the detecting includes detecting at least one of an evolving position of the body, orientation of the body, pose of the body, and motion of the body.

33. The method of claim 1, comprising analyzing the gesture into a sequence of gestural events.

34. The method of claim 33, comprising identifying the gesture.

35. The method of claim 34, wherein the identifying of the gesture includes identifying at least one of an evolving position of the body, orientation of the body, pose of the body, and motion of the body.

36. The method of claim 34, comprising generating a representation of the gestural events of the sequence of gestural events.

37. The method of claim 36, comprising distributing the representation of the gestural events to at least one control component coupled to the data space.

38. The method of claim 37, comprising synchronizing the representation of the gestural events with a graphical depiction of the data space.

39. The method of claim 38, comprising synchronizing the representation of the gestural events with a graphical depiction of the navigating through the data space.

40. The method of claim 37, comprising synchronizing the representation of the gestural events with an aural depiction of the data space.

41. The method of claim 40, comprising synchronizing the representation of the gestural events with an aural depiction of the navigating through the data space.

42. The method of claim 1, wherein the dataset represents spatial information.

43. The method of claim 42, wherein the dataset represents spatial information of at least one of phenomena, events, measurements, observations, and structure.

44. The method of claim 1, wherein the dataset represents non-spatial information.

45. The method of claim 1, wherein the gesture comprises linear spatial motion.

46. The method of claim 1, wherein the navigating comprises linear verging through the data space.

47. The method of claim 1, comprising:

rendering the dataset in a plurality of data frames that are graphical depictions of a plurality of regions of the data space;

displaying each data frame as a visible frame on a display.

48. The method of claim 47, wherein a size and an aspect ratio of the data frame coincide with the size and the aspect ratio of the display.

49. The method of claim 47, wherein a center and a normal vector of the data frame coincide with the center and the normal vector of the display.

50. The method of claim 47, wherein a position and an orientation of the data frame coincide with the position and the orientation of the display.

51. The method of claim 47, wherein each data frame comprises graphical data elements representing elements of the dataset.

52. The method of claim 51, wherein the graphical data elements are static elements.

53. The method of claim 51, wherein the graphical data elements are dynamic elements.

54. The method of claim 47, wherein the data frame is a two-dimensional construct.

55. The method of claim 54, wherein the data frame is resident in a three-dimensional graphics rendering environment having a coordinate system that coincides with coordinates that describe a local environment that includes the body.

56. The method of claim 47, wherein the navigating through the data space comprises navigating through the plurality of data frames.

57. The method of claim 1, comprising identifying the gesture, wherein the identifying includes identifying a pose and an orientation of a portion of the body.

58. The method of claim 1, wherein the detecting includes detecting at least one of a first set of appendages and a second set of appendages of the body.

59. The method of claim 1, wherein the detecting includes dynamically detecting a position of at least one tag.

60. The method of claim 1, wherein the detecting includes dynamically detecting and locating a marker on the body.

61. The method of claim 1, wherein the translating comprises translating information of the gesture to a gesture notation.

62. The method of claim 61, wherein the gesture notation represents a gesture vocabulary, and the gesture signal comprises communications of the gesture vocabulary.

63. The method of claim 62, wherein the gesture vocabulary represents in textual form instantaneous pose states of the body.

64. The method of claim 62, wherein the gesture vocabulary represents in textual form an orientation of the body.

65. The method of claim 62, wherein the gesture vocabulary represents in textual form a combination of orientations of the body.

66. The method of claim 62, wherein the gesture vocabulary includes a string of characters that represent a state of the body.

67. The method of claim 1, wherein the detecting comprises detecting when an extrapolated position of the body intersects virtual space, wherein the virtual space comprises space depicted on the display.

68. The method of claim 67, comprising controlling a virtual object in the virtual space when the extrapolated position intersects the virtual object.

69. The method of claim 68, comprising controlling a position of the virtual object in the virtual space in response to the extrapolated position in the virtual space.

70. The method of claim 1, comprising controlling scaling of the detecting and navigating to generate coincidence between virtual space and physical space, wherein the virtual space comprises space depicted on the display, wherein the physical space comprises space inhabited by the body.

71. The method of claim 1, comprising imaging the body with an imaging system.

72. The method of claim 71, wherein the imaging comprises generating wavefront coded images of the body.

73. The method of claim 72, wherein the gesture data comprises focus-resolved data of the body within a depth of field of the imaging system.

74. The method of claim 73, comprising generating intermediate images by coding images gathered by the imaging system.

75. The method of claim 74, wherein the intermediate images are blurred.

76. The method of claim 74, wherein the intermediate images are insensitive to changes in at least one of the body and a plurality of optical detectors of the imaging system that include defocus aberrations.

77. The method of claim 73, wherein the gesture data comprises focus-resolved range data of the body within the depth of field.

78. The method of claim 77, wherein the focus-resolved range data of the body within the depth of field is derived from an output of the imaging system.

79. The method of claim 73, wherein the gesture data comprises focus-resolved position data of the body within the depth of field.

80. The method of claim 79, wherein the focus-resolved position data of the body within the depth of field is derived from an output of the imaging system.

81. The method of claim 71, wherein the imaging system comprises a plurality of detectors.

82. The method of claim 81, wherein at least two of the detectors are wavefront coded cameras comprising a wavefront coding optical element.

83. The method of claim 81, wherein at least two of the optical detectors are wavefront coded cameras comprising a phase mask that increases a depth of focus of the imaging.

84. The method of claim 71, comprising generating modulation transfer functions and point spread functions that are invariant to a distance between the body and the imaging system.

85. The method of claim 71, comprising generating modulation transfer functions and point spread functions that are invariant with respect to defocus.

86. A system comprising:

a detector for receiving gesture data that represents a gesture made by a body; and

a processor coupled to the detector, the processor automatically detecting the gesture from the gesture data, wherein the gesture data is absolute three-space location data of an instantaneous state of the body at a point in time and physical space, the processor identifying the gesture using only the gesture data, the processor translating the gesture to a gesture signal, the processor controlling navigating through the data space in response to the gesture signal, wherein the data space is a data-representational space comprising a dataset represented in the physical space, the processor rendering the dataset in a plurality of data frames that are graphical depictions of a plurality of regions of the data space and displaying each data frame as a visible frame on a display, wherein the plurality of data frames comprises a stack of discrete data frames arranged in a plurality of planes of the data space, wherein each data frame lies in a separate plane of the plurality of planes.