Sensor-based distributed tangible user interface

• US 20090273560A1
• Filed: 02/04/2009
• Published: 11/05/2009
• Est. Priority Date: 02/04/2008
• Status: Abandoned Application

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Merrill, David Jeffrey, Maes, Patricia Emilia, Kalanithi, Jeevan James
• Application Number:
  US12/365,885
• Publication Number:
  US 20090273560A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information

Abstract

A distributed tangible user interface comprises compact, self-powered, tangible user interface manipulative devices having sensing, display, and wireless communication capabilities, along with one or more associated digital content or other interactive software management applications. The manipulative devices display visual representations of digital content or program controls and can be physically manipulated as a group by a user for interaction with the digital information or software application. A controller on each manipulative device receives and processes data from a movement sensor, initiating behavior on the manipulative and/or forwarding the results to a management application that uses the information to manage the digital content, software application, and/or the manipulative devices. The manipulative devices may also detect the proximity and identity of other manipulative devices, responding to and/or forwarding that information to the management application, and may have feedback devices for presenting responsive information to the user.

Claims

1. A tangible user interface, comprising:

a plurality of tangible user interface manipulative devices, each tangible user interface manipulative device being independently manipulable relative to the other tangible user interface manipulative devices, each tangible user interface manipulative device comprising;

at least one wireless communications device;

a visual display for digital content;

a power source;

at least one movement sensor; and

at least one controller adapted for;

receiving data from the movement sensor;

processing the received data to derive movement parameters; and

at least one of forwarding the derived movement parameters or initiating tangible user interface behaviour in response to the derived movement parameters; and

at least one management application, the management application being adapted for;

sending digital content or behavior instructions to individual tangible user interface manipulative devices;

receiving derived movement parameters from at least one of the tangible user interface manipulative devices;

processing derived movement parameters to derive instructions about management of the digital content or program behavior; and

changing program behavior or managing the digital content according to the derived instructions.

2. The tangible user interface of claim 1, the management application being further adapted for sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived instructions.

3. The tangible user interface of claim 1, the tangible user interface manipulative devices further comprising

at least one neighborhood wireless communications device for sensing nearby tangible user interface manipulative devices;

and

the at least one controller being further adapted for;

sensing, using the neighborhood wireless communication device, the position of at least one nearby tangible user interface manipulative device;

processing the position of, and any communication received from, the sensed nearby tangible user interface manipulative device in order to derive neighborhood information; and

at least one of forwarding the derived neighbourhood information to the management application or initiating tangible user interface behavior in response to the derived neighbourhood information.

4. The tangible user interface of claim 3, the management application being further adapted for sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived instructions.

5. The tangible user interface of claim 1, the tangible user interface manipulative devices further comprising at least one feedback device for presenting responsive information to a user.

6. The tangible user interface of claim 3, the tangible user interface manipulative devices further comprising at least one feedback device for presenting responsive information to a user.

7. The tangible user interface of claim 1, wherein the management application resides on a tangible user interface device.

8. The tangible user interface of claim 3, wherein the management application resides on a tangible user interface device.

9. The tangible user interface of claim 1, wherein the movement sensor is an accelerometer.

10. The tangible user interface of claim 1, the tangible user interface manipulative devices further comprising memory for storing at least one of digital content or program instructions.

11. A tangible user interface manipulative device, comprising

a visual display;

at least one wireless communications device, the wireless communications device being adapted for receiving behaviour commands or digital content for display on the visual display;

a power source;

at least one movement sensor; and

at least one controller adapted for;

receiving data from the movement sensor; and

at least one of;

processing the received data to derive movement parameters;

initiating behaviour as a result of the data or movement parameters; and

forwarding the derived movement parameters or the received data using the wireless communications device.

12. The tangible user interface manipulative device of claim 11, further comprising:

at least one neighborhood wireless communications device for sensing nearby tangible user interface manipulative devices; and

the at least one controller being further adapted for;

sensing, using the neighborhood wireless communication device, the position of at least one nearby tangible user interface manipulative device;

processing the position of, and any communication received from, the sensed nearby tangible user interface manipulative device in order to derive neighborhood information; and

at least one of forwarding the derived neighbourhood information using the wireless communications device or initiating tangible user interface behavior in response to the derived neighbourhood information.

13. The tangible user interface manipulative device of claim 11, wherein the derived movement parameters are forwarded to a computing device.

14. The tangible user interface manipulative device of claim 12, wherein the derived movement parameters and derived neighborhood information are forwarded to a computing device.

15. The tangible user interface manipulative device of claim 11, further comprising at least one feedback device for presenting responsive information to a user.

16. The tangible user interface manipulative device of claim 11, wherein the movement sensor is an accelerometer.

17. The tangible user interface manipulative devices of claim 11, further comprising memory for storing at least one of digital content or program instructions.

18. A method for facilitating user interaction with digital content or application programs, comprising the steps of:

displaying a visual representation of at least one of digital content or program control elements on a plurality of tangible user interface manipulative devices, such that a subset of the digital content or program control elements is displayed on any individual device;

detecting at least one of a manipulation of at least one of the tangible user interface manipulative devices or a location-based relationship between at least two of the tangible user interface manipulative devices; and

deriving digital content relationship information or instructions from the detected manipulation or relationship.

19. The method of claim 18, further comprising the step of forwarding the derived digital content relationship information or instructions to a computing device.

20. The method of claim 19, further comprising the step of managing the digital content or application program behavior according to the derived digital content relationship information or instructions received by the computing device.

21. The method of claim 20, further comprising the step of sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived digital content relationship information or instructions received by the computing device.

22. The method of claim 19, further comprising the steps of:

processing the derived digital content relationship information or instructions received by the computing device; and

managing the digital content or application program behavior according to the processed digital content relationship information or instructions.

23. The method of claim 22, further comprising the step of sending revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the processed digital content relationship information or instructions.

24. The method of claim 18, further comprising the step of presenting responsive information to a user via the tangible user interface manipulative device.

25. The method of claim 18, wherein the instructions relate to management of the digital content or program behavior.

26. The method of claim 18, wherein the instructions relate to management of the behavior of the tangible user interface manipulative device.