Manipulating association of data with a physical object

• US 20050275636A1
• Filed: 11/22/2004
• Published: 12/15/2005
• Est. Priority Date: 06/15/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Drucker, Steven M., Dehlin, Joel P., Roseway, Asta J., Wong, Curtis Glenn
• Granted Patent: US 7,358,962 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Data elements stored in a computing system and associated with a physical object are reassociated with the same or another physical object. An identifying characteristic presented by the physical object, such as a reflective pattern applied to the object, is detected when the object is positioned adjacent to the interactive display surface. Images or other files associated with the identifying characteristic are accessed and displayed on the interactive display surface. A gesture by a user adjacent to the interactive display surface is detected to reassociate a selected representation. For example, the user can change where the selected element fits in a sequence of data elements, or reassociate the selected element with a second object placed on the interactive display surface. In a networked environment, the reassociated element can be stored on a server and subsequently accessed from a different interactive surface via the network using the second physical object.

Claims

1. A method for reassociating data elements associated with a physical object placed adjacent to an interactive display surface connected to a computing device, the method comprising the steps of:

(a) reading a characteristic presented by the physical object when the physical object is disposed adjacent to the interactive display surface;

(b) accessing the data elements associated with the characteristic from data storage;

(c) showing on the interactive display surface a representation depicting each of the data elements;

(d) detecting adjacent to the interactive display surface a user movement indicating a user command to reassociate a selected representation of a data element;

(e) reassociating the data element corresponding to the selected representation; and

(f) visually confirming reassociation of the data element by showing the selected representation being reassociated on the interactive display surface.

2. The method of claim 1, wherein the characteristic includes a light reflective identification disposed on a surface of the physical object.

3. The method of claim 1, wherein the step of reading the characteristic presented by the physical object includes the steps of:

(a) transmitting infrared light through the interactive display surface toward a face of the interactive display surface adjacent to which the physical object is disposed; and

(b) recognizing the characteristic presented by physical object by imaging infrared light reflected thereby.

4. The method of claim 1, wherein the representation of each of the data elements includes one of:

(a) a content of the data element;

(b) a portion of the content of the data element;

(c) a name representing the data element; and

(d) an icon representing the data element.

5. The method of claim 4, wherein a manner of presenting the representation is determined by a location where the physical object is placed on the interactive display surface.

6. The method of claim 5, wherein the location where the physical object is placed causes the data elements to be presented in one of:

(a) a sort mode, wherein the representation of each of the data elements is in a generally tabular fashion and sorted based upon a defined parameter;

(b) a carousel mode wherein the representation of each of the data elements is presented in a sequentially-related list; and

(c) a show mode wherein the representation of each of the data elements is presented individually and sequentially.

7. The method of claim 1, wherein the user movement includes one of:

(a) a hand gesture;

(b) a movement of a pointing object presenting a pointing identifier; and

(c) a movement of the physical object.

8. The method of claim 1, wherein the step of reassociating the data element corresponding to the selected representation includes resequencing the data elements associated with the characteristic.

9. The method of claim 1, wherein the step of reassociating the data element corresponding to the selected representation includes the step of reassociating the data with a second characteristic presented by a second physical object placed adjacent to the interactive display surface.

10. The method of claim 9, further comprising the step of determining that the second characteristic is placed adjacent to a designated portion of the interactive display surface to cause the selected representation to be reassociated with the second characteristic.

11. The method of claim 9, wherein the step of reassociating the data element with the second characteristic includes one of the steps of:

(a) copying the data from a first storage location associated with the characteristic to a second storage location associated with the second characteristic; and

(b) moving the data from the first storage location associated with the characteristic to the second storage location associated with the second characteristic.

12. The method of claim 9, wherein the second characteristic is associated with one of:

(a) a local data storage associated with the computing device associated with the interactive display surface; and

(b) a remote data storage on a server accessible by a network such that the data elements associated with the second identifier are accessed via one of;

(i) the interactive display surface; and

(ii) a second interactive display surface in communication with the remote data storage on the server.

13. The method of claim 1, wherein the step of showing the selected representation being reassociated on the interactive display surface includes the step of presenting a reassociation animation between the selected representation and a reassociation point, said reassociation animation including one of the steps of:

(a) the reassociation point pulling the selected representation into the reassociation point;

(b) the selected representation vanishing into the reassociation point;

(c) the selected representation shrinking into the reassociation point; and

(d) the selected representation curving into the reassociation point.

14. The method of claim 13, further comprising the steps of identifying a type of user movement;

and determining the reassociation animation presented depending on the type of user movement that was identified.

15. The method of claim 1, further comprising the step of performing an opening animation visually indicating the emergence of the representations from an entry point associated with the characteristic.

16. The method of claim 1, further comprising the steps of:

(a) enabling a user to select an undo command to undo the reassociation of the data element;

(b) detecting selection of the undo command by the user; and

(c) undoing the reassociating of the data element in response to detecting that the undo command was selected by the user.

17. The method of claim 1, further comprising the steps of:

(a) detecting when the characteristic is no longer adjacent to the interactive display surface; and

(b) disabling a capability for reassociating the data elements associated with the characteristic.

18. A memory medium having machine executable instructions stored for carrying out the steps of claim 1.

19. A method for reassociating data elements associated with a first physical object placed adjacent to an interactive display surface connected to a computing device, the method comprising the steps of:

(a) reading a first characteristic presented by the first physical object when the first physical object is disposed adjacent to the interactive display surface;

(b) accessing the data elements associated with the first characteristic from data storage;

(c) showing on the interactive display surface a representation depicting each of the data elements;

(d) reading a second characteristic presented by a second physical object when the physical object is disposed adjacent to the interactive display surface;

(e) detecting adjacent to the interactive display surface a user movement indicating a user command to reassociate a selected representation with the second characteristic; and

(f) reassociating the data elements depicted by each selected representation with the second characteristic presented by the second physical object.

20. The method of claim 19, wherein at least one of the first characteristic presented by the first physical object and the second characteristic presented by the second physical object includes a light reflective identification.

21. The method of claim 19, wherein the step of identifying at least one of the first characteristic and the second characteristic includes the steps of:

(a) transmitting infrared light through the interactive display surface toward a face of the interactive display surface adjacent to which the first characteristic and the second characteristic are disposed; and

(b) recognizing at least one of the first characteristic and the second characteristic disposed adjacent to the interactive display surface by imaging infrared light reflected thereby.

22. The method of claim 19, wherein the representation of each of the data elements includes one of:

(a) a content of the data element;

(b) a portion of the content of the data element;

(c) a name representing the data element; and

(d) an icon representing the data element.

23. The method of claim 22, wherein a manner of presenting the representation is determined by a location where the first physical object is placed on the interactive display surface.

24. The method of claim 23, wherein the location where the first physical object is placed causes the data elements to be presented in one of:

(a) a sort mode wherein the representation of each of the data elements is in a generally tabular fashion, sorted in regard to a defined parameter;

(b) a carousel mode wherein the representation of each of the data elements is presented in a sequentially-related list; and

(c) a show mode wherein the representation of each of the data elements is presented individually and sequentially.

25. The method of claim 19, wherein the user movement includes one of:

(a) a hand gesture;

(b) a movement of a pointing object presenting a pointing identifier; and

(c) a movement of the physical object.

26. The method of claim 19, wherein the step of reassociating the data element with the second characteristic includes one of the steps of:

(a) copying the data from a first storage location associated with the first characteristic to a second storage location associated with the second characteristic; and

(b) moving the data from the first storage location associated with the first characteristic to the second storage location associated with the second characteristic.

27. The method of claim 19, wherein the second characteristic is associated with one of:

(a) a local data storage associated with the computing device that is associated with the interactive display surface; and

(b) a remote data storage on a server accessible over a network, such that the data elements associated with the second identifier are accessed via one of;

(i) the interactive display surface; and

(ii) a second interactive display surface that is in communication with the remote data storage on the server.

28. The method of claim 19, further comprising the step of visually confirming reassociation of the data element with the second characteristic by showing the selected representation being reassociated with the second characteristic presented by the second physical object on the interactive display surface.

29. The method of claim 28, wherein the step of showing the selected representation being reassociated with a reassociation point associated with the second characteristic presented by the second physical object on the interactive display surface includes carrying out one of the steps of:

(a) the reassociation point pulling the selected representation into the second characteristic presented by the second physical object;

(b) the selected representation vanishing into the reassociation point;

(c) the selected representation shrinking into the reassociation point; and

(d) the selected representation curving into the reassociation point.

30. The method of claim 29, further comprising the steps of identifying a type of user movement;

and determining the reassociation animation presented depending on the type of user movement that was identified.

31. The method of claim 19, further comprising the step of performing an opening animation visually indicating the emergence of the representations from an entry point associated with the first characteristic presented by the first physical object.

32. The method of claim 19, further comprising the steps of:

(a) enabling a user to select an undo command to undo the reassociation of the data element;

(b) detecting selection of the undo command by the user; and

(c) undoing the reassociating of the data element in response to detecting that the undo command was selected by the user.

33. The method of claim 19, further comprising the steps of:

(a) detecting when at least one of the first characteristic presented by the first physical object and the second characteristic presented by the second physical object is no longer adjacent to the interactive display surface; and

(b) disabling a capability for reassociating the data elements in response thereto.

34. A memory medium having machine instructions for carrying out the steps of claim 19.

35. A system for modifying an association between data and physical objects, comprising:

(a) an interactive display including;

(i) a projector;

(ii) a display surface that diffuses light and on which an image can be displayed by the projector, said display surface having a processing side and an interactive side, the processing side being opposite to the interactive side from which the image is viewed and adjacent to which physical objects can be placed;

(iii) a light source disposed on the processing side of the display surface, the light source emitting infrared light that is transmitted through the display surface to the interactive side; and

(iv) a light sensor disposed on the processing side of the display surface, the light sensor sensing infrared light reflected back from a physical object through the display surface;

(b) a processor in communication with the light sensor; and

(c) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) using the light sensor for reading a characteristic presented by the physical object when the physical object is disposed adjacent to the interactive display surface;

(ii) accessing the data elements associated with the characteristic from a data storage included in the memory;

(iii) showing on the interactive display surface a representation depicting each of the data elements;

(iv) detecting, adjacent to the interactive display surface, a user movement indicating a user command to reassociate a selected representation;

(v) reassociating the data element corresponding to the selected representation; and

(vi) visually confirming reassociation of the data element by showing the selected representation being reassociated on the interactive display surface.

36. The system of claim 35, wherein the characteristic includes a light reflective identification disposed on a surface of the physical object.

37. The system of claim 35, wherein the machine instructions stored in the memory further cause the processor to read the characteristic presented by the physical object by:

(a) transmitting infrared light from the light source through the interactive display surface toward a face of the interactive display surface adjacent to which the physical object is disposed; and

(b) recognizing the characteristic presented by physical object by imaging infrared light reflected thereby with the light sensor.

38. The system of claim 35, wherein the representation of each of the data elements includes one of:

(a) a content of the data element;

(b) a portion of the content of the data element;

(c) a name representing the data element; and

(d) an icon representing the data element.

39. The system of claim 38, wherein the machine instructions stored in the memory further cause the processor to present the representation in response to a location where the physical object is placed on the interactive display surface.

40. The system of claim 38 wherein, in response to the location where the physical object is placed, the machine instructions stored in the memory further cause the processor to present the data elements to be presented on the interactive display surface in one of:

(a) a sort mode wherein the representation of each of the data elements is in a generally tabular fashion, sorted in regard to a defined parameter;

(b) a carousel mode wherein the representation of each of the data elements is presented in a sequentially-related list; and

(c) a show mode wherein the representation of each of the data elements is presented individually and sequentially.

41. The system of claim 35, wherein the machine instructions stored in the memory further cause the processor to reassociate the data element corresponding to the selected representation by resequencing the data elements associated with the characteristic.

42. The system of claim 35, wherein the machine instructions stored in the memory further cause the processor to reassociate the data element corresponding to the selected representation by reassociating the data with a second characteristic presented by a second physical object placed adjacent to the interactive display surface.

43. The system of claim 42, wherein the machine instructions stored in the memory further cause the processor to employ the light sensor to determine that the second characteristic is placed adjacent to a designated portion of the interactive display surface, enabling the selected representation to be reassociated with the second characteristic.

44. The system of claim 42, wherein the machine instructions stored in the memory further cause the processor to reassociate the data element with the second characteristic by one of:

(a) copying the data from a first storage location associated with the characteristic to a second storage location associated with the second characteristic; and

(b) moving the data from the first storage location associated with the characteristic to the second storage location associated with the second characteristic.

45. The system of claim 42, wherein the second characteristic is associated with one of:

(a) a local data storage associated with the processor; and

(b) a remote data storage on a server accessible over a network such that the data elements associated with the second identifier can be accessed via one of;

(i) the interactive display surface; and

(ii) a second interactive display surface in communication with the remote data storage on the server.

46. The system of claim 35, wherein the machine instructions stored in the memory further cause the processor to show the selected representation being reassociated on the interactive display surface by presenting a reassociation animation between the selected representation and a reassociation point, said reassociation animation showing one of:

(a) the reassociation point pulling the selected representation into the reassociation point;

(b) the selected representation vanishing into the reassociation point;

(c) the selected representation shrinking into the reassociation point; and

(d) the selected representation curving into the reassociation point.

47. The system of claim 46, wherein the machine instructions stored in the memory further cause the processor to identify a type of user movement, and determine the reassociation animation presented, depending on the type of user movement that was identified.

48. The system of claim 35, wherein the machine instructions stored in the memory further cause the processor to:

(a) use the light sensor to detect when the second characteristic is no longer adjacent to the interactive display surface; and

, in response thereto,

(b) disable a capability for reassociating the data elements associated with the second characteristic.

49. A system for modifying an association between data and physical objects, comprising:

(a) an interactive display including;

(i) a projector;

(ii) a display surface that diffuses light and on which an image can be displayed by the projector, said display surface having a processing side and an interactive side, the processing side being opposite to the interactive side from which the image is viewed and adjacent to which physical objects can be placed;

(iii) a light source disposed on the processing side of the display surface, the light source emitting infrared light that is transmitted through the display surface to the interactive side; and

(iv) a light sensor disposed on the processing side of the display surface, the light sensor sensing infrared light reflected back from a physical object through the display surface;

(b) a processor in communication with the light'"'"'sensor; and

(c) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) using the light sensor for reading a first characteristic presented by the first physical object when the first physical object is disposed adjacent to the interactive display surface;

(ii) accessing the data elements associated with the first characteristic from a data storage included in the memory;

(iii) showing on the interactive display surface a representation depicting each of the data elements;

(iv) using the light sensor for reading a second characteristic presented by a second physical object when the physical object is disposed adjacent to the interactive display surface;

(v) using the light sensor for detecting, adjacent to the interactive display surface, a user movement indicating a user command to reassociate a selected representation with the second characteristic; and

(vi) reassociating tie data elements with the second characteristic presented by the second physical object.

50. The system of claim 49, wherein at least one of the first characteristic presented by the first physical object and the second characteristic presented by the second physical object includes a light reflective identification.

51. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to identify at least one of the first characteristic and the second characteristic by:

(a) transmitting infrared light from the light source through the interactive display surface toward a face of the interactive display surface adjacent to which the first characteristic and the second characteristic are disposed; and

(b) recognizing at least one of the first characteristic and the second characteristic by imaging infrared light reflected thereby, using the light sensor.

52. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to present the representation of each of the data elements as one of:

(a) a content of the data element;

(b) a portion of the content of the data element;

(c) a name representing the data element; and

(d) an icon representing the data element.

53. The system of claim 52, wherein the machine instructions stored in the memory further cause the processor to present the representation of each of the data elements in response to a location where the first physical object is placed on the interactive display surface.

54. The system of claim 49, wherein, in response to the location where the first physical object is placed, the machine instructions stored in the memory further cause the processor to present the data elements in one of:

(a) a sort mode wherein the representation of each of the data elements is in a generally tabular fashion, sorted in regard to a defined parameter;

(b) a carousel mode wherein the representation of each of the data elements is presented in a sequentially-related list; and

(c) a show mode wherein the representation of each of the data elements is presented individually and sequentially.

55. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to visually confirm reassociation of the data element with the second characteristic by showing the selected representation being reassociated with the second characteristic presented by the second physical object on the interactive display surface, on the interactive display surface.

56. The system of claim 55, wherein the machine instructions stored in the memory further cause the processor to show the selected representation being reassociated with a reassociation point associated with the second characteristic presented by the second physical object on the interactive display surface, by showing one of:

(a) the reassociation point pulling the selected representation into the second characteristic presented by the second physical object;

(b) the selected representation vanishing into the reassociation point;

(c) the selected representation shrinking into the reassociation point; and

(d) the selected representation curving into the reassociation point.

57. The system of claim 56, wherein the machine instructions stored in the memory further cause the processor to identify a type of user movement, and determine the reassociation animation presented depending on the type of user movement that was identified.

58. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to do one of:

(a) copy the data from a first storage location associated with the first characteristic to a second storage location associated with the second characteristic; and

(b) move the data from the first storage location associated with the first characteristic to the second storage location associated with the second characteristic.

59. The system of claim 49, wherein the second characteristic is associated with one of:

(a) a local data storage associated with the processor; and

(b) a remote data storage on a server accessible over a network such that the data elements associated with the second identifier can be accessed via one of;

(i) the interactive display surface; and

(ii) a second interactive display surface that is in communication with the remote data storage on the server.

60. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to:

(a) detect an undo command provided by a user; and

(b) undo the reassociating of the data element in response to the undo command being detected.

61. The system of claim 49, wherein the machine instructions stored in the memory further cause the processor to:

(a) detect when at least one of the first characteristic presented by the first physical object and the second characteristic presented by the second physical object is no longer adjacent to the interactive display surface; and

, in response thereto,

(b) disable a capability for reassociating the data elements.