Using physical objects to adjust attributes of an interactive display application

• US 20060001650A1
• Filed: 06/30/2004
• Published: 01/05/2006
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Drucker, Steven M., Dehlin, Joel P., Wilson, Andrew D., Robbins, Daniel C., Nye, Kristen Nilsen
• Granted Patent: US 7,743,348 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Input is provided to an application using a plurality of physical objects disposed adjacent to an interactive display surface. A primary location is determined where a primary physical object, e.g., a finger or thumb of the user, is positioned adjacent to the interactive display surface. An additional location is determined where an additional physical object is positioned adjacent to the interactive display surface. The attribute might be a size of an image or selected portion of the image that will be retained after cropping. A change in position of at least one of the objects is detected, and the attribute is adjusted based on the change in position of one or both objects. A range of selectable options of the application can also be display by touching the interactive display surface with one'"'"'s fingers or other objects, and one of the options can be selected with another object.

Claims

1. A method for providing input to an application executing on a computer system having an interactive display surface, the method comprising the steps of:

(a) executing the application on the computer system, the application having at least one attribute that can be adjusted during execution of the application;

(b) determining a primary location adjacent to the interactive display surface where a primary physical object has been positioned by a user;

(c) determining an additional location adjacent to the interactive display surface where an additional physical object has been positioned by a user;

(d) associating the attribute with the primary location and the additional location;

(e) determining a change in position of at least one of the primary location and the additional location; and

(f) adjusting the attribute based on the change in position of at least one of the primary location and the additional location.

2. The method of claim 1, wherein the steps of determining the primary location, the additional location, and the change in position of at least one of the primary location and the additional location comprise the steps of:

(a) transmitting infrared (IR) light through the interactive display surface toward a face of the interactive display surface adjacent to which the primary physical object and the secondary physical object are disposed;

(b) detecting the primary location by sensing IR light reflected from the primary physical object when the primary object has been positioned adjacent to the interactive display surface;

(c) detecting the additional location by sensing IR light reflected from the additional physical object when the secondary object has been positioned adjacent to the interactive display surface; and

(d) detecting the change in position of at least one of the primary location and the additional location by determining any change in position of the primary physical object and the secondary physical object, based upon the IR light that was sensed.

3. The method of claim 1, wherein at least one of the primary physical object and the additional physical object includes a digit on one of a user'"'"'s hands.

4. The method of claim 1, wherein the attribute includes a size of a virtual entity displayed on the interactive display surface, and the step of adjusting the attribute comprises the steps of:

(a) determining that one of the primary location and the additional location is disposed at one of a boundary of the virtual entity and within the virtual entity;

(b) determining that the primary location and the additional location are disposed in one of a substantially vertical alignment and a substantially horizontal alignment with each other; and

(c) resizing the virtual entity so that its extents in at least one dimension are defined by the primary location and the additional location, a center of the virtual entity being positioned at a midpoint between the primary location and the additional location.

5. The method of claim 4, wherein the virtual entity comprises an image.

6. The method of claim 4, wherein an aspect ratio of the virtual entity is preserved as the size of the virtual entity is changed.

7. The method of claim 4, wherein an aspect ratio of the virtual entity is changed as the size of the virtual entity is changed.

8. The method of claim 1, wherein the attribute includes a portion of a virtual entity displayed on the interactive display surface, and the step of adjusting the attribute comprises the steps of:

(a) determining that one of the primary location and the additional location is disposed at one of a boundary of the virtual entity and within the virtual entity;

(b) determining that the primary location and the additional location are disposed neither in a substantially vertical alignment nor a substantially horizontal alignment with each other; and

(c) cropping the virtual entity to eliminate all but a portion of the virtual entity with diagonally opposite corners corresponding to the primary location and the additional location.

9. The method of claim 1, wherein the step of adjusting the attribute comprises the steps of:

(a) presenting a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(b) associating the additional location with a property option selected from the range of property options, the property option selected being a property option closest to the additional location; and

(c) adjusting the attribute in accordance with the property option selected.

10. The method of claim 9, further comprising the steps of:

(a) detecting a secondary location adjacent to the interactive display surface where a secondary physical object has been disposed; and

(b) adjusting a size of the property selection range to generally fit a space defined by the primary location and the secondary location.

11. The method of claim 10, wherein the primary physical object and the secondary physical object include digits on a user'"'"'s hand.

12. The method of claim 11, wherein the additional physical object includes a digit on a user'"'"'s hand.

13. The method of claim 1, wherein the step of adjusting the attribute further comprises the steps of:

(a) presenting a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(b) associating the additional location with a property option selected from the range of property options, the property option selected being closest to the additional location;

(c) presenting an additional property selection range adjacent to the property selected, the additional property selection range including a range of additional property options associated with the property option selected, which are within the additional property selection range;

(d) associating an additional property option selected among the range of additional property options with a changed additional location, the additional property option selected being closest to the changed additional location; and

(e) adjusting the attribute in accordance with the additional property option selected.

14. A method for adjusting a presentation of an image on an interactive display surface that is coupled to a computer system, comprising the steps of:

(a) executing an image presentation application on the computer system, the image presentation application presenting at least one image on the interactive display surface;

(b) determining a primary location adjacent to the interactive display surface where a primary physical object has been disposed;

(c) determining an additional location adjacent to the interactive display surface where an additional physical object has been disposed;

(d) associating new boundaries of the image with the primary location and the additional location;

(e) determining any change in a position of at least one of the primary location and the additional location; and

(f) adjusting the new boundaries of the image based on any change in the position of at least one of the primary location and the additional location that was detected.

15. The method of claim 14, wherein the steps of determining the primary location, the additional location, and the change in position of at least one of the primary location and the additional location comprise the steps of:

(a) transmitting infrared (IR) light through the interactive display surface toward a face of the interactive display surface adjacent to which the primary physical object and the secondary physical object are disposed;

(b) detecting the primary location by sensing IR light reflected from the primary physical object when the primary object has been positioned adjacent to the interactive display surface;

(c) detecting the additional location by sensing IR light reflected from the additional physical object when the secondary object has been positioned adjacent to the interactive display surface; and

(d) detecting the change in position of at least one of the primary location and the additional location by determining any change in position of the primary physical object and the secondary physical object, based upon the IR light that was sensed.

16. The method of claim 14, wherein at least one of the primary physical object and the additional physical object includes a digit on one of a user'"'"'s hands.

17. The method of claim 14, wherein the step of associating new boundaries of the image comprises the step of changing a size of the image by:

(a) determining that the primary location and the additional location are generally one of vertically aligned and horizontally aligned with each other;

(b) setting opposite boundaries of the image to correspond with the primary location and the additional location, respectively; and

(c) moving at least one new boundary of the image to correspond with any change in the position of the primary location and an opposite new boundary to correspond with any change in position of the additional location, to change the size of the image.

18. The method of claim 17, wherein a center of the image remains at a midpoint between the primary location and the additional location as any change in at least one of the primary location and the additional location occurs.

19. The method of claim 14, wherein an aspect of the ratio of the image is preserved as the size of the image is changed.

20. The method of claim 14, wherein an aspect ratio of the image is changed as the size of the image is changed.

21. The method of claim 14, further comprising the step of changing a portion of the image displayed on the interactive display surface by:

(a) determining that the primary location and the additional location are neither generally vertically aligned nor generally horizontally aligned with each other;

(b) forming new image boundaries with diagonally opposite corners corresponding to the primary location and the additional location, respectively;

(c) adjusting at least one of the diagonally opposite corners to correspond to any change in the position of at least one of the primary location and the additional location; and

(d) cropping the image to eliminate all but the portion of the image bounded by the new image boundary.

22. A method for providing input to an application executing on a computer system coupled to an interactive display surface, comprising the steps of:

(a) executing an application on the computer system, the application having at least one attribute that can be changed;

(b) determining a primary location adjacent to the interactive display surface where a primary physical object has been disposed;

(c) presenting a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(d) determining an additional location adjacent to the interactive display surface where an additional physical object has been disposed;

(e) associating the additional location with a property option selected from the range of property options, the property option selected being closest to the additional location; and

(f) adjusting the attribute in accordance with the selected property option.

23. The method of claim 22, wherein the steps of determining the primary location, the additional location, and the change in position of at least one of the primary location and the additional location, comprise the steps of:

(a) transmitting infrared (IR) light through the interactive display surface toward a face of the interactive display surface adjacent to which the primary physical object and the secondary physical object are disposed;

(b) detecting the primary location by sensing IR light reflected from the primary physical object when the primary object has been disposed adjacent to the interactive display surface;

(c) detecting the additional location by sensing IR light reflected from the additional physical object when the secondary object has been disposed adjacent to the interactive display surface; and

(d) detecting any change in position of at least one of the primary location and the additional location by determining any change in the primary location and the additional location, by determining any change in position of the primary physical object and the secondary physical object, based upon the IR light that was sensed.

24. The method of claim 22, further comprising the steps of:

(a) determining a secondary location adjacent to the interactive display surface where a secondary physical object has been disposed; and

(b) adjusting a size of the property selection range to generally fit a space defined by the primary location and the secondary location.

25. The method of claim 24, wherein the primary physical object, the additional physical object, and the secondary physical object include digits on a user'"'"'s hand.

26. The method of claim 22, wherein the step of adjusting the attribute comprises the steps of:

(a) presenting a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(b) associating the additional location with a property option selected among the range of property options, the property option being closest to the additional location;

(c) presenting an additional property selection range adjacent to the property option selected, the additional property selection range including a range of additional property options associated with the property option selected, and included within the additional property selection range;

(d) associating an additional property option selected from the range of additional property options with a changed additional location, the additional property option selected being closest to the changed additional location; and

(e) adjusting the attribute in accordance with the additional property option selected.

27. A system for providing input to an application that is being executed, comprising:

(a) an interactive display surface adjacent to which a physical object is manipulated, said interactive display surface diffusing light, and having a processing side and an interactive side from which the image is viewed and adjacent to which the physical object can be disposed, the processing side being opposite to the interactive side;

(b) a projector that projects virtual entities onto the processing side of the interactive display surface, said virtual entities being visible from the interactive side;

(c) a light source disposed on the processing side of the interactive display surface, the light source emitting infrared (IR) light that is transmitted through the interactive display surface to the interactive side and reflected back through the interactive display surface by the physical object that is disposed adjacent to the interactive side of the interactive display surface;

(d) a light sensor disposed on the processing side of the interactive display surface, the light sensor sensing IR light reflected back from the physical object through the interactive display surface and imaging the interactive display surface to detect the physical object and its location;

(e) a processor in communication with the light sensor and the projector; and

(f) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) executing an application on the computer system, the application having a attribute that can be changed during execution of the application;

(ii) determining a primary location adjacent to the interactive display surface where a primary physical object has been disposed;

(iii) determining an additional location adjacent to the interactive display surface where an additional physical object has been disposed;

(iv) associating the attribute with the primary location and the additional location;

(v) determining a change in a position of at least one of the primary location and the additional location; and

(vi) adjusting the attribute based on the change in the position of at least one of the primary location and the additional location.

28. The system of claim 27, wherein at least one of the primary physical object and the additional physical object includes a digit on one of a user'"'"'s hands.

29. The system of claim 27, wherein the attribute includes a size of the virtual entity displayed on the interactive display surface, and the machine language instructions further cause the processor to:

(a) determine that one of the primary location and the additional location is disposed at one of a boundary of the virtual entity and within the virtual entity;

(b) determine that the primary location and the additional location are disposed in one of a substantially vertical alignment and a substantially horizontal alignment with each other; and

(c) resize the virtual entity so that its extents in at least one dimension are defined by the primary location and the additional location, a center of the virtual entity being positioned at a midpoint between the primary location and the additional location.

30. The system of claim 29, wherein the virtual entity comprises an image.

31. The system of claim 29, wherein the machine language instructions further cause the processor to preserve an aspect ratio of the virtual entity as the size of the virtual entity is changed.

32. The system of claim 29, wherein the machine language instructions further cause the processor to change an aspect ratio of the virtual entity as the size of the virtual entity is changed.

33. The system of claim 27, wherein the attribute includes a portion of a virtual entity displayed on the interactive display surface, and the machine language instructions further cause the processor to

(a) determine that one of the primary location and the additional location is disposed at one of a boundary of the virtual entity and within the virtual entity;

(b) determine that the primary location and the additional location are disposed neither in a substantially vertical alignment nor a substantially horizontal alignment with each other; and

(c) crop the virtual entity to eliminate all but a portion of the virtual entity with diagonally opposite corners corresponding to the primary location and the additional location.

34. The system of claim 27, wherein the machine language instructions further cause the processor to:

(a) present a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(b) associate the additional location with a property option selected from the range of property options, the property option selected being closest to the additional location; and

(c) adjust the attribute in accordance with the property option selected.

35. The system of claim 34, wherein the machine language instructions further cause the processor to:

(a) detect a secondary location adjacent to the interactive display surface where a secondary physical object has been disposed; and

(b) adjust a size of the property selection range to generally fit a space defined by the primary location and the secondary location.

36. The system of claim 35, wherein the primary physical object and the secondary physical object include digits on a user'"'"'s hand.

37. The system of claim 35, wherein the additional physical object includes a digit on a user'"'"'s hand.

38. The system of claim 27, wherein the machine language instructions further cause the processor to:

(a) present a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(b) associate the additional location with a property option selected from the range of property options, the property option selected being closest to the additional location;

(c) present an additional property selection range adjacent to the property option selected, the additional property selection range including a range of additional property options associated with the property option selected, and included in the additional property selection range;

(d) associate an additional property option selected from the range of additional property option with a changed additional location, the additional property option selected being closest to the changed additional location; and

(e) adjust the attribute in accordance with the additional property option selected.

39. A system for adjusting a presentation of an image, comprising:

(a) an interactive display surface adjacent to which a physical object is manipulated, said interactive display surface diffusing light, and having a processing side and an interactive side from which the image is viewed and adjacent to which the physical object can be placed, the processing side being opposite to the interactive side;

(b) a projector that projects graphic images onto the processing side of the interactive display surface, said graphic images being visible from the interactive side;

(c) a light source disposed on the processing side of the display surface, the light source emitting infrared (IR) light that is transmitted through the display surface to the interactive side and reflected back through the interactive display surface by the physical object that is disposed adjacent to the interactive side of the interactive display surface;

(d) a light sensor disposed on the processing side of the interactive display surface, the light sensor sensing IR light reflected back from the physical object through the interactive display surface and imaging the interactive display surface to detect the physical object and its location;

(e) a processor in communication with the light sensor and the projector; and

(f) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) executing an image presentation application on the computer system, the image presentation application presenting at least one image on the interactive display surface using the projector;

(ii) using the light sensor for determining a primary location adjacent to the interactive display surface where a primary physical object has been disposed;

(iii) using the light sensor for determining an additional location adjacent to the interactive display surface where an additional physical object has been disposed;

(iv) associating new boundaries of the image with the primary location and the additional location;

(v) using the light sensor for determining any change in a position of at least one of the primary location and the additional location; and

(vi) adjusting the new boundaries of the image based on any change in the position of at least one of the primary location and the additional location that was detected.

40. The system of claim 39, wherein at least one of the primary physical object and the additional physical object includes a digit on one of a user'"'"'s hands.

41. The system of claim 39, wherein the attribute includes a size of an image displayed on the interactive display surface, and the machine language instructions further cause the processor to:

(a) determine that the primary location and the additional location are generally one of vertically aligned and horizontally aligned with each other;

(b) set opposite boundaries of the image to correspond with the primary location and the additional location, respectively; and

(c) move at least one new boundary of the image to correspond with any change in the position of the primary location and an opposite new boundary to correspond with any change in position of the additional location, to change the size of the image.

42. The system of claim 41, wherein a center of the image remains at a midpoint between the primary location and the additional location as any change in at least one of the primary location and the additional location occurs.

43. The system of claim 41, wherein the machine language instructions further cause the processor to preserve an aspect ratio of the image as the size of the image is changed.

44. The system of claim 41, wherein the machine language instructions further cause the processor to change an aspect ratio of the image as the size of the image is changed.

45. The system of claim 39, wherein the attribute includes a portion of an image displayed on the interactive display surface, and the machine language instructions further cause the processor to:

(a) determine that the primary location and the additional location are neither generally vertically aligned nor generally horizontally aligned with each other;

(b) form new image boundaries with diagonally opposite corners corresponding to the primary location and the additional location, respectively;

(c)-adjust at least one of the diagonally opposite corners to correspond to any change in the position of at least one of the primary location and the additional location; and

(d) crop the image to eliminate all but the portion of the image bounded by the new image boundary.

46. A system for providing an input to an application, comprising:

(a) an interactive display surface adjacent to which a physical object is manipulated, said interactive display surface diffusing light, and having a processing side and an interactive side from which the image is viewed and adjacent to which the physical object can be placed, the processing side being opposite to the interactive side;

(b) a projector that projects graphic images onto the processing side of the interactive display surface, said graphic images being visible from the interactive side;

(c) a light source disposed on the processing side of the display surface, the light source emitting IR light that is transmitted through the display surface to the interactive side and reflected back through the interactive display surface by the physical object that is disposed adjacent to the interactive side of the interactive display surface;

(d) a light sensor disposed on the processing side of the interactive display surface, the light sensor sensing infrared (IR) light reflected back from the physical object through the interactive display surface and imaging the interactive display surface to detect the physical object and its location;

(e) a processor in communication with the light sensor and the projector; and

(f) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) presenting a property selection range adjacent to the primary location, the property selection range including a range of property options within the property selection range;

(ii) associating the additional location with a selected property option from the range of property options, the property option selected being closest to the additional location; and

(iii) adjusting the attribute in accordance with the property option selected, unless the property option selected causes an additional property selection range to be presented.

47. The system of claim 46, wherein the machine language instructions further cause the processor to:

(a) detect a secondary location adjacent to the interactive display surface where a secondary physical object is placed; and

(b) adjust a size of the property selection range to generally fit a space defined by the primary location and the secondary location.

48. The system of claim 46, wherein the primary physical object and the secondary physical object include digits on a user'"'"'s hand.

49. The system of claim 46, wherein the additional physical object includes a digit on a user'"'"'s hand.

50. The system of claim 46, wherein the machine language instructions further cause the processor to:

(a) present the additional property selection range adjacent to the property option selected, the additional property selection range including a range of additional property options associated with the property option selected within the additional property selection range;

(b) associate an additional property option selected from the range of additional property options with a changed additional location, the additional property option selected being closest to the changed additional location; and

(c) adjust the attribute in accordance with the additional property option selected.