Using Physical Objects in Conjunction with an Interactive Surface

• US 20100079369A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, Hartmann, Bjorn U., Wilson, Andrew D., Morris, Meredith J.
• Granted Patent: US 8,427,424 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

An interaction management module (IMM) is described for allowing users to engage an interactive surface in a collaborative environment using various input devices, such as keyboard-type devices and mouse-type devices. The IMM displays digital objects on the interactive surface that are associated with the devices in various ways. The digital objects can include input display interfaces, cursors, soft-key input mechanisms, and so on. Further, the IMM provides a mechanism for establishing a frame of reference for governing the placement of each cursor on the interactive surface. Further, the IMM provides a mechanism for allowing users to make a digital copy of a physical article placed on the interactive surface. The IMM also provides a mechanism which duplicates actions taken on the digital copy with respect to the physical article, and vice versa.

Claims

1. A method for engaging an interactive surface, comprising:

detecting placement of an input device at a selected location and orientation on the interactive surface, the input device configured to provide alphanumeric text entry; and

displaying a digital object on the interactive surface in positional association with the input device,

the digital object providing a display interface that presents a visual representation of information entered by a user using the input device.

2. The method of claim 1, further comprising displaying a visual attribute which associates the input device with the digital object.

3. The method of claim 1, wherein the digital object comprises a first digital object, and wherein the interactive surface also displays a second digital object, the method further comprising:

detecting placement of any input device in proximity to the second digital object; and

automatically associating said any input device with the second digital object.

4. The method of claim 1, wherein the digital object comprises a first digital object, and wherein the interactive surface also displays a second digital object, the method further comprising:

detecting placement of the second digital object in proximity to any input device; and

automatically associating said any input device with the second digital object.

5. The method of claim 1, wherein the input device comprises a first input device, the method further comprising:

detecting placement of a second input device on the interactive surface in proximity to the first input device; and

in response to the detecting, automatically associating the second input device with the digital object, such that both the first input device and the second input device are associated with the digital object.

6. The method of claim 1, wherein the input device comprises a first input device and the digital object comprises a first digital object, the method further comprising:

detecting placement of a second input device on the interactive surface, the interactive surface displaying a second digital object in positional association with the second input device;

detecting movement of the second input device in proximity to the first input device or the movement of the first input device in proximity to the second input device; and

in response to detecting the movement, merging content provided by the first digital object with content provided by the second digital object.

7. The method of claim 1, wherein the displaying of the digital object comprises:

placing the digital object on the interactive surface at a location which;

a) satisfies at least one placement constraint to an extent deemed appropriate; and

b) reduces interference between the digital object and other existing objects on the interactive surface.

8. The method of claim 7, wherein the location at which the digital object is placed is determined by forming an integral image.

9. A computer-readable medium for storing computer-readable instructions, the computer-readable instructions providing an interaction management module when executed by one or more processing devices, the computer-readable instructions comprising:

logic configured to detect a user'"'"'s placement of a mouse-type device on an interactive surface;

logic configured to determine an absolute position of the mouse-type device and an orientation of the mouse-type device;

logic configured to define a frame of reference based on the absolute position and the orientation of the mouse-type device; and

logic configured to display a cursor on the interactive surface relative to the frame of reference that has been defined.

10. The computer-readable medium of claim 9, wherein said logic configured to define the frame of reference is operative to identify a reference object on the interactive surface, and to define the frame of reference based on that reference object.

11. The computer-readable medium of claim 9, wherein said logic configured to define the frame of reference is operative to define the frame of reference relative to the orientation of the mouse-type device.

12. The computer-readable medium of claim 9, further comprising logic configured to display a visual attribute which associates the mouse-type device and the cursor.

13. The computer-readable medium of claim 9, further comprising:

logic configured to detect that the user has activated a control provided by the mouse-type device; and

logic configured to operate the mouse-type device in a touch mode, wherein, in the touch mode, the cursor serves as a simulated contact point that engages the interactive surface.

14. An interactive management module for managing an interactive surface, comprising:

logic configured to detect placement of an article on the interactive surface;

logic configured to display an interface object on the interactive surface for use in association with the article;

logic configured to detect a copy-related activation by a user of the interface object; and

logic configured to generate, in response to the activation, a digital copy of the article and display the digital copy of the article on the interactive surface at a location identified by the user.

15. The interactive management module of claim 14, wherein the copy-related activation by the user corresponds to a dragging motion executed by the user by pulling on the interface object.

16. The interactive management module of claim 14, further comprising:

logic configured to detect a selection by the user of a particular digital part of the digital copy; and

logic configured to highlight or mark a physical part of the article that is a counterpart to the particular digital part of the digital copy.

17. The interactive management module of claim 14, further comprising:

logic configured to detect that the user has moved the digital copy to an editing region of the interactive surface;

logic configured to detect a physical mark applied by the user to the digital copy within the editing region; and

logic configured to generate another digital copy of the article, the other digital copy including a digital representation of the physical mark.

18. The interactive management module of claim 14, further comprising logic configured to record a plurality of representations of at least part of the interactive surface at different respective points in time, and logic configured to present information pertaining to the plurality of representations.

19. The interactive management module of claim 18, wherein said logic configured to present information is configured to present a spatial organization of the plurality of representations.

20. The interactive management module of claim 18, wherein said logic configured to present information is configured to present a temporal sequence of the plurality of representations.