Controlling Virtual Objects

• US 20140108979A1
• Filed: 10/17/2012
• Published: 04/17/2014
• Est. Priority Date: 10/17/2012
• Status: Abandoned Application

• Current Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Application Number:
  US13/654,363
• Publication Number:
  US 20140108979A1
• Time in Patent Office: Days
• US Class Current: 715/765
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Controlling virtual objects displayed on a display device comprises controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement. User interaction relative to the display device on which the multiple virtual objects are displayed is sensed. Positions of the multiple virtual objects on the display device at a time corresponding to the sensed user interaction is determined. A subset of the multiple virtual objects as candidates for restriction based on the sensed user interaction and the determined positions of the multiple virtual objects on the display device at the time corresponding to the sensed user interaction is determined. An operation related to restricting movement of the determined subset of virtual objects based on the first type of input is performed.

Claims

1. A method for controlling virtual objects displayed on a display device, the method comprising:

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement;

sensing user interaction relative to the display device on which the multiple virtual objects are displayed;

determining positions of the multiple virtual objects on the display device at a time corresponding to the sensed user interaction;

determining a subset of the multiple virtual objects as candidates for restriction based on the sensed user interaction and the determined positions of the multiple virtual objects on the display device at the time corresponding to the sensed user interaction; and

performing an operation related to restricting movement of the determined subset of virtual objects based on the first type of input.

2. The method of claim 1, wherein performing the operation related to restricting movement of the determined subset of virtual objects based on the first type of input comprises restricting, for the determined subset of virtual objects, movement that occurs based on the first type of input while enabling alteration of the determined subset of virtual objects based on the second type of input, the movement that occurs based on the first type of input for the determined subset of virtual objects being less than movement that occurs based on the first type of input for the multiple virtual objects that are excluded from the determined subset of virtual objects.

3. The method of claim 1:

wherein sensing user interaction relative to the display device on which the multiple virtual objects are displayed includes sensing a stylus touching the display device and determining a position of the stylus touching the display device; and

wherein determining a subset of the multiple virtual objects as candidates for restriction includes;

comparing the position of the stylus with the determined positions of the multiple virtual objects on the display device;

based on the comparison, determining that the position of the stylus contacts a first virtual object included in the multiple virtual objects on the display device; and

including the first virtual object in the subset of virtual objects based on the determination that the position of the stylus contacts the first virtual object.

4. The method of claim 1:

wherein sensing user interaction relative to the display device on which the multiple virtual objects are displayed includes sensing a stylus touching the display device and determining a position of the stylus touching the display device; and

wherein determining a subset of the multiple virtual objects as candidates for restriction includes;

comparing the position of the stylus with the determined positions of the multiple virtual objects on the display device;

based on the comparison, determining that the position of the stylus is within a threshold distance of a first virtual object included in the multiple virtual objects on the display device; and

including the first virtual object in the subset of virtual objects based on the determination that the position of the stylus is within the threshold distance of the first virtual object.

5. The method of claim 1:

wherein sensing user interaction relative to the display device on which the multiple virtual objects are displayed includes sensing a stylus hovering over the display device and determining a position of the stylus hovering over the display device; and

wherein determining a subset of the multiple virtual objects as candidates for restriction includes;

comparing the position of the stylus with the determined positions of the multiple virtual objects on the display device;

based on the comparison, determining that the position of the stylus is directly over a first virtual object included in the multiple virtual objects on the display device; and

including the first virtual object in the subset of virtual objects based on the determination that the position of the stylus is directly over the first virtual object.

6. The method of claim 1:

wherein sensing user interaction relative to the display device on which the multiple virtual objects are displayed includes sensing a stylus hovering over the display device and determining a position of the stylus hovering over the display device; and

wherein determining a subset of the multiple virtual objects as candidates for restriction includes;

comparing the position of the stylus with the determined positions of the multiple virtual objects on the display device;

based on the comparison, determining that the position of the stylus is within a threshold distance of a first virtual object included in the multiple virtual objects on the display device; and

including the first virtual object in the subset of virtual objects based on the determination that the position of the stylus is within the threshold distance of the first virtual object.

7. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes:

determining a location of a designated alteration area defined over a portion of less than all of the display device;

comparing the location of the designated alteration area with the determined positions of the multiple virtual objects on the display device;

based on the comparison, determining that a first virtual object included in the multiple virtual objects is positioned within the designated alteration area; and

including the first virtual object in the subset of virtual objects based on the determination that the first virtual object is positioned within the designated alteration area.

8. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes:

determining that a first virtual object included in the multiple virtual objects has a hierarchical relationship with a second virtual object included in the determined subset of the multiple virtual objects identified as candidates for restriction; and

including the first virtual object in the subset of virtual objects based on the determination that the first virtual object has the hierarchical relationship with the second virtual object.

9. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes:

determining that movement of a first virtual object included in the multiple virtual objects results in movement of a second virtual object included in the determined subset of the multiple virtual objects identified as candidates for restriction; and

including the first virtual object in the subset of virtual objects based on the determination that movement of the first virtual object results in movement of the second virtual object.

10. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes determining positions of the multiple virtual objects relative to the sensed user interaction.

11. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes:

determining that a user is in an alteration position based on the sensed user interaction; and

determining a subset of the multiple virtual objects as candidates for restriction based on the determination that the user is in the alteration position.

12. The method of claim 1, wherein determining a subset of the multiple virtual objects as candidates for restriction includes:

determining an object type of at least one of the multiple virtual objects; and

determining whether to include the at least one of the multiple virtual objects in the subset of the multiple virtual objects based on the object type of the at least one of the multiple virtual objects.

13. The method of claim 1, wherein determining the subset of the multiple virtual objects as candidates for restriction comprises:

computing, for the multiple virtual objects, scores based on the sensed user interaction and the determined positions;

accessing one or more thresholds;

comparing the computed scores to the one or more thresholds; and

determining the subset of the multiple virtual objects as candidates for restriction based on the comparison of the computed scores to the one or more thresholds.

14. The method of claim 13, wherein computing, for the multiple virtual objects, scores based on the sensed user interaction and the determined positions comprises:

computing a score for a particular virtual object included in the multiple virtual objects by;

determining a location of the particular virtual object on the display device relative to a designated alteration area that spans a portion of less than all of the display device;

determining a location of the particular virtual object on the display device relative to a stylus;

determining a location of the particular virtual object on the display device relative to touch input sensed by a detection unit associated with the display device;

determining whether the touch input sensed by the detection unit associated with the display device indicates that a user is in a position for performing alteration based on the second type of input;

determining a type of the particular virtual object;

determining a relationship of the particular virtual object to any other virtual objects identified as candidates for restriction; and

computing, for the particular virtual object, the score based on the location of the particular virtual object on the display device relative to the designated alteration area, the location of the particular virtual object on the display device relative to the stylus, the location of the particular virtual object on the display device relative to the touch input, the determination of whether the touch input indicates that the user is in the position for performing alteration, the type of the particular virtual object, and the relationship of the particular virtual object to any other virtual objects identified as candidates for restriction.

15. The method of claim 13, wherein accessing one or more thresholds comprises accessing multiple thresholds that define at least three levels of restriction,

wherein comparing the computed scores to the one or more thresholds comprises comparing the computed scores to the multiple thresholds that define at least three levels of restriction, and

wherein determining the subset of the multiple virtual objects as candidates for restriction based on the comparison of the computed scores to the one or more thresholds comprises setting, for each of the multiple virtual objects and from among the at least three levels of restriction, a restriction level based on the comparison of the computed scores to the multiple thresholds.

16. The method of claim 15, wherein performing the operation related to restricting movement of the determined subset of virtual objects based on the first type of input includes performing different restriction actions for different restriction levels.

17. The method of claim 1, wherein the first type of input is touch input from a user'"'"'s hand and the second type of input is input from stylus.

18. The method of claim 1, wherein the multiple virtual objects include virtual pages that are altered by being written upon based on the second type of user input.

19. A system, comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input;

sensing user interaction relative to the display device on which the multiple virtual objects are displayed;

determining positions of the multiple virtual objects on the display device at a time corresponding to the sensed user interaction;

determining a subset of the multiple virtual objects as candidates for restriction based on the sensed user interaction and the determined positions of the multiple virtual objects on the display device at the time corresponding to the sensed user interaction; and

performing an operation related to restricting movement of the determined subset of virtual objects based on the first type of input.

20. A non-transitory computer-readable storage medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input;

sensing user interaction relative to the display device on which the multiple virtual objects are displayed;

determining positions of the multiple virtual objects on the display device at a time corresponding to the sensed user interaction;

determining a subset of the multiple virtual objects as candidates for restriction based on the sensed user interaction and the determined positions of the multiple virtual objects on the display device at the time corresponding to the sensed user interaction; and

performing an operation related to restricting movement of the determined subset of virtual objects based on the first type of input.