Controlling Virtual Objects

• US 20140104320A1
• Filed: 10/17/2012
• Published: 04/17/2014
• Est. Priority Date: 10/17/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Granted Patent: US 9,589,538 B2
• Time in Patent Office: Days
• US Class Current: 345/681
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09G 5/32 with means for controlling the display position see provisionally G09G5/42

Abstract

Controlling virtual objects displayed on a display device includes controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement. A subset of the multiple virtual objects as candidates for restriction is identified, and based on identifying the subset of virtual objects as candidates for restriction, a responsiveness to the first type of input for the subset of virtual objects is restricted. The first type of input applied to a first virtual object included in the subset of virtual objects and a second virtual object included in the multiple virtual objects is detected, with the second virtual object being excluded from the subset of virtual objects. Based on detecting the first type of input applied to the first virtual object and the second virtual object, movement of the first virtual object is controlled in accordance with the restricted responsiveness to the first type of input, and movement of the second virtual object is controlled without restriction.

Claims

1. A method for controlling virtual objects displayed on a display device, the method comprising:

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement;

identifying a subset of the multiple virtual objects as candidates for restriction;

based on identifying the subset of virtual objects as candidates for restriction, restricting, for the subset of virtual objects, responsiveness to the first type of input;

detecting the first type of input applied to a first virtual object included in the subset of virtual objects and a second virtual object included in the multiple virtual objects, but excluded from the subset of virtual objects;

based on detecting the first type of input applied to the first virtual object and the second virtual object;

controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input, and

controlling movement of the second virtual object without restriction.

2. The method of claim 1, wherein performing the operation related to restricting, for the subset of virtual objects, responsiveness to the first type of input comprises restricting, for the subset of virtual objects, movement that occurs based on the first type of input while enabling alteration of the subset of virtual objects based on the second type of input, the movement that occurs based on the first type of input for the subset of virtual objects being less than movement that occurs based on the first type of input for the multiple virtual objects that are excluded from the subset of virtual objects.

3. The method of claim 1, wherein restricting, for the subset of virtual objects, responsiveness to the first type of input comprises locking the subset of virtual objects such that the subset of virtual objects are unresponsive to the first type of input, and

wherein controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input comprises:

ignoring the first type of input applied to the first virtual object, and

maintaining a position of the first virtual object despite the first type of input applied to the first virtual object.

4. The method of claim 1, wherein restricting, for the subset of virtual objects, responsiveness to the first type of input includes increasing inertia of the subset of virtual objects such that the subset of virtual objects are responsive to the first type of input, but less responsive than prior to the increase in inertia.

5. The method of claim 1, wherein restricting, for the subset of virtual objects, responsiveness to the first type of input includes:

detecting a stylus hovering over the first virtual object;

based on detecting the stylus hovering over the first virtual object, increasing inertia of the first virtual object such that the first virtual object is responsive to the first type of input, but less responsive than prior to the increase in inertia;

detecting the stylus contacting the first virtual object; and

based on detecting the stylus contacting the first virtual object, locking the first virtual object such that the first virtual object is unresponsive to the first type of input.

6. The method of claim 1:

wherein restricting, for the subset of virtual objects, responsiveness to the first type of input comprises configuring the subset of virtual objects to remain unresponsive to the first type of input until a threshold level of the first type of input has been met; and

wherein controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input comprises damping motion of the first virtual object until the first type of input applied to the first virtual object meets the threshold level.

7. The method of claim 1, further comprising:

detecting an alteration event of the first virtual object;

determining a time window to apply based upon the detected alteration event of the first virtual object and a history of alteration activity of the first virtual object;

detecting a potential end of the alteration event;

in response to detecting the potential end of the alteration event, continuing to restrict movement of the first virtual object continuously during the determined time window;

monitoring, during the determined time window, for input indicating return to the alteration event; and

releasing, based on the determined time window expiring without detecting the input indicating return to the alteration event during the monitoring, the restriction on movement of the first virtual object.

8. The method of claim 1:

wherein restricting, for the subset of virtual objects, responsiveness to the first type of input comprises;

determining, for the first virtual object, a first constraint to place on a first type of motion that occurs based on the first type of input;

determining, for the first virtual object, a second constraint to place on a second type of motion that occurs based on the first type of input, the second constraint being different from the first constraint and the second type of motion being different from the first type of motion;

restricting, for the first virtual object, responsiveness to the first type of motion that occurs based on the first type of input in accordance with the first constraint; and

restricting, for the first virtual object, responsiveness to the second type of motion that occurs based on the first type of input in accordance with the second constraint; and

wherein controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input comprises;

controlling the first type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint; and

controlling the second type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint.

9. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a first constraint to place on translation that occurs based on the first type of input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a second constraint to place on rotation that occurs based on the first type of input;

wherein controlling the first type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint comprises controlling translation of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint comprises controlling rotation of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint.

10. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a first constraint to place on scaling that occurs based on the first type of input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a second constraint to place on rotation that occurs based on the first type of input;

wherein controlling the first type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint comprises controlling scaling of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint comprises controlling rotation of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint.

11. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a first constraint to place on translation that occurs based on the first type of input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the first type of input comprises determining, for the first virtual object, a second constraint to place on scaling that occurs based on the first type of input;

wherein controlling the first type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint comprises controlling translation of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint comprises controlling scaling of the first virtual object that occurs based on the first type of input applied to the first virtual object in accordance with the second constraint.

12. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate the subset of virtual objects identified as candidates for restriction.

13. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate a manner in which at least one of the subset of virtual objects identified as candidates for restriction has been restricted.

14. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate that the first type of input has been applied to the first virtual object, but that the first type of input has been applied to the first virtual object has been restricted.

15. The method of claim 1, further comprising:

storing, in a memory device of the display device, information regarding a state of the display device, the state including location, position, and restrictions placed on virtual objects displayed on the display device;

detecting that a prior restriction decision applied to a particular virtual object was incorrect;

updating, based upon detecting that the prior restriction decision applied to the particular virtual object was incorrect, the prior restriction decision applied to the particular virtual object to a correct restriction decision; and

reverting, based on the information regarding the state of the display device and the correct restriction decision, a state of the display device to reflect a state that would have resulted had the correct restriction decision been made.

16. The method of claim 15:

wherein detecting that the prior restriction decision applied to the particular virtual object was incorrect comprises detecting that a decision not to restrict the particular virtual object was incorrect;

wherein updating the prior restriction decision applied to the particular virtual object to the correct restriction decision comprises updating the decision not to restrict the particular virtual object to a decision to restrict the particular virtual object; and

wherein reverting a state of the display device to reflect a state that would have resulted had the correct restriction decision been made comprises reverting a state of the display device to reflect a state that would have resulted had the particular virtual object been restricted.

17. The method of claim 15:

wherein detecting that the prior restriction decision applied to the particular virtual object was incorrect comprises detecting that a decision to restrict the particular virtual object was incorrect;

wherein updating the prior restriction decision applied to the particular virtual object to the correct restriction decision comprises updating the decision to restrict the particular virtual object to a decision not to restrict the particular virtual object; and

wherein reverting a state of the display device to reflect a state that would have resulted had the correct restriction decision been made comprises reverting a state of the display device to reflect a state that would have resulted had the particular virtual object not been restricted.

18. The method of claim 1, wherein the first type of input includes touch input from a user'"'"'s hand and the second type of input includes input from a stylus.

19. A system, comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement;

identifying a subset of the multiple virtual objects as candidates for restriction;

based on identifying the subset of virtual objects as candidates for restriction, restricting, for the subset of virtual objects, responsiveness to the first type of input;

detecting the first type of input applied to a first virtual object included in the subset of virtual objects and a second virtual object included in the multiple virtual objects, but excluded from the subset of virtual objects;

based on detecting the first type of input applied to the first virtual object and the second virtual object;

controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input, and

controlling movement of the second virtual object without restriction.

20. A non-transitory computer-readable storage medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement;

identifying a subset of the multiple virtual objects as candidates for restriction;

based on identifying the subset of virtual objects as candidates for restriction, restricting, for the subset of virtual objects, responsiveness to the first type of input;

detecting the first type of input applied to a first virtual object included in the subset of virtual objects and a second virtual object included in the multiple virtual objects, but excluded from the subset of virtual objects;

based on detecting the first type of input applied to the first virtual object and the second virtual object;

controlling movement of the first virtual object in accordance with the restricted responsiveness to the first type of input, and

controlling movement of the second virtual object without restriction.