Controlling virtual objects

• US 9,589,538 B2
• Filed: 10/17/2012
• Issued: 03/07/2017
• Est. Priority Date: 10/17/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Primary Examiner(s): CHUANG, JUNG-MU T
• Application Number:
  US13/654,365
• Publication Number:
  US 20140104320A1
• Time in Patent Office: 1,602 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09G 5/32 with means for controlling the display position see provisionally G09G5/42

Abstract

Controlling virtual objects displayed on a display device includes controlling display, on a display device, of multiple virtual objects, each of the multiple virtual objects being capable of movement based on a first type of input and being capable of alteration based on a second type of input that is different than the first type of input, the alteration being different from movement. A subset of the multiple virtual objects as candidates for restriction is identified, and based on identifying the subset of virtual objects as candidates for restriction, a responsiveness to the first type of input for the subset of virtual objects is restricted. The first type of input applied to a first virtual object included in the subset of virtual objects and a second virtual object included in the multiple virtual objects is detected, with the second virtual object being excluded from the subset of virtual objects. Based on detecting the first type of input applied to the first virtual object and the second virtual object, movement of the first virtual object is controlled in accordance with the restricted responsiveness to the first type of input, and movement of the second virtual object is controlled without restriction.

Claims

1. A method comprising:

displaying multiple virtual objects on a display of a display device;

storing, in non-volatile memory of the display device, information regarding a state of the display device, the state including a location, a position, and restrictions placed on each virtual object of the multiple virtual objects displayed on the display device;

detecting a stylus input on a subset of the multiple virtual objects;

based on at least detecting the stylus input on the subset of the multiple virtual objects, restricting, for the subset of the multiple virtual objects, responsiveness to a gesture input;

detecting the gesture input applied to a first virtual object included in the subset of the multiple virtual objects and a second virtual object included in the multiple virtual objects, but excluded from the subset of the multiple virtual objects;

based on detecting the gesture input applied to the first virtual object and the second virtual object;

storing, in non-volatile memory of the display device, the gesture input applied to the first virtual object and the second virtual object, and the restriction to responsiveness to the gesture input for the first virtual object;

controlling a movement of the first virtual object in accordance with the restricted responsiveness to the gesture input, and

controlling a movement of the second virtual object without restriction;

detecting that a prior restriction decision applied to the first virtual object was incorrect;

updating, based on at least detecting that the prior restriction decision applied to the first virtual object was incorrect, the prior restriction decision applied to the first virtual object to a correct restriction decision; and

reverting, based on at least the information regarding the state of the display device, the stored gesture input applied to the first virtual object, and the correct restriction decision, a state of the display device to reflect a state that would have resulted had the correct restriction decision been made.

2. The method of claim 1, wherein performing the operation related to restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input comprises restricting, for the subset of the multiple virtual objects, a movement that occurs based on the gesture input, the movement that occurs based on the gesture input for the subset of the multiple virtual objects being less than a movement that occurs based on the gesture input for the multiple virtual objects that are excluded from the subset of the multiple virtual objects.

3. The method of claim 1, wherein restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input comprises locking the subset of the multiple virtual objects such that the subset of the multiple virtual objects are unresponsive to the gesture input, and wherein controlling a movement of the first virtual object in accordance with the restricted responsiveness to the gesture input comprises:

ignoring the gesture input applied to the first virtual object, and

maintaining a position of the first virtual object despite the gesture input applied to the first virtual object.

4. The method of claim 1, wherein restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input includes increasing an inertia of the subset of the multiple virtual objects such that the subset of the multiple virtual objects are responsive to the gesture input, but less responsive than prior to the increase in the inertia.

5. The method of claim 1, wherein restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input includes:

detecting a stylus hovering over the first virtual object;

based on detecting the stylus hovering over the first virtual object, increasing an inertia of the first virtual object such that the first virtual object is responsive to the gesture input, but less responsive than prior to the increase in the inertia;

detecting the stylus contacting the first virtual object; and

based on detecting the stylus contacting the first virtual object, locking the first virtual object such that the first virtual object is unresponsive to the gesture input.

6. The method of claim 1:

wherein restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input comprises configuring the subset of the multiple virtual objects to remain unresponsive to the gesture input until a threshold level of the gesture input has been met;

and wherein controlling a movement of the first virtual object in accordance with the restricted responsiveness to the gesture input comprises damping a motion of the first virtual object until the gesture input applied to the first virtual object meets the threshold level.

7. The method of claim 1, further comprising:

detecting an alteration event of the first virtual object;

determining a time window to apply based upon the detected alteration event of the first virtual object and a history of alteration activity of the first virtual object;

detecting a potential end of the alteration event;

based on at least detecting the potential end of the alteration event, continuing to restrict the movement of the first virtual object continuously during the determined time window;

monitoring, during the determined time window, for an input indicating a return to the alteration event; and

releasing, based on the determined time window expiring without detecting the input indicating the return to the alteration event during the monitoring, the restriction on the movement of the first virtual object.

8. The method of claim 1:

wherein restricting, for the subset of the multiple virtual objects, responsiveness to the gesture input comprises;

determining, for the first virtual object, a first constraint to place on a first type of motion that occurs based on the gesture input;

determining, for the first virtual object, a second constraint to place on a second type of motion that occurs based on the gesture input, the second constraint being different from the first constraint and the second type of motion being different from the first type of motion;

restricting, for the first virtual object, responsiveness to the first type of motion that occurs based on the gesture input in accordance with the first constraint; and

restricting, for the first virtual object, responsiveness to the second type of motion that occurs based on the gesture input in accordance with the second constraint; and

wherein controlling a movement of the first virtual object in accordance with the restricted responsiveness to the gesture input comprises;

controlling the first type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint; and

controlling the second type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint.

9. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the first constraint to place on a translation that occurs based on the gesture input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the second constraint to place on a rotation that occurs based on the gesture input;

wherein controlling the first type of motion that occurs based on the first type of input applied to the first virtual object in accordance with the first constraint comprises controlling the translation of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint comprises controlling the rotation of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint.

10. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the first constraint to place on a scaling that occurs based on the gesture input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the second constraint to place on a rotation that occurs based on the gesture input;

wherein controlling the first type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint comprises controlling the scaling of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint comprises controlling the rotation of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint.

11. The method of claim 8:

wherein determining, for the first virtual object, the first constraint to place on the first type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the first constraint to place on a translation that occurs based on the gesture input;

wherein determining, for the first virtual object, the second constraint to place on the second type of motion that occurs based on the gesture input comprises determining, for the first virtual object, the second constraint to place on a scaling that occurs based on the gesture input;

wherein controlling the first type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint comprises controlling the translation of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the first constraint; and

wherein controlling the second type of motion that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint comprises controlling the scaling of the first virtual object that occurs based on the gesture input applied to the first virtual object in accordance with the second constraint.

12. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate the subset of the multiple virtual objects that have restricted responsiveness to the gesture input.

13. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate a manner in which at least one of the subset of the multiple virtual objects has been restricted.

14. The method of claim 1, further comprising updating the display of the multiple virtual objects to indicate that the gesture input has been applied to the first virtual object, but that the gesture input that has been applied to the first virtual object has been restricted.

15. The method of claim 1, further comprising:

detecting that not applying the prior restriction decision to the second virtual object was incorrect;

updating, based on at least detecting that not applying the prior restriction decision to the second virtual object was incorrect, the prior restriction decision to a second correct restriction decision applied to the second virtual object; and

reverting, based on at least the information regarding the state of the display device, the stored gesture input applied to the second virtual object, and the second correct restriction decision, a state of the display device to reflect a state that would have resulted had the second correct restriction decision been made.

16. The method of claim 1:

wherein detecting that the prior restriction decision applied to the first virtual object was incorrect comprises detecting that a decision not to restrict the first virtual object was incorrect;

wherein updating the prior restriction decision applied to the first virtual object to the correct restriction decision comprises updating the decision not to restrict the first virtual object to a decision to restrict the first virtual object; and

wherein reverting a state of the display device to reflect a state that would have resulted had the correct restriction decision been made comprises reverting a state of the display device to reflect a state that would have resulted had the first virtual object been restricted.

17. The method of claim 1:

wherein detecting that the prior restriction decision applied to the first virtual object was incorrect comprises detecting that a decision to restrict the first virtual object was incorrect;

wherein updating the prior restriction decision applied the first virtual object to the correct restriction decision comprises updating the decision to restrict the first virtual object to a decision not to restrict the first virtual object; and

wherein reverting a state of the display device to reflect a state that would have resulted had the correct restriction decision been made comprises reverting a state of the display device to reflect a state that would have resulted had the first virtual object not been restricted.

18. A computing system, comprising:

a display device;

a non-volatile memory device; and

a processor configured to;

display multiple virtual objects on the display device;

store, in the non-volatile memory device, information regarding a state of the display device, the state including a location, a position, and restrictions placed on each virtual object of the multiple virtual objects displayed on the display device;

detect a stylus input on a subset of the multiple virtual objects;

based on at least detecting the stylus input on the subset of the multiple virtual objects, restricting, for the subset of the multiple virtual objects, responsiveness to a gesture input;

detect the gesture input applied to a first virtual object included in the subset of the multiple virtual objects and a second virtual object included in the multiple virtual objects, but excluded from the subset of the multiple virtual objects;

based on detecting the gesture input applied to the first virtual object and the second virtual object;

store, in the non-volatile memory device, the gesture input applied to the first virtual object and the second virtual object, and the restriction to responsiveness to the gesture input for the first virtual object,

control a movement of the first virtual object in accordance with the restricted responsiveness to the gesture input,

control a movement of the second virtual object without restriction,

detect that a prior restriction decision applied to the first virtual object was incorrect,

update, based on at least detecting that the prior restriction decision applied to the first virtual object was incorrect, the prior restriction decision applied to the first virtual object to a correct restriction decision, and

revert, based on at least the information regarding the state of the display device, the stored gesture input applied to the first virtual object, and the correct restriction decision, a state of the display device to reflect a state that would have resulted had the correct restriction decision been made.