Dynamic interactive objects

• US 10,620,775 B2
• Filed: 05/19/2014
• Issued: 04/14/2020
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Ultrahaptics Ip Two Limited (Ultraleap Holdings Ltd.)
• Inventors: Holz, David S.
• Primary Examiner(s): Parker, Jeanette J
• Application Number:
  US14/281,825
• Publication Number:
  US 20140344731A1
• Time in Patent Office: 2,157 Days
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects

Abstract

Aspects of the systems and methods are described providing for modifying a presented interactive element or object, such as a cursor, based on user-input gestures, the presented environment of the cursor, or any combination thereof. The color, size, shape, transparency, and/or responsiveness of the cursor may change based on the gesture velocity, acceleration, or path. In one implementation, the cursor “stretches” to graphically indicate the velocity and/or acceleration of the gesture. The display properties of the cursor may also change if, for example, the area of the screen occupied by the cursor is dark, bright, textured, or is otherwise complicated. In another implementation, the cursor is drawn using sub-pixel smoothing to improve its visual quality.

Claims

1. A system for providing visual feedback in response to a multi-finger gesture, the system comprising:

an image-capture device configured to capture digital images of two or more fingers of a user'"'"'s hands being tracked in a 3D space, each of the two or more fingers having a separate interactive object, of at least two interactive objects, associated therewith;

a computer memory configured to store a digital representation of a path of a movement of the two or more fingers of the user'"'"'s hands based on the digital images captured; and

a processor implementing;

presenting, on a presentation device, the interactive objects associated with the two or more fingers, wherein a position of each presented interactive object changes in response to the movement of the associated finger of the two or more fingers of the user'"'"'s hands;

determining a property of the movement of at least one of the two or more fingers of the user'"'"'s hands and modifying a first presentation property selected from a set consisting of a color, a size and a shape of at least one of the presented interactive objects based on the determined property of the movement; and

determining a use mode for at least one of the presented interactive objects and modifying a second presentation property of the at least one of the presented interactive objects based on the use mode determined for the at least one of the presented interactive objects, wherein the use mode is determined based on an input from an application or an input from a user,

wherein the use mode is determined to be one of;

a first mode in which the at least one of the presented interactive objects moves but does not interact with any presented elements; and

a second mode in which the at least one of the presented interactive objects interacts with a presented element, and

wherein the at least one of the presented interactive objects is a cursor.

2. The system of claim 1, wherein the property of the movement comprises at least one of a velocity of at least one of the two or more fingers of the user'"'"'s hands along the path and an acceleration of at least one of the two or more fingers of the user'"'"'s hands along the path.

3. The system of claim 1, further comprising a display configured to display the presented interactive objects.

4. The system of claim 1, wherein modifying a presentation property of the at least one of the presented interactive objects comprises changing at least one of a color, a size, a shape, and a transparency of the at least one of the presented interactive objects.

5. The system of claim 1, wherein modifying a presentation property of the at least one of the presented interactive objects comprises at least one of stretching the at least one of the presented interactive objects based on at least one of a velocity and an acceleration of at least one of the two or more fingers of the user'"'"'s hands, changing a color of the at least one of the presented interactive objects based on at least one of the velocity and the acceleration of the at least one of the two or more fingers of the user'"'"'s hands, and changing a color of the at least one of the presented interactive objects based on at least one of a color, a texture, and a pattern of a screen area along the path.

6. The system of claim 1,

wherein the processor further implements detecting that the path of the movement does not correspond to a known gesture, and

wherein the modifying of the first presentation property of the at least one of the presented interactive objects comprises changing a color of the at least one of the presented interactive objects in response to detecting that the path of the movement does not correspond to the known gesture.

7. The system of claim 1, wherein the processor further implements detecting a presented element to which at least one of the presented interactive objects is moving towards and changing a color of at least one of the presented interactive objects to indicate interaction with the presented element.

8. A method for providing visual feedback in response to a multi-finger gesture, the method comprising:

capturing, using an image-capture device, digital images of two or more fingers of a user'"'"'s hands being tracked in a 3D space, each of the two or more fingers having a separate interactive object, of at least two interactive objects, associated therewith;

storing, in a computer memory, a digital representation of a path of a movement of the two or more fingers of the user'"'"'s hands based on the digital images captured;

presenting, on a presentation device, the interactive objects associated with the two or more fingers, wherein a position of each presented interactive object changes in response to the movement of the associated finger of the two or more fingers of the user'"'"'s hands;

determining a property of the movement of at least one of the two or more fingers of the user'"'"'s hands and modifying a first presentation property selected from a set consisting of a color, a size, and a shape of at least one of the presented interactive objects based on the determined property of the movement; and

determining a use mode for at least one of the presented interactive objects and modifying a second presentation property of the at least one of the presented interactive objects based on the use mode determined for the at least one of the presented interactive objects, wherein the use mode is determined based on an input from an application or an input from a user,

wherein the use mode is determined to be one of;

a first mode in which the at least one of the presented interactive objects moves but does not interact with any presented elements; and

a second mode in which the at least one of the presented interactive objects interacts with a presented element, and

wherein the at least one of the presented interactive objects is a cursor.

9. The method of claim 8, wherein the property of the movement comprises at least one of a velocity of at least one of the two or more fingers of the user'"'"'s hands along the path and an acceleration of at least one of the two or more fingers of the user'"'"'s hands along the path.

10. The method of claim 8, further comprising displaying, on a display, the presented interactive objects.

11. The method of claim 8, wherein modifying a presentation property of the at least one of the presented interactive objects comprises changing at least one of a color, a size, a shape, and a transparency of the at least one of the presented interactive objects.

12. The method of claim 8, wherein the modifying of the first presentation property of the at least one of the presented interactive objects comprises at least one of:

stretching the at least one of the presented interactive objects based on at least one of a velocity and an acceleration of at least one of the two or more fingers of the user'"'"'s hands,

changing a color of the at least one of the presented interactive objects based on at least one of the velocity and the acceleration of the at least one of the two or more fingers of the user'"'"'s hands, and

changing a color of the at least one of the presented interactive objects based on at least one of a color, a texture, and a pattern of a screen area along the path.

13. The method of claim 8, further including detecting that the path of the movement does not correspond to a known gesture,

wherein the modifying of the first presentation property of the at least one of the presented interactive objects comprises changing a color of the at least one of the presented interactive objects in response to detecting that the path of the movement does not correspond to the known gesture.

14. The method of claim 8, further including detecting a presented element to which at least one of the presented interactive objects is moving towards and modifying the second presentation property of the at least one of the presented interactive objects to indicate interaction with the presented element.

15. The method of claim 14, further including clicking on the presented element when the at least one of the presented interactive objects interacts with the presented element.

16. The method of claim 8, wherein at least one of the presented interactive objects is presented in a first color when the at least one of the presented interactive objects is in a first use mode;

and

the at least one of the presented interactive objects is presented in a second color when the at least one of the presented interactive objects is in a second use mode, wherein the first and second colors are visually distinct.

17. The method of claim 8, further including selecting a current use mode based upon a user'"'"'s pattern of action.

18. The method of claim 8, further including selecting a current use mode based upon a command received from a running application.

19. The method of claim 8, further including selecting a current use mode based upon a command received from a user.

20. A method for providing visual feedback in response to a multi-finger gesture, the method comprising:

capturing, using an image-capture device, digital images of two or more fingers of a user'"'"'s hands being tracked in a 3D space, each of the two or more fingers having a separate interactive object, of at least two interactive objects, associated therewith and the separate interactive objects associated with the two or more fingers being for presentation on a screen;

storing, in a computer memory, a digital representation of a path of a movement of the two or more fingers of the user'"'"'s hands based on the digital images captured;

determining a current use mode for a presented interactive object displayed on the screen and associated with one of the two or more fingers, the use mode selected from one of a movement mode and an action mode,

wherein when a movement mode is selected, the presented interactive object moves but does not interact with any presented elements,

wherein when an action mode is selected, the presented interactive object interacts with a presented element displayed on the screen but does not move from the presented element, and

wherein selection of current use mode is in response to a command from a running application;

modifying a first presentation property of the presented interactive object based on the determined current use mode of the presented interactive object;

when the determined current use mode is movement mode (i) changing a position of the presented interactive object displayed on the screen in response to the movement of at least one of the two or more fingers of the user'"'"'s hands and (ii) modifying at least a second presentation property of the presented interactive object based on a property of the movement; and

when the determined current use mode is action mode, the presented interactive object interacts with a presented element displayed on the screen in response to a user command by at least one of the two or more fingers of the user'"'"'s hands,

wherein the first presentation property is a different property from the second presentation property,

wherein the presented interactive object is presented in a first color when the presented interactive object is in a first use mode, the presented interactive object is presented in a second color when the presented interactive object is in a second use mode, and wherein the first and second colors are visually distinct, and

wherein the presented interactive object is a cursor.