×

Contextual cursor display based on hand tracking

  • US 10,409,443 B2
  • Filed: 06/24/2015
  • Issued: 09/10/2019
  • Est. Priority Date: 06/24/2015
  • Status: Active Grant
First Claim
Patent Images

1. A head mounted display device for contextual cursors in a virtual or augmented reality environment comprising:

  • a camera configured to capture a series of images;

    a display; and

    a processor configured to;

    display, to a user of the head mounted display device, a virtual cursor on the display so as to appear at a location in a three dimensional environment;

    determine a context based on the captured series of images by detecting a recognized virtual object in the three dimensional environment based on a gaze direction of the user;

    determine whether an intersection between the recognized virtual object and the gaze direction of the user is located at a planar surface, a resizing area, or a moving area of the recognized virtual object;

    select a programmatic function based on the determined context and whether the gaze direction is located at the planar surface, the resizing area, or the moving area of the recognized virtual object, wherein;

    when the intersection is located at the planar surface, a planar surface interaction function is selected, the planar surface interaction function being a drawing function, the drawing function being a programmatic function to draw an image onto the planar surface of the recognized object;

    when the intersection is located at the resizing area, a resizing function is selected, the resizing function being a programmatic function to change a size of the recognized object; and

    when the intersection is located at the moving area, a moving function is selected, the moving function being a programmatic function to move an object location of the recognized object to a different location in the three dimensional environment;

    determine whether motion of a hand of the user in the images is trackable;

    in response to at least determining that motion of the hand is trackable, modify a visual appearance of the virtual cursor to indicate the selected programmatic function and that motion of the hand is trackable;

    track motion of the hand in the images to identify a hand gesture of the user;

    in response to at least identifying the hand gesture, execute the programmatic function; and

    in response to at least determining that motion of the hand is not trackable, modify the visual appearance of the virtual cursor to indicate that motion of the hand is not trackable.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×