Performing virtual reality input

• US 10,509,535 B2
• Filed: 10/24/2017
• Issued: 12/17/2019
• Est. Priority Date: 10/26/2016
• Status: Active Grant

• Current Assignee: Advanced New Technologies Company Limited (Ant Group Co., Ltd.)
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Yin, Huanmi
• Primary Examiner(s): Rodriguez, Daniel
• Application Number:
  US15/792,628
• Publication Number:
  US 20180113599A1
• Time in Patent Office: 784 Days
• Field of Search: 715850
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

A virtual reality (VR) device presents a virtual element in a VR scenario of a VR application. An effective interaction region (EIR) of the virtual element is determined that the virtual element can effectively interact with. A trajectory of an operational focus of the VR application is determined based on a movement sensed by the VR device. The virtual element is selected if an interaction between the trajectory of the operational focus and the EIR satisfies one or more conditions for triggering a selection event of the virtual button.

Claims

1. A computer-implemented method, comprising:

presenting, by a virtual reality (VR) device, a virtual element in a VR scenario of a VR application;

determining an effective interaction region (EIR) of the virtual element that the virtual element can effectively interact with, the EIR having an area less than an area of the virtual element;

determining, based on a movement sensed by the VR device, coordinates of an operation focus of the VR application in an XY reference in the VR scenario;

generating, based upon the determined coordinates, a trajectory of the operational focus of the VR application in the VR scenario;

detecting, by the VR application, whether the trajectory passes through the EIR of the virtual element; and

selecting, by the VR device, the virtual element if a continuous V-shaped movement interaction between the trajectory of the operational focus and the EIR satisfies one or more conditions of interaction for triggering a selection event of the virtual element, the one or more conditions of interaction include initiation and termination of the continuous V-shaped movement interaction at locations outside of the area of the virtual element, and the trajectory of the operational focus intersects with the two parallel boundaries of the EIR more than four times within a predetermined time period.

2. The computer-implemented method of claim 1, wherein the EIR overlaps at least a portion of the area of the virtual element.

3. The computer-implemented method of claim 1, wherein the movement sensed by the VR device includes at least one of a head movement or a hand movement.

4. The computer-implemented method of claim 1, wherein the virtual element is a virtual button or a virtual key that is presented for performing user input.

5. The computer-implemented method of claim 1, wherein the operational focus is a cursor presented in the VR scenario and is determined based on a VR device user'"'"'s visual focus, head position, or hand position.

6. The computer-implemented method of claim 1, wherein the one or more conditions of interaction include the trajectory of the operational focus intersects with the two parallel boundaries of the EIR at least five times.

7. The computer-implemented method of claim 1, wherein the one or more conditions of interaction include the trajectory of the operational focus forms an enclosure in the EIR.

8. The computer-implemented method of claim 7, wherein an area of the enclosure is greater than a predetermined percentage of the area of the EIR.

9. The computer-implemented method of claim 1, wherein the one or more conditions of interaction include that the continuous V-shaped movement interaction between the trajectory of the operational focus and the EIR is performed in a time period less than or equal to a predetermined time limit.

10. The computer-implemented method of claim 1, wherein the trajectory of the operational focus intersects with a first of the two parallel boundaries of the EIR at least twice after the trajectory of the operational focus intersects with a second of the two parallel boundaries of the EIR.

11. The computer-implemented method of claim 1, wherein the trajectory of the operational focus intersects with each of the two parallel boundaries of the EIR along a first trajectory direction and along a second trajectory direction substantial opposite to the first trajectory direction.

12. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

presenting, by a virtual reality (VR) device, a virtual element in a VR scenario of a VR application;

determining an effective interaction region (EIR) of the virtual element that the virtual element can effectively interact with, the EIR having an area less than an area of the virtual element;

determining, based on a movement sensed by the VR device, coordinates of an operation focus of the VR application in an XY reference in the VR scenario;

generating, based upon the determined coordinates, a trajectory of the operational focus of the VR application in the VR scenario;

detecting, by the VR application, whether the trajectory passes through the EIR of the virtual element; and

selecting, by the VR device, the virtual element if a continuous V-shaped movement interaction between the trajectory of the operational focus and the EIR satisfies one or more conditions of interaction for triggering a selection event of the virtual element, the one or more conditions of interaction include initiation and termination of the continuous V-shaped movement interaction at locations outside of the area of the virtual element, and the trajectory of the operational focus intersects with the two parallel boundaries of the EIR for more than four times within a predetermined time period.

13. The non-transitory, computer-readable medium of claim 12, wherein the EIR overlaps at least a portion of the area of the virtual element.

14. The non-transitory, computer-readable medium of claim 12, wherein the movement sensed by the VR device includes at least one of a head movement or a hand movement.

15. The non-transitory, computer-readable medium of claim 12, wherein the virtual element is a virtual button or a virtual key that is presented for performing user input.

16. The non-transitory, computer-readable medium of claim 12, wherein the operational focus is a cursor presented in the virtual scenario and is determined based on a VR device user'"'"'s visual focus, head position, or hand position.

17. The non-transitory, computer-readable medium of claim 12, wherein the one or more conditions of interaction include the trajectory of the operational focus intersects with the two parallel boundaries of the EIR at least five times.

18. The non-transitory, computer-readable medium of claim 12, wherein the one or more conditions of interaction include the trajectory of the operational focus forms an enclosure in the EIR.

19. The non-transitory, computer-readable medium of claim 18, wherein an area of the enclosure is greater than a predetermined percentage of the area of the EIR.

20. The non-transitory, computer-readable medium of claim 12, wherein the one or more conditions of interaction include that the continuous V-shaped movement interaction between the trajectory of the operational focus and the EIR is performed in a time period less than or equal to a predetermined time limit.

21. The non-transitory, computer-readable medium of claim 12, wherein the trajectory of the operational focus intersects with a first of the two parallel boundaries of the EIR at least twice after the trajectory of the operational focus intersects with a second of the two parallel boundaries of the EIR.

22. The non-transitory, computer-readable medium of claim 12, wherein the trajectory of the operational focus intersects with each of the two parallel boundaries of the EIR along a first trajectory direction and along a second trajectory direction substantial opposite to the first trajectory direction.

23. A computer-implemented system, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform operations comprising;

presenting, by a virtual reality (VR) device, a virtual element in a VR scenario of a VR application;

determining an effective interaction region (EIR) of the virtual element that the virtual element can effectively interact with, the EIR having an area less than an area of the virtual element;

determining, based on a movement sensed by the VR device, coordinates of an operation focus of the VR application in an XY reference in the VR scenario;

generating, based upon the determined coordinates, a trajectory of the operational focus of the VR application in the VR scenario;

detecting, by the VR application, whether the trajectory passes through the EIR of the virtual element; and

selecting, by the VR device, the virtual element if a continuous V-shaped movement interaction between the trajectory of the operational focus and the EIR satisfies one or more conditions of interaction for triggering a selection event of the virtual element, the one or more conditions of interaction include initiation and termination of the continuous V-shaped movement interaction at locations outside of the area of the virtual element, and the trajectory of the operational focus intersects with the two parallel boundaries of the EIR for more than four times within a predetermined time period.

24. The computer-implemented system of claim 23, wherein the virtual element is a virtual button or a virtual key that is presented for performing user input.

25. The computer-implemented system of claim 23, wherein the trajectory of the operational focus intersects with a first of the two parallel boundaries of the EIR at least twice after the trajectory of the operational focus intersects with a second of the two parallel boundaries of the EIR.

26. The computer-implemented system of claim 23, wherein the trajectory of the operational focus intersects with each of the two parallel boundaries of the EIR along a first trajectory direction and along a second trajectory direction substantial opposite to the first trajectory direction.