System and method for assisting a user in remaining in a selected area while the user is in a virtual reality environment

• US 9,851,786 B2
• Filed: 07/07/2015
• Issued: 12/26/2017
• Est. Priority Date: 12/10/2014
• Status: Active Grant

• Current Assignee: Penumbra, Inc.
• Original Assignee: Sixense Entertainment Incorporated
• Inventors: Pedrotti, Owen Walter, Diaz, Alejandro Santiago, Braman, Steven L., Woodall, Danny G., Rubin, Amir
• Primary Examiner(s): Cobb, Michael J
• Application Number:
  US14/793,731
• Publication Number:
  US 20160171771A1
• Time in Patent Office: 903 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0167 Emergency system, e.g. to prevent injuries
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/012 Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves

Abstract

A method and apparatus is disclosed for assisting a user, wearing a head mounted display (HMD) that covers a user'"'"'s field of vision and has a tracker providing information regarding the position and orientation of the HMD, in locating a physical controller located on a physical base station. A processor causes the HMD to display a virtual world, including a virtual representation of the physical base station and physical controller along with a virtual hand that helps guide the user to the physical base station to allow the user to pick up the physical controller. Another embodiment allows an area in the physical world to be defined within which the user should remain, for example to avoid physical obstacles. The processor causes the HMD to display a warning, such as a virtual fence, to alert the user if the user approaches to within a preselected distance of the boundary.

Claims

1. A method of assisting a user wearing a head-mounted display (HMD) in remaining within a selected area of the physical world, the user holding a handheld controller having a controller tracker which provides information regarding a position of the handheld controller, the method comprising:

determining, by a processor, a definition of a boundary of the selected area by;

determining in sequence, by the processor, the location of the handheld controller, in the physical world and relative to a tracker located on the user'"'"'s body, at each of a plurality of points using information from the controller tracker; and

connecting in sequence, by the processor, the plurality of determined locations to form the boundary;

determining, by the processor, the position of the handheld controller in the physical world using information from the controller tracker;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a virtual world;

determining, by the processor, from the position of the handheld controller that the handheld controller is within a first preselected distance from the boundary;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a first visual warning in the virtual world.

2. The method of claim 1 wherein:

generating and outputting instructions to the HMD to display the virtual world further comprises generating and outputting instructions to the HMD to display on the HMD a virtual hand in the virtual world, the virtual hand having an apparent location in the virtual world that corresponds to the position of the handheld controller in the physical world; and

the first visual warning is a pattern of grid lines in the vicinity of the displayed virtual hand.

3. The method of claim 1, wherein the HMD has a HMD tracker which provides information regarding the position and orientation of the HMD, the method further comprising:

determining, by the processor, the position of the HMD in the physical world using information from the HMD tracker;

determining, by the processor, from the position of the HMD that the HMD is within a second preselected distance of the boundary;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a second visual warning in the virtual world.

4. The method of claim 3 wherein the second visual warning is a pattern of grid lines having an apparent location in the virtual world immediately in front of the user.

5. The method of claim 3 wherein the first visual warning and the second visual warning are the same.

6. The method of claim 3 wherein the first visual warning and the second visual warning are different.

7. The method of claim 3 wherein the first preselected distance and the second preselected distance are the same distance.

8. The method of claim 3 wherein the first preselected distance and the second preselected distance are different distances.

9. A method of assisting a user wearing a head-mounted display (HMD) in remaining within a selected area of the physical world, the user holding a handheld controller having a controller tracker which provides information regarding a position of the handheld controller, the method comprising:

determining, by a processor, a definition of a boundary of the selected area by;

determining, by the processor, a location of the handheld controller in the physical world using information from the controller tracker, the determined position being relative to a tracker located on the user'"'"'s body;

receiving, by the processor, a distance measurement; and

defining, by the processor, the boundary as having a circular shape, centered at the determined location and having a radius equal to the received distance measurement;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a virtual world;

determining, by the processor, from the position of the handheld controller that the handheld controller is within a first preselected distance from the boundary;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a first visual warning in the virtual world.

10. A system to assist a user in remaining in a selected area of the physical world, the user wearing a head-mounted display (HMD) and holding a handheld controller, comprising:

a controller tracker on the handheld controller which provides information regarding a position of the handheld controller; and

a processor configured to;

determine a definition of a boundary of the selected area by

determining in sequence the location of the handheld controller, in the physical world and relative to a tracker located on the user'"'"'s body, at each of a plurality of points using information from the controller tracker, and

connecting in sequence the plurality of determined locations to form the boundary;

determine the position of the handheld controller in the physical world using information from the controller tracker;

generate and output instructions to the HMD to display on the HMD a virtual world;

determine from the position of the handheld controller that the handheld controller is within a first preselected distance from the boundary; and

generate and output instructions to the HMD to display on the HMD a first visual warning in the virtual world.

11. The system of claim 10 wherein the processor is further configured to generate and output instructions to the HMD to display on the HMD a virtual hand in the virtual world, the virtual hand having an apparent location in the virtual world that corresponds to the position of the handheld controller in the physical world;

and

wherein the first visual warning is a pattern of grid lines in the vicinity of the displayed virtual hand.

12. The system of claim 10 further comprising a HMD tracker on the HMD which provides information regarding the position and orientation of the HMD, and wherein the processor is further configured to:

determine the position of the HMD in the physical world using information from the HMD tracker;

determine from the position of the HMD that the HMD is within a second preselected distance of the boundary;

generate and output instructions to the HMD to display on the HMD the visual warning.

13. The system of claim 12 wherein the second visual warning is a pattern of grid lines having an apparent location in the virtual world immediately in front of the user.

14. The system of claim 12 wherein the first visual warning and the second visual warning are the same.

15. The system of claim 12 wherein the first visual warning and the second visual warning are different.

16. The system of claim 12 wherein the first preselected distance and the second preselected distance are the same distance.

17. The system of claim 12 wherein the first preselected distance and the second preselected distance are different distances.

18. A system to assist a user in remaining in a selected area of the physical world, the user wearing a head-mounted display (HMD) and holding a handheld controller, the system comprising:

a controller tracker on the handheld controller which provides information regarding a position of the handheld controller; and

a processor configured to determine a definition of a boundary of the selected area by;

determining in sequence the location of the handheld controller in the physical world at each of a plurality of points using information from the controller tracker, the determined position being relative to a tracker located on the user'"'"'s body; and

connecting in sequence the plurality of determined locations to form the boundary;

determine a location of the handheld controller in the physical world using information from the controller tracker;

receive a distance measurement; and

define the boundary as having a circular shape, centered at the determined location and having a radius equal to the received distance measurement.

19. A non-transitory computer-readable medium having embodied thereon a program, the program being executable by a processor to perform a method of assisting a user wearing a head-mounted display (HMD) in remaining within a selected area of the physical world, the user holding a handheld controller having a controller tracker which provides information regarding a position and orientation of the controller, the method comprising the steps of:

determining, by a processor, a definition of a boundary of the selected area by;

determining in sequence, by the processor, the location of the handheld controller, in the physical world and relative to a tracker located on the user'"'"'s body, at each of a plurality of points using information from the controller tracker; and

connecting in sequence, by the processor, the plurality of determined locations to form the boundary;

determining, by the processor, the position of the handheld controller in the physical world using information from the controller tracker;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a virtual world;

determining, by the processor, from the position of the handheld controller that the handheld controller is within a first preselected distance from the boundary;

generating and outputting, by the processor, instructions to the HMD to display on the HMD a first visual warning in the virtual world.