Methods and apparatus for compensating for vehicular motion

• US 9,821,920 B2
• Filed: 08/18/2015
• Issued: 11/21/2017
• Est. Priority Date: 08/18/2015
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Cole, David, Moss, Alan McKay
• Primary Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US14/829,562
• Publication Number:
  US 20170050743A1
• Time in Patent Office: 826 Days
• CPC Class Codes: B64D 11/0015 Arrangements for entertainment or communications, e.g. radio, television
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

A head mounted virtual reality (VR) device including an inertial measurement unit (IMU) is located in a vehicle which may be, and sometimes is, moving. Detected motion attributable to vehicle motion is filtered out based on one or more or all of: vehicle type information, information derived from sensors located in the vehicle external to the head mounted VR device, and/or captured images including a reference point or reference object within the vehicle. An image portion of a simulated VR environment is selected and presented to the user of the head mounted VR device based on the filtered motion information. Thus, the image portion presented to the user of the head mounted VR device is substantially unaffected by vehicle motion and corresponds to user induced head motion.

Claims

1. A virtual reality (VR) method comprising:

receiving inertial measurement unit (IMU) measurement information obtained from an inertial measurement unit (IMU) within a head mounted VR device;

processing the received inertial measurement unit information to filter out at least some detected motion attributable to motion of a vehicle to generate filtered information, said filtered information including at least one of;

filtered orientation information or filtered motion information, wherein processing the received inertial measurement unit information to filter out at least some detected motion attributable to motion of the vehicle includes;

filtering out detected motion consistent with motion of a vehicle which is not consistent with expected user directed motion of a user'"'"'s head; and

using the filtered information in controlling which portion of a simulated environment is displayed to a user at a given time.

2. The method of claim 1, wherein said filtering out detected motion consistent with motion of a vehicle is based on information about a type of vehicle in which the virtual reality device is located.

3. The method of claim 2, further comprising:

receiving information from a user indicating the type of vehicle in which the virtual reality device is located or automatically determining the type of vehicle in which the virtual reality device is located.

4. The method of claim 3, further comprising:

wherein said step of filtering out detected motion consistent with motion of a vehicle includes using a first filter if the type of vehicle is an airplane type and a second filter if the type of vehicle is an automobile type, said first filter type taking into consideration expected changes in head angle corresponding to a plane taking off or landing.

5. The method of claim 1, further comprising:

receiving at least one of vehicle orientation information or vehicle motion information obtained from a sensor device located in the vehicle external to the head mounted VR device.

6. The method of claim 5, wherein said sensor device is one of a sensor device built into the vehicle or a sensor device placed in the vehicle by a user of said head mounted VR device.

7. The method of claim 6, wherein filtering out detected motion consistent with motion of a vehicle which is not consistent with expected user directed motion of a user'"'"'s head includes:

subtracting from motion detected by said IMU in the head mounted device at least some motion detected by said sensor.

8. The method of claim 1, further comprising:

capturing images using a camera included in said head mounted VR device;

determining if there is a change in the position of a reference point or reference object in the captured images.

9. The method of claim 8,

wherein processing the received inertial measurement unit information to filter out at least some detected motion attributable to motion of a vehicle to generate filtered information includes filtering out motion detected during a period of time in which there is no change in the position of the reference point or reference object in the captured images.

10. A head mounted virtual reality (VR) device comprising:

an inertial measurement unit (IMU);

a module configured to receive inertial measurement unit (IMU) measurement information obtained from the inertial measurement unit (IMU) within the head mounted VR device;

a module configured to process the received inertial measurement unit information to filter out at least some detected motion attributable to motion of a vehicle to generate filtered information, wherein said module configured to process the received inertial measurement unit information to filter out at least some detected motion attributable to motion of the vehicle includes;

a module configured to filter out detected motion consistent with motion of a vehicle which is not consistent with expected user directed motion of a user'"'"'s head; and

a module configured to use the filtered information in controlling which portion of a simulated environment is displayed to a user at a given time.

11. The head mounted VR device of claim 10, wherein said filtering out detected motion consistent with motion of a vehicle is based on information about a type of vehicle in which the virtual reality device is located.

12. The head mounted VR device of claim 11, further comprising at least one of:

a module configured to receive information from a user indicating the type of vehicle in which the virtual reality device is located;

or

a module configured automatically determine the type of vehicle in which the virtual reality device is located.

13. The head mounted VR device of claim 12,

wherein said module configured to filter out detected motion consistent with motion of a vehicle uses a first filter if the type of vehicle is an airplane type and a second filter if the type of vehicle is an automobile type, said first filter type taking into consideration expected changes in head angle corresponding to a plane taking off or landing.

14. The head mounted VR device of claim 10, further comprising:

a module configured to receive at least one of vehicle orientation or vehicle motion information obtained from a sensor device located in the vehicle external to the head mounted VR device.

15. The head mounted VR device of claim 14, wherein said sensor device is one of a sensor device built into the vehicle or a sensor device placed in the vehicle by a user of said head mounted VR device.

16. The head mounted VR device of claim 15, wherein said module configured to filter out detected motion consistent with motion of a vehicle which is not consistent with expected user directed motion of a user'"'"'s head includes:

a module configured to subtract from motion detected by said IMU in the head mounted device at least some motion detected by said sensor device.

17. The head mounted virtual reality (VR) device of claim 10, further comprising:

a camera;

a module configured to control said camera included in said head mounted VR device to capture images;

stored information corresponding to one or more reference points or reference objects which may appear in captured images; and

a module configured to determine if there is a change in the position of a reference point or reference object in the captured images.

18. The head mounted VR device of claim 17, further comprising:

a module configured to filter out motion detected during a period of time in which there is no change in the position of the reference point or reference object in the captured images.

19. A non-transitory computer readable medium including computer executable instructions which, when executed by a computer of a virtual reality (VR) device including memory and a display, control the VR device to perform the steps of:

receiving inertial measurement unit measurement information obtained from an inertial measurement unit of the VR device;

processing the received inertial measurement unit information to filter out at least some detected motion attributable to motion of a vehicle to generate filtered information, said filtered information including at least one of;

filtered orientation information or filtered motion information, wherein processing the received inertial measurement unit information to filter out at least some detected motion attributable to motion of the vehicle includes;

filtering out detected motion consistent with motion of a vehicle which is not consistent with expected user directed motion of a user'"'"'s head; and

using the filtered information in controlling which portion of a simulated environment is displayed to a user at a given time.