Remote control augmented motion data capture

• US 9,377,626 B2
• Filed: 02/18/2015
• Issued: 06/28/2016
• Est. Priority Date: 02/18/2014
• Status: Expired due to Fees

• Current Assignee: Merge Labs, Inc.
• Original Assignee: Merge Labs, Inc.
• Inventors: Lyons, Franklin A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US14/625,602
• Publication Number:
  US 20150235426A1
• Time in Patent Office: 496 Days
• Field of Search: 345/7, 345/8, 345/156, 345/161, 345/173, 345/184, 345/620, 345/625, 345/633, 345/661, 359/630, 359/631, 348/36, 348/77, 348/207, 349/13
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/98 Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0154 with movable elements
• CPC Class Codes: G02B 2027/0169 Supporting or connecting means other than the external walls
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/022 Viewing apparatus G02B27/04, G02B27/06, G02B27/08 take precedence
• CPC Class Codes: G02B 27/028 characterised by the supporting structure
• CPC Class Codes: G02B 30/34 Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
• CPC Class Codes: G02B 7/023 permitting adjustment
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0231 : Cordless keyboards
• CPC Class Codes: G06F 3/0338 : with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/03545 : Pens or stylus
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A head mounted display system for use with a mobile computing device, comprises a soft main body made entirely of a soft and compressible material, the main body has a retention pocket entirely formed by the material and configured to accept and secure the mobile computing device and a lens assembly comprising two lenses configured to focus vision on respective areas of a display screen of the mobile computing device, the lens assembly held within one or more apertures formed in the main body entirely by the material, the two lenses mounted for independent movement with respect to each other, such that a split screen image may be viewed through the two lenses on the display screen.

Claims

1. A head mounted display system comprising:

a main body configured to be worn on a human head and defining a retention pocket configured to accept and secure therein a mobile computing device having a display and hardware for producing mobile computing device motion data used to perform head tracking of the human head when the main body is worn; and

a remote controller, distinct from, but in communication with, the mobile computing device, which when detachably attached to the exterior of the main body begins providing motion data to the mobile computing device that is used by the mobile computing device in performing head tracking of the human head and causes the mobile computing device to cease using mobile computing device motion data to perform head tracking of the human head, the remote controller incorporating a remote controller processor, at least one motion sensor for providing the motion data, and a sensor for detecting attachment of the remote controller to the main body and, when detachably secured to the main body as determined by the sensor.

2. The head mounted display system of claim 1 wherein the sensor comprises a depressible button such that when the remote controller is detachably secured to the exterior of the main body, the depressible button is depressed by the exterior of the main body thereby indicating that the remote controller is attached to the exterior of the main body.

3. The head mounted display system of claim 2 wherein the sensor comprises a depressible button configured so as to be depressed and retained in a depressed state for the duration of the attachment.

4. The head mounted display system of claim 1 wherein the at least one motion sensor comprises a gyroscope and the at least one motion sensor provides motion data to the computing device so that images displayed on the display may be updated to reflect motion of the head mounted display system.

5. The head mounted display system of claim 1 wherein the at least one motion sensor comprises a gyroscope and the at least one motion sensor is used in conjunction with the remote controller processer to generate motion data calculations in place of any made by the computing device that are then passed on to the computing device so that images displayed on the display may be updated to reflect motion of the head mounted display system.

6. The head mounted display system of claim 1 wherein the remote controller includes a plurality of unique control buttons on a front face thereof none of which are alike, wherein the control buttons face away from the main body when the remote controller is secured to the exterior of the main body.

7. The head mounted display system of claim 1 wherein the remote controller is a first remote controller and further including a second remote controller detachably secured to the exterior of the main body on a side opposite the first remote controller, the second remote controller, distinct from the first remote controller and the mobile computing device, detachably secured to the exterior of the main body and being in communication with the computing device, the second remote controller incorporating a remote controller processor and at least one second motion sensor and, when detachably secured to the main body, the second remote controller providing motion data from the at least one second motion sensor to the computing device to combine with any motion data produced by the mobile computing device and the first remote controller to assist the computing device in performing head tracking of the head mounted display system.

8. The head mounted display system of claim 1, wherein the remote controller has a forward-facing camera lens when the remote controller is secured to the exterior of the main body, wherein images received by the camera lens are communicated to the mobile computing device to be displayed on the display of the computing device.

9. A method of performing head tracking for a head mounted display system comprising:

processing mobile computing device motion data, provided by a mobile computing device including a display to thereby derive motion instructions suitable for use in performing head tracking in order to update an image shown on the display;

automatically detecting detachable attachment of a remote controller, distinct from the mobile computing device, to the main body using a sensor on the remote controller;

disabling head tracking using the mobile computing device motion data in response to attachment of the remote controller detected by the sensor;

generating motion data using at least one motion sensor within the remote controller detachably attached to the head-mounted main body of the system for the duration of the sensor detection that the remote controller is attached to the head-mounted main body;

communicating the motion data from the remote controller to the mobile computing device;

processing the motion data from the remote controller to perform head tracking; and

updating an image shown on the display of the computing device according to the processed motion data.

10. The method of claim 9 wherein the automatically detecting attachment of the remote controller comprises detecting that a depressible button has been depressed by the exterior of the main body when the remote controller is attached to the exterior of the main body thereby indicating that the remote controller is attached to the exterior of the main body.

11. The method of claim 9 further comprising automatically detecting that the remote controller has been detached from the head-mounted main body.

12. The method of claim 11 further comprising ceasing head tracking using the motion data and resuming head tracking using mobile computing device motion data in response to detachment of the remote controller from the head mounted display.

13. The method of claim 12 further comprising using the remote controller to generate control data that is used for purposes other than to perform head tracking.

14. The method of claim 13 wherein the purposes are selected from the group comprising controlling an avatar visible on the display, inputting commands for interaction with the environment shown on the display, and making selections from a menu on the display.

15. A head mounted display system comprising:

a main body configured to be worn on a human head and defining a retention pocket configured to accept and secure therein a mobile computing device having a processor and a display, the mobile computing device capable of creating mobile computing device motion data that is used to perform head tracking in order to update an image shown on the display;

the main body further comprising an attachment structure on the exterior of the main body for attaching a remote controller to the exterior of the main body;

a set of two lenses disposed within the main body between the eyes of a wearer and the display of the mobile computing device, the lenses for focusing the vision of the wearer on respective portions of the display so at to generate a stereoscopic three-dimensional image to the wearer; and

a remote controller, distinct from, but in communication with, the mobile computing device, which when detachably attached to the exterior of the main body by mating with the attachment structure and, thereby, causing depression of an integrated switch indicating that the remote control is detachably attached to the main body, the remote controller begins providing motion data to the mobile computing device that is used by the mobile computing device in performing head tracking of the human head and causes the mobile computing device to cease using mobile computing device motion data to perform head tracking of the human head, the remote controller incorporating a remote controller processor and at least one motion sensor for providing the motion data.

16. The system of claim 15 wherein:

engagement of the docking feature with the switch causes the remote control processor to commence providing motion data to the mobile computing device and causes the mobile computing device to cease providing mobile computing device motion data for use in performing head tracking.

17. The head mounted display system of claim 15 wherein the remote controller includes a plurality of unique control buttons on a front face thereof none of which are alike, wherein the control buttons face away from the main body when the remote controller is secured to the exterior of the main body.

18. The head mounted display system of claim 15 wherein the remote controller is a first remote controller and further including a second remote controller detachably secured to a second attachment structure having a second docking feature on the exterior of the main body on a side opposite the first remote controller causing the second docking feature to depress a second integrated switch indicating that the second remote control is detachably connected to the main body, the second remote controller also being in communication with the mobile computing device, the second remote controller incorporating a remote controller processor and at least one second motion sensor and, when detachably secured to the main body, the second remote controller providing motion data used to perform head tracking of the head mounted display system in place of the mobile computing device motion data.

19. The head mounted display system of claim 15, wherein the remote controller has a forward-facing camera lens when the remote controller is secured to the exterior of the main body, wherein images received by the camera lens can be displayed on the display of the computing device.

20. The head mounted display system of claim 15 wherein disengagement of the docking feature with the switch causes the remote to cease generating motion data for use in performing head tracking and causes the mobile computing device to commence generating mobile computing device motion data for use in performing head tracking.