SENSOR BASED UI IN HMD INCORPORATING LIGHT TURNING ELEMENT

• US 20160054802A1
• Filed: 08/21/2014
• Published: 02/25/2016
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Dickerson, Andrew J., Van Den Heuvel, James E., Kim, Sophie J.
• Granted Patent: US 9,910,504 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0165 associated with a head-down display
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A head-mounted display (HMD) system includes a head-mounted frame that encases a portable electronic device (PED) that includes a touch-less sensor, and the PED. The frame includes: a front slot to hold the PED and maintain contact with the PED'"'"'s display face; an entry via dimensioned to prevent blockage of the sensor'"'"'s field of view; and a light turning element positioned to redirect the sensor'"'"'s field of view through an exit via in the frame and into open space outside of the frame. The PED includes: the sensor to detect a touch-less gesture; an image display to display image content on the PED'"'"'s front display face; and processing circuitry configured to change modes from handheld mode to HMD operation mode based on a determination that the PED is encased by the frame and to select the image content based on input signals received from the sensor and the mode.

Claims

1. A head-mounted display (HMD) system comprising:

a head-mounted frame configured to receive and encase a portable electronic device (PED) that includes a touch-less sensor, the frame includes;

a front slot configured to hold the PED and maintain contact with the display face of the PED,

an entry via dimensioned to prevent blockage of the sensor'"'"'s field of view, and

a light turning element positioned to redirect the sensor'"'"'s field of view through an exit via in the head-mounted frame and into open space outside of the head-mounted frame; and

the PED including;

the sensor configured to detect a touch-less gesture,

an image display configured to display image content on a front display face of the PED, and

processing circuitry configured to change modes from a handheld mode to a head-mounted display (HMD) operation mode based on a determination that the PED is encased by the frame and to select the image content based on input signals received from the sensor and the mode.

2. The HMD system of claim 1, wherein the frame is configured to slidably receive the PED in the front slot.

3. The HMD system of claim 1, wherein the frame is configured to block ambient light from the display face of the PED and permit ambient light within the field of view of the sensor.

4. The HMD system of claim 1, wherein the exit via includes a protective cover through which light in the redirected field of view passes.

5. The HMD system of claim 1, wherein the light turning element includes:

a first reflective surface configured to reflect a first type of light, and

a second reflective surface configured to redirect a second type of light.

6. The HMD system of claim 1, wherein the light turning element includes at least one of an optical prism and a mirror, and

wherein the light turning element is configured to reflect signals emitted by the sensor through the exit via to the open space outside of the head-mounted frame.

7. The HMD system of claim 6, wherein the sensor is a proximity sensor configured to emit electromagnetic radiation and in response to receiving return electromagnetic radiation reflected off an object in the proximity sensor'"'"'s field of view, transmit an input signal to the processing circuitry indicating a user selection;

and

wherein the light turning element further configured to;

in response to the emitted electromagnetic radiation falling incident upon the object, receive a portion of the electromagnetic radiation reflected by the object, through the exit via, and

orthogonally reflect the portion of the reflected electromagnetic radiation through the entry via to the proximity sensor.

8. The HMD system of claim 1, wherein:

the sensor comprises first and second non-touch sensors, each configured to detect a touch-less gesture;

the light turning element comprises;

a first light turning element positioned to redirect the first sensor'"'"'s field of view through a first exit via to the open space outside a first side of the frame, and

a second light turning element positioned to redirect the second sensor'"'"'s field of view through a second exit via to the open space another side of the frame; and

the head mount frame further comprises the second exit via on a different side of the head-mounted frame than the first exit via.

9. An apparatus comprising:

a head-mounted frame configured to receive a portable electronic device (PED) that includes a touch-less sensor, the frame includes;

a front slot configured to hold the PED and maintain contact with the display face of the PED;

an entry via dimensioned to prevent blockage of the sensor'"'"'s field of view; and

a light turning element positioned to redirect the sensor'"'"'s field of view through an exit via in the head-mounted frame and into open space outside of the head-mounted frame.

10. The apparatus of claim 9, wherein the frame is configured to slidably receive the PED in the front slot.

11. The apparatus of claim 9, wherein the frame is configured to block ambient light from the display face of the PED and permit ambient light within the field of view of the sensor.

12. The apparatus of claim 9, wherein the exit via includes a protective cover through which light in the redirected field of view passes.

13. The apparatus of claim 9, wherein the light turning element includes:

a first reflective surface configured to reflect a first type of light, and

a second reflective surface configured to redirect a second type of light.

14. The apparatus of claim 9, wherein the light turning element includes at least one of an optical prism and a mirror, and

wherein the light turning element is configured to reflect signals emitted by the sensor through the exit via to the open space outside of the head-mounted frame.

15. The apparatus of claim 14, wherein the light turning element further configured to:

in response to electromagnetic radiation emitted by the sensor falling incident upon an object in the sensor'"'"'s redirected field of view, receive a portion of the electromagnetic radiation reflected by the object, through the exit via, and

orthogonally reflect the portion of the reflected electromagnetic radiation through the entry via to the proximity sensor as return electromagnetic radiation.

16. The apparatus of claim 9, wherein:

the sensor comprises first and second non-touch sensors;

the light turning element comprises;

a first light turning element positioned to redirect the first sensor'"'"'s field of view through a first exit via to the open space outside a first side of the frame, and

a second light turning element positioned to redirect the second sensor'"'"'s field of view through a second exit via to the open space another side of the frame; and

the head mount frame further comprises the second exit via on a different side of the head-mounted frame than the first exit via.

17. A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that, when executed by processing circuitry, causes the processing circuitry to:

in response determining that a portable electronic device (PED) is encased by a head-mounted frame, change from a handheld mode to a head-mounted display (HMD) operation mode;

receive input signals from a non-touch sensor of the PED, each non-touch sensor configured to output the input signals in response to detecting a non-touch gesture in the sensor'"'"'s field of view, the non-touch gesture indicating a user selection, wherein the gesture indicates a different user selection when the PED is in the handheld mode than when the PED is in the HMD operation mode;

select image content based on the received input signals;

binocularly render the selected image content to left screen and right screen portions of a bifurcated image display.

18. The non-transitory computer readable medium of claim 17, the computer readable program code for receiving input signals from the non-touch sensor of the PED in the HMD operation mode further comprises computer readable program code that, when executed by the processing circuitry, causes the processing circuitry to:

display a cursor and interactive image content;

determine a direction of motion of the PED;

in response to receiving an input signal from the non-touch sensor indicating that no object is in close proximity to the non-touch sensor, one of;

display the cursor as an unclicked stationary cursor and movement of the interactive image content in a direction corresponding to the direction of motion of the PED with respect to the stationary cursor, and

display the interactive image content as stationary and movement of the cursor in a direction corresponding to the direction of motion of the PED with respect to the stationary interactive image content, and display the cursor as an unclicked moveable cursor; and

while the cursor points to a portion of the interactive image content, in response to receiving an input signal from the non-touch sensor indicating that an object is in close proximity to the non-touch sensor;

process the input signal as an equivalent of a touch-down on the pointed-to portion of the interactive image content in the hand-held mode,

change the selected image content to include;

a change of the cursor to a clicked cursor including different appearance than the unclicked stationary cursor and the unclicked moveable cursor, and the pointed-to portion of the interactive image content.

19. The non-transitory computer readable medium of claim 17, the computer readable program code for receiving input signals from the non-touch sensor of the PED in the HMD operation mode further comprises computer readable program code that, when executed by the processing circuitry, causes the processing circuitry to:

display a video on the image display;

while displaying the video, in response to receiving an input signal from the sensor indicating that an object in the sensor'"'"'s field of view is in close proximity to the sensor;

change the selected image content to include image content captured by a back camera, and

stop rendering the video.

20. The non-transitory computer readable medium of claim 17, the computer readable program code for receiving input signals from the non-touch sensor of the PED in the HMD operation mode further comprises computer readable program code that, when executed by the processing circuitry, causes the processing circuitry to:

display a first portion of a carousel of selectable options;

while displaying the first portion of the carousel, in response to receiving an input signal from the sensor indicating that an object moved across the sensor'"'"'s field of view in a direction in a linear motion, change the selected image content to include;

a scroll of intermediate selectable options between the first portion and the second portion select display followed by the second portion of the carousel.