Head mounted display, motion detector, motion detection method, image presentation system and program

• US 9,448,405 B2
• Filed: 10/08/2013
• Issued: 09/20/2016
• Est. Priority Date: 11/06/2012
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Yamamoto, Hirotsugu
• Primary Examiner(s): Li, Lin
• Application Number:
  US14/048,753
• Publication Number:
  US 20140125579A1
• Time in Patent Office: 1,078 Days
• Field of Search: 345/8, 345157-158, 345/419, 345/633, 348 46- 47, 348/115, 348/241, 359/630, 361/681, 396/52
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Disclosed herein is a head mounted display including: an enclosure accommodating a presentation section adapted to present a three-dimensional image and located in front of the eyes of a viewer when worn on the head of the viewer; and an imaging element provided in the enclosure and adapted to turn light external to the enclosure into an image, in which the imaging element images light that is vertically downward relative to the enclosure and forward in the direction of line of sight of the viewer when the enclosure is worn on the head of the viewer.

Claims

1. A head mounted display comprising:

an enclosure accommodating a presentation section adapted to present a three-dimensional image and located in front of eyes of a viewer when worn on a head of the viewer;

a torso fastened imaging device configured for electronic communication with said presentation section, wherein the torso fastened imaging device images a subject including light radiated by a light emitting element of the enclosure;

an imaging element provided in the enclosure and adapted to turn light external to the enclosure into an image, wherein the light is in at least one of an invisible range and a visible range and is emitted by a light-emitting element attached to a grip section gripped by the viewer and that moves along with a motion of the viewer'"'"'s hand, and the imaging element images the light that is vertically downward relative to the enclosure and forward in a direction of a line of sight of the viewer when the enclosure is worn on the head of the viewer; and

a hand tracking block adapted to track a change in position of the light-emitting element by analyzing the image captured by the imaging element.

2. The head mounted display of claim 1, wherein

among sides making up the enclosure, at least part of a bottom side located vertically downward relative to the enclosure when the enclosure is worn on the head of the viewer has a sloped area that tilts vertically upward, and

the imaging element is provided on the sloped area.

3. The head mounted display of claim 1, wherein the imaging element acquires the light external to the enclosure via a fisheye lens and turns the light into an image.

4. The head mounted display of claim 1, wherein the hand tracking block comprises:

a bright spot detection unit adapted to detect a bright spot of the light in the image captured by the imaging element;

a vector acquisition unit adapted to acquire a vector having, as an end point, the bright spot detected by the bright spot detection unit, and, as a start point, a center of the image captured by the imaging element;

a pitch acquisition unit adapted to acquire, as a pitch of the light-emitting element relative to the imaging element, a length of the vector acquired by the vector acquisition unit; and

a yaw acquisition unit adapted to acquire, as a yaw of the light-emitting element relative to the imaging element, a declination of the vector acquired by the vector acquisition unit.

5. The head mounted display of claim 1, wherein

the imaging element generates the image by turning the light that is in the visible range into the image, the head mounted display further comprising:

an image combining section adapted to combine the visible light image and a virtual image to be presented by the presentation section.

6. The head mounted display of claim 5 further comprising:

a gesture recognition unit adapted to recognize a gesture of the viewer captured by the imaging element.

7. A motion detector comprising:

an image acquisition section adapted to acquire an image, captured by an imaging element of an enclosure attached to the head of a viewer by imaging infrared light radiated by a light-emitting element attached to a grip section gripped by the viewer and that moves along with the motion of the user'"'"'s hand, wherein the imaging element images the light that is vertically downward and forward in a direction of a line of sight of the viewer;

a hand tracking block adapted to track a change in position of the light-emitting element by analyzing the image captured by the imaging element;

a torso fastened imaging device configured for electronic communication with said image acquisition section, wherein the torso fastened imaging device images a subject including light radiated by a light emitting element of the enclosure;

a bright spot detection unit adapted to detect a bright spot of the infrared light in the image captured by the imaging element;

a vector acquisition unit adapted to acquire a vector having, as an end point, the bright spot detected by the bright spot detection unit, and, as a start point, a center of the image captured by the imaging element;

a pitch acquisition unit adapted to acquire, as a pitch of the light-emitting element relative to the imaging element, a length of the vector acquired by the vector acquisition unit; and

a yaw acquisition unit adapted to acquire, as a yaw of the light-emitting element relative to the imaging element, a declination of the vector acquired by the vector acquisition unit.

8. A motion detection method causing a processor to:

acquire an image captured by an imaging element attached to the head of a user by imaging infrared light radiated by a light-emitting element that moves along with the motion of the user'"'"'s hand;

acquire an image, captured by an imaging element of an enclosure attached to the head of a user, by imaging infrared light radiated by a light-emitting element attached to a grip section gripped by the viewer and that moves along with motion of the user'"'"'s hand, wherein the imaging element images the light that is vertically downward and forward in a direction of a line of sight of the user;

tracking a change in position of the light-emitting element by analyzing the image captured by the imaging element;

imaging a subject including light radiated by a light emitting element of the enclosure via a torso fastened imaging device;

detect a bright spot of infrared light in the acquired image;

acquire a vector having, as an end point, the detected bright spot, and, as a start point, a center of the image captured by the imaging element;

acquire a length of the acquired vector; and

acquire a declination of the acquired vector.

9. A non-transitory, computer readable recording medium containing a computer program, which when executed by a processor, causes the processor to perform the functions of:

acquiring an image captured by an imaging element of an enclosure attached to the head of a viewer, by imaging infrared light radiated by a light-emitting element attached to a grip section gripped by the viewer and that that moves along with the motion of the viewer'"'"'s hand, wherein the imaging element image the light that is vertically downward and forward in a direction of a line of sight of the viewer;

tracking a change in position of the light-emitting element by analyzing the image captured by the imaging element;

imaging a subject including light radiated by a light emitting element of the enclosure via a torso fastened imaging device;

detecting a bright spot of infrared light in the acquired image;

acquiring a vector having, as an end point, the detected bright spot, and, as a start point, a center of the image captured by the imaging element;

acquiring a length of the acquired vector; and

acquiring a declination of the acquired vector.

10. An image presentation system comprising:

a head mounted display;

a torso fastened imaging device configured for electronic communication with a presentation section, wherein the torso fastened imaging device images a subject including light radiated by a light emitting element of an enclosure of the head mounted display; and

a light-emitting element attached to a grip section gripped by a viewer and adapted to radiate infrared light and that move along with a motion of viewer'"'"'s hand,

the head mounted display includes;

the enclosure accommodating said presentation section adapted to present a three-dimensional image and located in front of eyes of the viewer when worn on a head of the viewer;

an imaging element provided in the enclosure and adapted to image the infrared light radiated by the light-emitting element that moves along with the motion of the grip section, wherein the imaging element images the light that is vertically downward relative to the enclosure and forward in a direction of a line of sight of the viewer when the enclosure is worn on the head of the viewer; and

a motion tracking section adapted to track the change in relative position of the light-emitting element with respect to the imaging element by analyzing the infrared light image captured by the imaging element.