Head-mounted display device, control method for the head-mounted display device, and authentication system

• US 10,191,555 B2
• Filed: 03/21/2016
• Issued: 01/29/2019
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Fujimaki, Yutaka
• Primary Examiner(s): Xavier, Antonio
• Application Number:
  US15/075,822
• Publication Number:
  US 20160202486A1
• Time in Patent Office: 1,044 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A head-mounted display device that enables a user to simultaneously visually recognize a virtual image and an outside scene includes a photographing unit configured to photograph at least a part of a visual field direction of the user in a state in which the user wears the head-mounted display device and acquire a motion of the user; a track acquiring unit configured to acquire a track of the motion of the user from the motion photographed by the photographing unit; and an authentication processing unit configured to authenticate, using the track acquired by the track acquiring unit, whether the user is a proper user.

Claims

1. A head-mounted display device that enables a user to simultaneously visually recognize a virtual image and an outside scene, the head-mounted display device comprising:

a camera configured to;

photograph at least a part of a visual field of the user in a state in which the user wears the head-mounted display device; and

acquire a motion of the user, the camera starting photographing of the motion when it detects a preset start trigger, and stopping photographing of the motion when it detects a preset end trigger;

a storage medium configured to store a track for authentication, which is registered in advance by a proper user; and

a processor configured to;

acquire a track of the motion of the user from the motion photographed by the camera;

authenticate, using the acquired track, whether the user is a proper user of the head-mounted display device;

determine that the user is the proper user when similarity between the acquired track and the stored track is equal to or higher than a first threshold; and

determine that the user is not the proper user when similarity between the acquired track and a second track is equal to or higher than a second threshold larger than the first threshold, the second track being obtained by converting a past acquired track,

wherein the start trigger and end trigger are different from the acquired track, and at least one of the start trigger and the end trigger are detection of stillness of the motion of the user for a predetermined time or more.

2. The head-mounted display device according to claim 1, wherein the processor is further configured to:

inform the user of an overlapping region where a display region of the virtual image and a photographing region of the camera overlap, the photographing region being a region, smaller than a total photographing region of the camera, where performance of the camera is assured.

3. The head-mounted display device according to claim 1, wherein at least one of the start trigger and the end trigger is detection of passage of the motion of the user in a predetermined position.

4. The head-mounted display device according to claim 1, wherein at least one of the start trigger and the end trigger is detection of a predetermined shape of a part of a body of the user.

5. The head-mounted display device according to claim 1, wherein the processor is further configured to:

specify the track of the motion according to changes in a moving direction and a movement amount of a part of a body of the user.

6. The head-mounted display device according to claim 5, wherein the processor is further configured to:

set the part of the body of the user as a specific point;

repeatedly acquire a position of the specific point at a predetermined time interval;

calculate, as the moving direction, a direction of a vector imaginarily connecting a position of the specific point acquired in n-th acquisition and a position of the specific point acquired in n+1th acquisition; and

calculate, as the movement amount, magnitude of the vector imaginarily connecting the position of the specific point acquired in the n-th acquisition and the position of the specific point acquired in the n+1th acquisition.

7. The head-mounted display device according to claim 1, wherein the processor is further configured to:

specify the track of the motion according to changes in a moving direction and a movement amount of an arbitrary object held by the user.

8. The head-mounted display device according to claim 7, wherein the processor is further configured to:

set an arbitrary place of the arbitrary object as a specific point;

repeatedly acquire a position of the specific point at a predetermined time interval;

calculate, as the moving direction, a direction of a vector imaginarily connecting a position of the specific point acquired in n-th acquisition and a position of the specific point acquired in n+1th acquisition; and

calculate, as the movement amount, magnitude of the vector imaginarily connecting the position of the specific point acquired in the n-th acquisition and the position of the specific point acquired in the n+1th acquisition.

9. The head-mounted display device according to claim 1, wherein the past acquired track is converted using at least one method among:

non-magnification, lateral direction enlargement, longitudinal direction enlargement, lateral direction reduction, and longitudinal direction reduction.

10. A head-mounted display device that enables a user to simultaneously visually recognize a virtual image and an outside scene, the head-mounted display device comprising:

a camera configured to;

photograph at least a part of a visual field of the user in a state in which the user wears the head-mounted display device; and

acquire a motion of the user, the photographing unit starts photographing the motion when it detects a preset start trigger, and stops photographing the motion when it detects a preset end trigger;

a storage medium configured to store a track for authentication, which is registered in advance by the proper user; and

a processor configured to;

acquire a track of the motion of the user from the motion photographed by the camera;

specify the track of the motion according to changes in a moving direction and a movement amount of a part of a body of the user;

authenticate, using the acquired track, whether the user is a proper user of the head-mounted display device;

determine that the user is the proper user when similarity between the acquired track and the stored track is equal to or higher than a first threshold; and

determine that the user is not the proper user when similarity between the acquired track and a second track is equal to or higher than a

second threshold larger than the first threshold, the second track being obtained by converting a past acquired track,

wherein the start trigger and end trigger are different from the acquired track.

11. The head-mounted display device according to claim 10, wherein the processor is further configured to:

inform the user of an overlapping region where a display region of the virtual image and a photographing region of the camera overlap, the photographing region being a region, smaller than a total photographing region of the camera, where performance of the camera is assured.

12. The head-mounted display device according to claim 10, wherein at least one of the start trigger and the end trigger is detection of stillness of the motion of the user for a predetermined time or more.

13. The head-mounted display device according to claim 10, wherein at least one of the start trigger and the end trigger is detection of passage of the motion of the user in a predetermined position.

14. The head-mounted display device according to claim 10, wherein at least one of the start trigger and the end trigger is detection of a predetermined shape of a part of a body of the user.

15. The head-mounted display device according to claim 10, wherein the processor is further configured to:

set the part of the body of the user as a specific point;

repeatedly acquire a position of the specific point at a predetermined time interval;

calculate, as the moving direction, a direction of a vector imaginarily connecting a position of the specific point acquired in n-th acquisition and a position of the specific point acquired in n+1th acquisition; and

calculate, as the movement amount, magnitude of the vector imaginarily connecting the position of the specific point acquired in the n-th acquisition and the position of the specific point acquired in the n+1th acquisition.

16. The head-mounted display device according to claim 10, wherein the past acquired track is converted using at least one method among:

non-magnification, lateral direction enlargement, longitudinal direction enlargement, lateral direction reduction, and longitudinal direction reduction.