TRACKING DISPLAY SYSTEM, TRACKING DISPLAY PROGRAM, TRACKING DISPLAY METHOD, WEARABLE DEVICE USING THESE, TRACKING DISPLAY PROGRAM FOR WEARABLE DEVICE, AND MANIPULATION METHOD FOR WEARABLE DEVICE

• US 20160004320A1
• Filed: 06/24/2013
• Published: 01/07/2016
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Chikara Minami
• Original Assignee: Brilliantservice Co., Ltd.
• Inventors: Lundberg, Johannes, Kajii, Yusuke
• Granted Patent: US 9,651,782 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: G06V 40/11 : Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 : Recognition of static hand signs
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/332 : Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/395 : with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes

Abstract

Provided are a tracking display system, a tracking display program, a tracking display method, a wearable device using these, a tracking display program for the wearable device, and a manipulation method for the wearable device that enhance the manipulability of a program in a stereoscopic space. The tracking display system of the present invention includes: a display unit that displays a view object in a virtual space; a detection unit that detects an image of a target in a real space, the target being for selecting the view object; and a control unit that controls at least a position of the view object. Further, the control unit displays the view object such that the view object tracks a motion of the target, in a case where the control unit determines that the view object and the image of the target overlap with each other in the display unit.

Claims

1. A tracking display system comprising:

a display unit that displays a view object in a virtual space;

a detection unit that detects an image of a target in a real space, the target being for selecting the view object; and

a control unit that controls at least a position of the view object, wherein

the control unit displays the view object such that the view object tracks a motion of the target, in a case where the control unit determines that the view object and the image of the target overlap with each other in the display unit.

2. The tracking display system according to claim 1, wherein

the target is at least any of fingers, palms, hands, and arms of a human body of a manipulator, and

the detection unit is at least any of a depth level sensor and an image sensor.

3. The tracking display system according to claim 1, wherein the control unit determines whether or not the target overlaps with the view object continuously for a predetermined time.

4. The tracking display system according to claim 3, wherein the control unit displays passage of the predetermined time with respect to the view object.

5. The tracking display system according to claim 1, wherein, in a case where the target moves beyond a predetermined distance from a tracking start position of the view object, the control unit ends the tracking display, and displays the view object such that the view object returns to the tracking start position.

6. The tracking display system according to claim 1, wherein the view object is at least any of an icon, a menu item, a text link, and a determination button.

8. A wearable device comprising the tracking display system according to claim 1.

7. A tracking display system comprising:

a view object displayed in a virtual space;

a target in a real space, for selecting the view object by a manipulator;

a detection unit that detects a position of the target in the real space; and

a control unit that controls at least a position of the view object in the virtual space, wherein

the control unit displays the view object such that the view object tracks a motion of the target, in a case where the control unit determines that the target exists in a predetermined space defined by central projection from a point of view of the manipulator to the view object.

9. A tracking display program comprising:

a display process of displaying a view object in a virtual space;

a detection process of detecting an image of a target in a real space, the target being for selecting the view object; and

a control process of controlling at least a position of the view object, wherein

the control process includes displaying the view object such that the view object tracks a motion of the target, in a case where it is determined that the view object and the image of the target overlap with each other.

11. A tracking display program for a wearable device, comprising the tracking display program according to claim 9.

10. A tracking display program comprising:

a display process of displaying a view object in a virtual space;

a selection process of selecting the view object by a manipulator using a target in a real space;

a detection process of detecting a position in the real space selected in the selection process; and

a control process of controlling at least a position of the view object in the virtual space, wherein

the control process includes displaying the view object such that the view object tracks a motion of the target, in a case where it is determined that the target exists in a predetermined space defined by central projection from a point of view of the manipulator to the view object.

12. A tracking display method comprising:

a display step of displaying a view object in a virtual space;

a detection step of detecting an image of a target in a real space, the target being for selecting the view object; and

a control step of controlling at least a position of the view object, wherein

the control step includes displaying the view object such that the view object tracks a motion of the target, in a case where it is determined that the view object and the image of the target overlap with each other.

14. A manipulation method for a wearable device, comprising the tracking display method according to claim 12.

13. A tracking display method comprising:

a display step of displaying a view object in a virtual space;

a selection step of selecting the view object by a manipulator using a target in a real space;

a detection step of detecting a position in the real space selected in the selection step; and

a control step of controlling at least a position of the view object in the virtual space, wherein

the control step includes displaying the view object such that the view object tracks a motion of the target, in a case where it is determined that the target exists in a predetermined space defined by central projection from a point of view of the manipulator to the view object.