AUGMENTED REALITY OPERATION SYSTEM, AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM STORING AUGMENTED REALITY OPERATION PROGRAM

• US 20160316081A1
• Filed: 04/22/2016
• Published: 10/27/2016
• Est. Priority Date: 04/25/2015
• Status: Active Grant

• Current Assignee: KYOCERA Document Solutions, Inc. (Kyocera Corporation)
• Original Assignee: KYOCERA Document Solutions, Inc. (Kyocera Corporation)
• Inventors: SHIMAMOTO, Kunihiko, MATSUKI, Yoshitaka, KAWASAKI, Tomohiro
• Granted Patent: US 9,628,646 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/12 Digital output to print unit , e.g. line printer, chain printer
• CPC Class Codes: G06F 3/1205 resulting in increased flexibility in print job configuration, e.g. job settings, print requirements, job tickets
• CPC Class Codes: G06F 3/1253 Configuration of print job parameters, e.g. using UI at the client
• CPC Class Codes: H04N 1/00413 using menus, i.e. presenting the user with a plurality of selectable options H04N1/00464 takes precedence
• CPC Class Codes: H04N 1/00493 Particular location of the interface or console

Abstract

An augmented reality operation system includes: an image-taking device, a display device, an MFP, an augmented reality processing section displaying, on the display device, a synthetic image obtained by synthesizing an operation image illustrating an operation section for operating the MFP with a taken image taken by the image-taking device, and an operation reception section receiving operation on the operation section in the operation image. The operation reception section, based on a position (on the synthetic image) of a hand taken by the image-taking device and a position of the operation section on the synthetic image, receives the operation on the operation section.

Claims

1. An augmented reality operation system comprising:

an image-taking device;

a display device;

an electronic device;

an augmented reality processing section displaying, on the display device, a synthetic image obtained by synthesizing an operation image illustrating an operation section for operating the electronic device in a manner such that the operation image is viewed to be located at a predefined position in a virtual space in a taken image taken by the image-taking device; and

an operation reception section receiving operation on the operation section illustrated by the operation image,

wherein the operation reception section, based on a position (on the synthetic image) of an image illustrating a hand included in the taken image taken by the image-taking device and a position of the operation section illustrated by the operation image on the synthetic image, receives the operation for the electronic device performed on the operation section illustrated by the operation image.

2. The augmented reality operation system according to claim 1,

wherein, upon occurrence of a predefined change in a position displayed on the display device as a result of overlapping of the image illustrating the hand with the operation image, the operation reception section receives the operation corresponding to a position of the image illustrating the hand in the operation image.

3. The augmented reality operation system according to claim 1, further comprising:

an audio output device; and

a stereophonic section making the audio output device stereophonically output operation sound for the operation section in a manner such that the operation sound is outputted at a position in a real space corresponding to a position of the operation section illustrated by the operation image on the synthetic image.

4. The augmented reality operation system according to claim 3,

wherein, instead of the image illustrating the hand in the synthetic image, the augmented reality processing section generates the synthetic image in which a prepared image of a virtual hand is arranged, at the position of the image illustrating the hand.

5. The augmented reality operation system according to claim 3,

wherein, together with the image illustrating the hand in the synthetic image, the augmented reality processing section generates the synthetic image in which a prepared image of a virtual hand is arranged, at the position of the image illustrating the hand.

6. The augmented reality operation system according to claim 3,

wherein the augmented reality processing section uses at least part of the image illustrating the hand as the image of the virtual hand.

7. A non-transitory computer-readable recording medium storing an augmented reality operation program, the augmented reality operation program causing a computer to function as:

an augmented reality processing section displaying, on a display device, a synthetic image obtained by synthesizing an operation image illustrating an operation section for operating an electronic device with a taken image taken by an image-taking device; and

an operation reception section receiving operation on the operation section illustrated by the operation image,

wherein the augmented reality operation program further causes the computer to function in a manner such that, based on a position on the synthetic image of an image illustrating a hand included in the taken image taken by the image-taking device and on a position of the operation section illustrated by the operation image on the synthetic image, the operation reception section receives the operation for the electronic device performed on the operation section illustrated by the operation image.