×

Augmented reality operation system and augmented reality operation method

  • US 9,628,646 B2
  • Filed: 04/22/2016
  • Issued: 04/18/2017
  • Est. Priority Date: 04/25/2015
  • Status: Active Grant
First Claim
Patent Images

1. An augmented reality operation system comprising:

  • an image-taking device;

    a display device;

    a multifunction peripheral (MFP); and

    an augmented reality (AR) server including a storage device having an AR operation program stored thereon, a processor, and an audio output device,wherein the processor executes the AR operation program to thereby;

    generate a synthetic image and start to display the synthetic image on the display device, the synthetic image being obtained by synthesizing an operation image illustrating an operation section for operating the MFP with a taken image taken by the image-taking device, the operation image on the synthetic image being synthesized with the taken image in a manner such that the operation image appears, for a user viewing a display screen of the display device, to be arranged in a space between the MFP and the display device;

    judge, after the start of the display of the synthetic image, whether or not an image illustrating the user'"'"'s hand is present on the taken image taken by the image-taking device;

    upon judgment that the image illustrating the user'"'"'s hand is present on the taken image, (i) delete the image illustrating the user'"'"'s hand included in the taken image from the synthetic image through image processing, (ii) by processing the image illustrating the user'"'"'s hand through image processing to clarify contrast with respect to surrounding images, generate an image of a virtual hand in which at least part of the image illustrating the hand is used, and (iii) further generate, instead of the image illustrating the hand on the synthetic image, the synthetic image in which the generated virtual hand is arranged at the position of the image illustrating the hand, and display the synthetic image on the display device, andwhen a position of a fingertip of the image of the virtual hand overlaps with a position of a setting reception image in the operation image included in the synthetic image and also when a size of an image illustrating the fingertip decreases while the position of the fingertip does not change, judge that the setting reception image has been pressed, and make the audio output device stereophonically output operation sound for the operation section in a manner such that the operation sound is outputted at a position in a real space corresponding to a position of the operation section illustrated by the setting reception image on the synthetic image to thereby receive the operation for the MFP performed on the operation section illustrated by the operation image.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×