Information processing apparatus and information processing method

• US 10,948,977 B2
• Filed: 09/10/2019
• Issued: 03/16/2021
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ishihara, Atsushi, Ishikawa, Tsuyoshi, Aga, Hiroyuki, Kawasaki, Koichi, Nishibe, Mitsuru, Kusano, Yuji
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US16/566,477
• Publication Number:
  US 20200004323A1
• Time in Patent Office: 553 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

An information processing apparatus including circuitry configured to acquire information indicating a spatial relationship between a real object and a virtual object, and initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world, and wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

Claims

1. A display control apparatus, comprising:

a sensor configured to acquire positional information of a hand of a user of a head-mounted display; and

a processor configured to;

set a virtual light source corresponding to a gaze direction of the user;

control the head-mounted display to display a virtual object over a view of a real world; and

control, based on the positional information and the virtual light source, the head-mounted display to display a virtual shadow of the hand to the virtual object.

2. The display control apparatus according to claim 1,

wherein the processor is configured to set the virtual light source to radiate a virtual light forward from a head of the user.

3. The display control apparatus according to claim 1,

wherein the processor is configured to control, based on the positional information, the head-mounted display to decrease a size of the virtual shadow as a distance between the hand and the virtual object decreases.

4. The display control apparatus according to claim 1,

wherein the processor is configured to

set an effective range of visual feedback of the virtual shadow in a depth direction, excluding a range around a face of the user;

and

control the head-mounted display not to display the virtual shadow when the hand is not within the effective range.

5. The display control apparatus according to claim 1,

wherein the head-mounted display is an optical see-through head-mounted display configured to stereoscopically display, using optical units for left and right eyes of the user, the virtual object and the virtual shadow by adjusting a convergence of the user.

6. The display control apparatus according to claim 1,

wherein the head-mounted display is a smartphone,

wherein the sensor is an imaging device that is installed in the smartphone and configured to capture the real world, and

wherein the processor is further configured to control the smartphone to display the view of the real world, a first virtual object, and a second virtual object simultaneously.

7. A display control method, comprising:

acquiring, from a sensor of a head-mounted display, positional information of a hand of a user of the head-mounted display;

setting a virtual light source corresponding to a gaze direction of the user;

controlling the head-mounted display to display a virtual object over a view of a real world; and

controlling, based on the positional information and the virtual light source, the head-mounted display to display a virtual shadow of the hand to the virtual object.

8. A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a display control apparatus for a head-mounted display cause the display control apparatus to execute operations, the operations comprising:

acquiring, from a sensor of a head-mounted display, positional information of a hand of a user of the head-mounted display;

setting a virtual light source corresponding to a gaze direction of the user;

controlling the head-mounted display to display a virtual object over a view of a real world; and

controlling, based on the positional information and the virtual light source, the head-mounted display to display a virtual shadow of the hand to the virtual object.