Image processing method and image processing apparatus

• US 8,022,967 B2
• Filed: 06/01/2005
• Issued: 09/20/2011
• Est. Priority Date: 06/03/2004
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Okuno, Yasuhiro, Ohshima, Toshikazu, Tanimura, Kaname
• Primary Examiner(s): Wang; Andrew
• Assistant Examiner(s): Merouan; Abderrahim
• Application Number:
  US11/140,996
• Publication Number:
  US 20060044327A1
• Time in Patent Office: 2,302 Days
• Field of Search: 345/418, 345/619, 345/629, 345/632, 345/633
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

An image processing method includes the steps of acquiring an image of a physical space, acquiring a position and orientation of a viewpoint of the image, generating an image of a virtual object, detecting an area which consists of pixels each having a predetermined pixel value, and superimposing the image of the virtual object on the image of the physical space. The superimposition step includes calculating a distance between a position of the virtual object and a position of the viewpoint, acquiring an instruction indicating whether or not the virtual object is emphasis-displayed, and setting a flag indicating whether or not the image of the virtual object is to be set as a masked target. The masking process image of the virtual object is superimposed or not on the image of the physical space depending if the image of the virtual object is set as the masked target.

Claims

1. An image processing method comprising:

a physical image acquisition step of acquiring an image of a physical space;

a viewpoint position and orientation acquisition step of acquiring a position and orientation of a viewpoint of the image of the physical space;

a generation step of generating an image of a virtual object, based on the acquired position and orientation;

a detection step of detecting, as an object area, an area which consists of pixels each having a predetermined pixel value, from the image of the physical space; and

a superimposition step of superimposing the image of the virtual object on the image of the physical space,

wherein the superimposition step includes;

a calculation step of calculating a distance between a position of the virtual object and a position of the viewpoint;

an acquisition step of acquiring, from an operation panel operated by an operator, an operator instruction indicating whether or not the virtual object is to be emphasis-displayed;

a setting step of changing a flag to indicate that the image of the virtual object is to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is not to be emphasis-displayed, b) the calculated distance is not smaller than a predetermined distance, and changing the flag to indicate that the image of the virtual object is not to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is to be emphasis-displayed, b) the calculated distance is smaller than the predetermined distance;

a step of, while the flag is set to indicate that the image of the virtual object is to be set as the masked target, applying a masking process to a portion of the image of the virtual object which overlaps with the object area, and superimposing the masking processed image of the virtual object on the image of the physical space; and

a step of, while the flag is set to indicate that the image of the virtual object is not to be set as the masked target, without applying a masking process to the portion of the image of the virtual object, superimposing the image of the virtual object on the image of the physical space.

2. A computer-readable storage medium encoded with a computer program to implement an image processing method comprising:

a physical image acquisition step of acquiring an image of a physical space;

a viewpoint position and orientation acquisition step of acquiring a position and orientation of a viewpoint of the image of the physical space;

a generation step of generating an image of a virtual object, based on the acquired position and orientation;

a detection step of detecting, as an object area, an area which consists of pixels each having a predetermined pixel value, from the image of the physical space; and

a superimposition step of superimposing the image of the virtual object on the image of the physical space,

wherein the superimposition step includes;

a calculation step of calculating a distance between a position of the virtual object and a position of the viewpoint;

an acquisition step of acquiring, from an operation panel operated by an operator, an operator instruction indicating whether or not the virtual object is to be emphasis-displayed;

a setting step of changing a flag to indicate that the image of the virtual object is to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is not to be emphasis-displayed and, b) the calculated distance is not smaller than a predetermined distance, and changing the flag to indicate that the image of the virtual object is not to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is to be emphasis-displayed, b) the calculated distance is smaller than the predetermined distance;

a step of, while the flag is set to indicate that the image of the virtual object is to be set as the masked target, applying a masking process to a portion of the image of the virtual object which overlaps with the object area, and superimposing the masking processed image of the virtual object on the image of the physical space; and

a step of, while the flag is set to indicate that the image of the virtual object is not to be set as the masked target, without applying a masking process to the portion of the image of the virtual object, superimposing the image of the virtual object on the image of the physical space.

3. An image processing apparatus comprising:

a physical image acquisition unit adapted to acquire an image of a physical space;

a viewpoint position and orientation acquisition unit adapted to acquire a position and orientation of a viewpoint of the image of the physical space;

a generation unit adapted to generate an image of a virtual object, based on the acquired position and orientation;

a detection unit adapted to detect, as an object area, an area which consists of pixels each having a predetermined pixel value, from the image of the physical space; and

a superimposition unit adapted to superimpose the image of the virtual object on the image of the physical space,

wherein the superimposition unit includes;

a calculation unit adapted to calculate a distance between a position of the virtual object and a position of the viewpoint;

an acquisition unit adapted to acquire, from an operation panel operated by an operator, an operator instruction indicating whether or not the virtual object is to be emphasis-displayed;

a setting unit adapted to (1) a flag to indicate that the image of the virtual object is to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is not to be emphasis-displayed, b) the calculated distance is not smaller than a predetermined distance, and changing the flag to indicate that the image of the virtual object is not to be set as a masked target when both;

a) the operator instruction indicates that the virtual object is to be emphasis-displayed, b) the calculated distance is smaller than the predetermined distance, (2) while the flag is set to indicate that the image of the virtual object is to be set as the masked target, apply a masking process to a portion of the image of the virtual object which overlaps with the object area, and superimpose the masking processed image of the virtual object on the image of the physical space, and (3) while the flag is set to indicate that the image of the virtual object is not to be set as the masked target, without applying a masking process to the portion of the image of the virtual object, superimpose the image of the virtual object on the image of the physical space.