Image processing device, image processing method, and program

• US 9,268,410 B2
• Filed: 11/29/2012
• Issued: 02/23/2016
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kasahara, Shunichi
• Primary Examiner(s): CERULLO, LILIANA P
• Application Number:
  US14/358,459
• Publication Number:
  US 20140320404A1
• Time in Patent Office: 1,181 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts

Abstract

Provided is an image processing device including a recognition unit configured to recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image, a calculation unit configured to calculate an inverse matrix of the environment recognition matrix, and a manipulation control unit configured to control manipulation of a virtual object arranged in the environment in a three-dimensional manipulation amount according to a difference between a first position or a first attitude based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

Claims

1. An image processing device comprising:

circuitry configured to;

recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;

calculate an inverse matrix of the environment recognition matrix; and

control manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

2. The image processing device according to claim 1, wherein the manipulation amount is a movement amount according to the difference between the first position and the second position.

3. The image processing device according to claim 1, wherein the manipulation amount is a rotation amount according to the difference between the first attitude and the second attitude.

4. The image processing device according to claim 1,

wherein the first time point is a time point at which a first user input is detected in the terminal, and

wherein the second time point is a time point at which a second user input is detected in the terminal.

5. The image processing device according to claim 4, wherein the first user input and the second user input correspond to a start and an end of a series of manipulations, respectively.

6. The image processing device according to claim 4, wherein the first position or the second position is a position offset in a direction on a screen of the terminal according to a user input position on the screen.

7. The image processing device according to claim 4, wherein the circuitry is configured to specify the virtual object to be manipulated based on the first user input.

8. The image processing device according to claim 7, wherein the circuitry is configured to specify a virtual object associated with a real object in the environment specified through the first user input as the virtual object to be manipulated.

9. The image processing device according to claim 1, wherein at least one of functions performed by the circuitry is realized by a device present in a cloud computing environment instead of the image processing device.

10. An image processing method comprising:

recognizing an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;

calculating an inverse matrix of the environment recognition matrix; and

controlling, using circuitry, manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

11. A non-transitory computer readable medium including executable instructions, which when executed by a computer cause the computer to:

recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;

calculate an inverse matrix of the environment recognition matrix; and

control manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.