IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, PROGRAM, AND IMAGE PROCESSING SYSTEM

• US 20110205243A1
• Filed: 02/08/2011
• Published: 08/25/2011
• Est. Priority Date: 02/24/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: MATSUDA, Kouichi
• Granted Patent: US 8,773,466 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

There is provided an image processing apparatus including: an input image acquisition unit for obtaining an input image generated by taking an image of a real space; an image recognition unit for recognizing, when a first user-input representing a start of manipulation is detected, a manipulator used for manipulating a virtual object, wherein the manipulator appears in the input image; a calculation unit for calculating, according to a result of the recognition of the manipulator provided by the image recognition unit, a position on a screen of a display device at which the virtual object is to be displayed; a display control unit for displaying the virtual object at the position of the screen of the display device calculated by the calculation unit; and a communication unit for transmitting, when the first user-input is detected, a first notification signal for notifying the start of manipulation to another apparatus displaying the same virtual object.

Claims

1. An image processing apparatus comprising:

an input image acquisition unit for obtaining an input image generated by taking an image of a real space;

an image recognition unit for recognizing, when a first user-input representing a start of manipulation is detected, a manipulator used for manipulating a virtual object, wherein the manipulator appears in the input image;

a calculation unit for calculating, according to a result of the recognition of the manipulator provided by the image recognition unit, a position on a screen of a display device at which the virtual object is to be displayed;

a display control unit for displaying the virtual object at the position of the screen of the display device calculated by the calculation unit; and

a communication unit for transmitting, when the first user-input is detected, a first notification signal for notifying the start of manipulation to another apparatus displaying the same virtual object.

2. The image processing apparatus according to claim 1,

wherein the first notification signal includes identification data for identifying the virtual object or position data representing a position of the virtual object.

3. The image processing apparatus according to claim 1,

wherein, when a second user-input representing an end of manipulation is detected, the communication unit transmits a second notification signal to the another apparatus so as to notify the end of manipulation.

4. The image processing apparatus according to claim 3,

wherein the second notification signal includes position data representing a position of the virtual object calculated by the calculation unit.

5. The image processing apparatus according to claim 1,

wherein the image recognition unit also recognizes the manipulator appearing in the input image when the communication unit receives the first notification signal from another apparatus.

6. The image processing apparatus according to claim 5,

wherein the image recognition unit does not recognize the manipulator when the first notification signal received by the communication unit is not related to virtual objects which are to be displayed by the image processing apparatus.

7. The image processing apparatus according to claim 1,

wherein the communication unit transmits the first notification signal to the another apparatus, when the first user-input is detected and thereafter the manipulator is determined to be in contact with the virtual object as a result of the recognition of the manipulator provided by the image recognition unit.

8. The image processing apparatus according to claim 1,

wherein the communication unit transmits the first notification signal to an information processing apparatus that manages a group of apparatuses displaying the same virtual object.

9. The image processing apparatus according to claim 8,

wherein the first notification signal includes identification data for identifying the virtual object or position data representing a position of the virtual object, and the first notification signal is transferred via the information processing apparatus to another apparatus displaying the same virtual object.

10. The image processing apparatus according to claim 1, further comprising:

a storage unit storing a manipulator image representing the manipulator,

wherein the image recognition unit uses the manipulator image stored in the storage unit to recognize the manipulator appearing in the input image.

11. An image processing method performed by an image processing apparatus, comprising the steps of:

obtaining an input image generated by taking an image of a real space;

detecting a first user-input representing a start of manipulation;

transmitting a first notification signal to another apparatus displaying the same virtual object as a virtual object which is to be manipulated by the image processing apparatus so as to notify a start of manipulation;

recognizing a manipulator, appearing in the input image, that is used for manipulation of the virtual object;

calculating, according to a result of the recognition of the manipulator, a position on a screen of a display device at which the virtual object is to be displayed; and

displaying the virtual object at the calculated position of the screen of the display device.

12. A program for causing a computer controlling an image processing apparatus to function as:

an input image acquisition unit for obtaining an input image generated by taking an image of a real space;

an image recognition unit for recognizing, when a first user-input representing a start of manipulation is detected, a manipulator used for manipulating a virtual object, wherein the manipulator appears in the input image;

a calculation unit for calculating, according to a result of the recognition of the manipulator provided by the image recognition unit, a position on a screen of a display device at which the virtual object is to be displayed;

a display control unit for displaying the virtual object at the position of the screen of the display device calculated by the calculation unit; and

a communication unit for transmitting, when the first user-input is detected, a first notification signal for notifying the start of manipulation to another apparatus displaying the same virtual object.

13. An image processing system including two or more image processing apparatuses, wherein each of the image processing apparatuses comprises:

an input image acquisition unit for obtaining an input image generated by taking an image of a real space;

an image recognition unit for recognizing, when a first user-input representing a start of manipulation is detected, a manipulator used for manipulating a virtual object, wherein the manipulator appears in the input image;

a calculation unit for calculating, according to a result of the recognition of the manipulator provided by the image recognition unit, a position on a screen of a display device at which the virtual object is to be displayed;

a display control unit for displaying the virtual object at the position of the screen of the display device calculated by the calculation unit; and

a communication unit for transmitting, when the first user-input is detected, a first notification signal for notifying the start of manipulation to another apparatus displaying the same virtual object,

wherein the image recognition unit also recognizes the manipulator appearing in the input image, when the communication unit receives the first notification signal from another apparatus.