INFORMATION PROCESSOR, PROCESSING METHOD AND PROGRAM

• US 20120124509A1
• Filed: 06/30/2010
• Published: 05/17/2012
• Est. Priority Date: 07/21/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Matsuda, Kouichi, Fukuchi, Masaki
• Granted Patent: US 8,751,969 B2
• Time in Patent Office: Days
• US Class Current: 715/782
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts

Abstract

The present invention provides a configuration that allows for a cursor or other object that has moved outside the display section to be displayed as a virtual object and observed. A cursor or object lying in an area outside the area of the display section of a PC or other device is displayed as a virtual object. For example, the display of goggles worn by the user displays a display device such as a PC and the area outside the display device. The three-dimensional position of the cursor or object that has probably moved in response to user operation is calculated, after which the cursor or object is displayed as a virtual object at the calculated position. Further, object information for the object specified by the cursor is acquired and presented.

Claims

1. An information processor comprising:

a coordinate processing module adapted to determine whether the position of a cursor, i.e., a position indicator displayed on a first display section, is located in or outside the area of the first display section and output cursor position information to a virtual object management section if the cursor is located outside the area of the first display section;

a camera adapted to capture an image made up of a real object including the first display section;

a three-dimensional information analysis section adapted to analyze the three-dimensional position of the real object included in a camera-captured image;

a second display section adapted to display the camera-captured image; and

a virtual object management section adapted to generate a virtual object different from the real object included in the camera-captured image and generate a composite image including the generated virtual object and the real object so as to display the composite image on the second display section, wherein

the virtual object management section calculates the three-dimensional position of the cursor based on the cursor position information supplied from the coordinate processing module so as to display, on the second display section, a composite image in which the object is placed at the calculated position as a virtual object.

2. The information processor of claim 1 comprising:

an application execution section adapted to process a specified object specified by the position indicator, wherein

the application execution section determines whether the specified object is located in or outside the area of the first display section and outputs object position information to the virtual object management section if the specified object is located outside the area of the first display section, and wherein

the virtual object management section calculates the three-dimensional position of the object based on the object position information supplied from the coordinate processing module so as to display, on the second display section, a composite image in which the object is placed at the calculated position as a virtual object.

3. The information processor of claim 2, wherein

if the three-dimensional position of the object calculated based on the object position information supplied from the coordinate processing module includes the display area of the first display section, the virtual object management section displays, on the second display section, a composite image with an object area image overlapping the display area of the first display section deleted.

4. The information processor of claim 1 further comprising:

an object information acquisition section adapted to acquire image data of a real object specified by the cursor placed as the virtual object and search data based on the acquired image data so as to acquire object information, wherein

the object information acquisition section outputs the acquired object information to the first display section as display data.

5. The information processor of claim 4, wherein

the object information acquisition section accesses a database in which real object image data and object information are associated with each other or a server so as to acquire object information through a search based on the real object image data.

6. The information processor of claim 1, wherein

the virtual object management section calculates a plane including the display surface of the first display section based on three-dimensional position information of components making up the first display section included in the camera-captured image and calculates the three-dimensional position of the cursor so that the cursor position is placed on the plane.

7. The information processor of claim 1, wherein

the cursor is a mouse cursor that moves by mouse operation, and

the coordinate processing module receives mouse cursor displacement information resulting from the mouse operation and determines whether the mouse cursor is located in or outside the area of the first display section.

8. An information processing method performed by an information processor, the information processing method comprising:

a coordinate processing step of a coordinate processing module determining whether the position of a cursor, i.e., a position indicator displayed on a first display section, is located in or outside the area of the first display section and outputting cursor position information to a virtual object management section if the cursor is located outside the area of the first display section;

an image capture step of a camera capturing an image of a real object including the first display section;

a three-dimensional information analysis step of a three-dimensional information analysis section analyzing the three-dimensional position of the real object included in a camera-captured image; and

a virtual object management step of a virtual object management section generating a virtual object different from the real object included in the camera-captured image and generating a composite image including the generated virtual object and the real object so as to display the composite image on the second display section, wherein

the virtual object management step is a step of calculating the three-dimensional position of a cursor based on cursor position information supplied from the coordinate processing module so as to display, on the second display section, a composite image in which the cursor is placed at the calculated position as a virtual object.

9. A program causing an information processor to process information, the program comprising:

a coordinate processing step of causing a coordinate processing module to determine whether the position of a cursor, i.e., a position indicator displayed on a first display section, is located in or outside the area of the first display section and output cursor position information to a virtual object management section if the cursor is located outside the area of the first display section;

an image capture step of causing a camera to capture an image of a real object including the first display section;

a three-dimensional information analysis step of causing a three-dimensional information analysis section to analyze the three-dimensional position of the real object included in a camera-captured image; and

a virtual object management step of causing a virtual object management section to generate a virtual object different from the real object included in the camera-captured image and generate a composite image including the generated virtual object and the real object so as to display the composite image on the second display section, wherein

the virtual object management step is a step of causing the virtual object management section to calculate the three-dimensional position of a cursor based on cursor position information supplied from the coordinate processing module so as to display, on the second display section, a composite image in which the cursor is placed at the calculated position as a virtual object.