INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM

• US 20120032955A1
• Filed: 03/17/2010
• Published: 02/09/2012
• Est. Priority Date: 04/23/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Matsuda, Kouichi
• Granted Patent: US 8,994,721 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

A configuration is realized which enables data not displayed on a display section to be always observed as a virtual object. A configuration is realized with which, in various kinds of display apparatus such as a PC and a portable telephone having a display section, non-displayed data that extends off the area of the display section is displayed as a virtual object at a position contiguous to the display data of the display section. This configuration allows the user to always observe and view not only data displayed on a limited display area of a PC or the like but also data that has gone off the display section, thereby enhancing data processing efficiency.

Claims

1. An information processing apparatus comprising:

an application executing section that stores non-displayed data and positional information of the non-displayed data into a memory in data processing using a first display section, the non-displayed data being contiguous to display data of the first display section;

a three-dimensional information analysis section that analyzes a three-dimensional position of a real object including the first display section included in a camera-captured image; and

a virtual object management section that takes input of the non-displayed data and the positional information of the non-displayed data which are stored in the memory, and three-dimensional positional information of the first display section analyzed by the three-dimensional information analysis section, generates a composite image formed by the real object included in the camera-captured image and a virtual object formed by the non-displayed data, and displays the composite image on a second display section,

wherein the virtual object management section sets a display position of the virtual object on the second display section, at a position located in a direction of extension from a boundary of the display data of the first display section displayed on the second display section to an outside of the first display section.

2. The information processing apparatus according to claim 1, wherein the virtual object management section transforms coordinate data, which is the positional information of the non-displayed data stored in the memory, into a camera coordinate system serving as an indicator of a display position in a display image of the second display section, and determines the display position of the virtual object by applying the transformed coordinate data.

3. The information processing apparatus according to claim 2, wherein the object management section judges whether or not an intersection occurs between the display position of the virtual object transformed into the camera coordinate system, and a plane forming the real object included in the camera-captured image, and if the intersection occurs, executes a coordinate re-transformation process so as to position the transformed coordinate data in the plane forming the real object, and determines the display position of the virtual object by applying the re-transformed coordinate data.

4. The information processing apparatus according to any one of claims 1 to 3, wherein:

the information processing apparatus includes

a first apparatus having the application executing section, and

a second apparatus having the three-dimensional information analysis section, and the virtual object management section; and

data stored into a first memory in the first apparatus by the application executing section is transferred to the second apparatus via a communication section, and in the second apparatus, the three-dimensional information analysis section and the virtual object management section perform processing using received data.

5. An information processing method which is executed in an information processing apparatus, comprising:

an application executing step of an application executing section storing non-displayed data and positional information of the non-displayed data into a memory in data processing using a first display section, the non-displayed data being contiguous to display data of the first display section;

a three-dimensional information analysis step of a three-dimensional information analysis section analyzing a three-dimensional position of a real object including the first display section included in a camera-captured image; and

a virtual object management step of a virtual object management section taking input of the non-displayed data and the positional information of the non-displayed data which are stored in the memory, and three-dimensional positional information of the first display section analyzed by the three-dimensional information analysis section, generating a composite image formed by the real object included in the camera-captured image and a virtual object formed by the non-displayed data, and displaying the composite image on a second display section,

wherein the virtual object management step has a step of setting a display position of the virtual object on the second display section, at a position located in a direction of extension from a boundary of the display data of the first display section displayed on the second display section to an outside of the first display section.

6. A program for causing information processing to be executed in an information processing apparatus, comprising:

an application executing step of causing an application executing section to store non-displayed data and positional information of the non-displayed data into a memory in data processing using a first display section, the non-displayed data being contiguous to display data of the first display section;

a three-dimensional information analysis step of causing a three-dimensional information analysis section to analyze a three-dimensional position of a real object including the first display section included in a camera-captured image; and

a virtual object management step of causing a virtual object management section to take input of the non-displayed data and the positional information of the non-displayed data which are stored in the memory, and three-dimensional positional information of the first display section analyzed by the three-dimensional information analysis section, generate a composite image formed by the real object included in the camera-captured image and a virtual object formed by the non-displayed data, and display the composite image on a second display section,

wherein the virtual object management step has a step of causing a display position of the virtual object on the second display section to be set, at a position located in a direction of extension from a boundary of the display data of the first display section displayed on the second display section to an outside of the first display section.