INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD AND PROGRAM

• US 20120092333A1
• Filed: 03/17/2010
• Published: 04/19/2012
• Est. Priority Date: 04/28/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Matsuda, Kouichi
• Granted Patent: US 9,772,683 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A configuration is provided in which data that is not displayed on a display is always observable as an virtual object. For example, in data processing using a PC, data that is not displayed on the display of the PC, such as data having been cut or copied, can be set as a virtual object and pasted on a finger of a hand of a user or the like to be always observable. According to this configuration, data that is not displayed on the display area of the PC can be pasted on a space other than the display area of the PC and displayed to be observable, which allows improving of the efficiency of data processing.

Claims

1-9. -9. (canceled)

10. An information processing apparatus comprising:

a memory for storing processed data generated in data processing using a first display;

a 3D information analyzer for analyzing the 3D position of a real object included in a camera-shot image; and

a virtual object manager for receiving analysis information input from the 3D information analyzer, determining a constituent part of the real object to be a virtual object display position and displaying on a second display a combined image in which the real object and the virtual object are combined,

wherein the virtual object manager

receives analysis information input from the 3D information analyzer, determines a constituent part of the real object included in the image displayed on the second display to be a virtual object display position and registers the determination information in a virtual object information management table, and

in response to a processing request input regarding the virtual object displayed on the second display, outputs data corresponding to the virtual object subject to the processing request to a data processor that performs data processing using the first display or a memory accessible to the data processor.

11. The information processing apparatus according to claim 10, further comprising a communication section and

performing communication with the data processor that performs data processing using the first display to transmit or receive data for data processing using the first display.

12. The information processing apparatus according to claim 10, wherein the 3D information analyzer includes a hand recognition module that recognizes each finger of a hand of a user included in the camera-shot image and analyzes the 3D position of the each finger of the hand of the user recognized by the hand recognition module, and

wherein the virtual object manager

determines the position of the each finger of the hand of the user to be the virtual object display position, registers the position of the each finger of the hand of the user in the virtual object information management table and displays on the second display a combined image in which the virtual object is combined with the position of the each finger of the hand of the user.

13. The information processing apparatus according to claim 10, wherein the data stored in the memory is data having been cut or copied in the data processing using the first display.

14. The information processing apparatus according to claim 10, wherein the virtual object manager

registers state information on whether or not a virtual object is displayed at the display position on the second display corresponding to the virtual object display position registered in the virtual object information management table.

15. The information processing apparatus according to claim 10, wherein the virtual object manager

refers to the state information in the virtual object information management table to determine whether a new virtual object can be displayed or not and deletes the virtual object being displayed on the second display if requested by a user input.

16. An information processing method performed in an information processing apparatus, the method comprising:

a step in which a controller stores processed data to be applied to data processing using a first display, in a memory;

a 3D information analysis step in which a 3D information analyzer analyzes the 3D position of a real object included in a camera-shot image;

a step in which a virtual object manager receives analysis information input from the 3D information analyzer, determines a constituent part of the real object to be a virtual object display position and displays on a second display a combined image in which the real object and the virtual object are combined;

a step in which the virtual object manager receives analysis information input from the 3D information analyzer, determines a constituent part of the real object included in the image displayed on the second display to be a virtual object display position and registers the determination information in a virtual object information management table; and

a step in which, in response to a processing request input regarding the virtual object displayed on the second display, the virtual object manager outputs data corresponding to the virtual object subject to the processing request to a data processor that performs data processing using the first display or a memory accessible to the data processor.

17. A program for causing an information processing apparatus to perform information processing, the program comprising:

a step of causing a controller to store processed data to be applied to data processing using a first display, in a memory;

a 3D information analysis step of causing a 3D information analyzer to analyze the 3D position of a real object included in a camera-shot image;

a step of causing a virtual object manager to receive analysis information input from the 3D information analyzer, to determine a constituent part of the real object to be a virtual object display position and to display on a second display a combined image in which the real object and the virtual object are combined;

a step of causing the virtual object manager to receive analysis information input from the 3D information analyzer, to determine a constituent part of the real object included in the image displayed on the second display to be a virtual object display position and to register the determination information in a virtual object information management table; and

a step of causing, in response to a processing request input regarding the virtual object displayed on the second display, the virtual object manager to output data corresponding to the virtual object subject to the processing request to a data processor that performs data processing using the first display or a memory accessible to the data processor.