FUNCTION EXTENSION DEVICE, FUNCTION EXTENSION METHOD, COMPUTER-READABLE RECORDING MEDIUM, AND INTEGRATED CIRCUIT

• US 20130156266A1
• Filed: 01/20/2012
• Published: 06/20/2013
• Est. Priority Date: 02/01/2011
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Horii, Yuki
• Granted Patent: US 8,965,049 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor

Abstract

An object recognition unit recognizes, from real-space video data, a body included in the video data. A function setting unit retains function information in which is prescribed a function configured from a pair of operation and processing that can be set for each type of body. In addition, the function setting unit sets, to each body recognized by the object recognition unit, a function that can be set, based on the type of each body. A selection determination unit determines a selected body selected by a user as the body to be operated among the respective bodies recognized by the object recognition unit 1101. An operation determination unit determines the operation that the user has performed on the selected body. A processing determination unit determines the processing for the operation that has been determined by the operation determination unit among the operations configuring the function set by the function setting unit.

Claims

1. A function extension device for incorporating a body in a real space into an extended real space, and setting a function to the body,

the function extension device comprising:

an object recognition unit which recognizes, from real-space video data acquired by a video input device, a body included in the video data, and recognizes a type of recognized body and a position of the recognized body in the real space;

a function setting unit which retains function information in which is prescribed a function configured from a pair of operation and processing that can be set for each type of body, refers to the function information when a body is recognized by the object recognition unit, and sets a function to the recognized body based on the type of recognized body;

an input recognition unit which recognizes an input operation performed by a user in the real space and an input position in the real space where the input operation has been performed;

a selection determination unit which determines a selected body selected by the user as a body to be operated based on the position of the body in the real space recognized by the object recognition unit and the input position in the real position recognized by the input recognition unit;

an operation determination unit which determines the operation of the user by using the input operation recognized by the input recognition unit;

a processing determination unit which determines the processing for the operation that has been determined by the operation determination unit among the operations configuring the function set by the function setting unit to the selected body selected by the selection determination unit; and

an execution unit which executes the processing determined by the processing determination unit.

2. The function extension device according to claim 1, wherein

the object recognition unit identifies, for each recognized body, the type of recognized body, and

the function setting unit sets a function to each body even when the bodies are of the same type.

3. The function extension device according to claim 2, wherein

the input recognition unit acquires a user identifier for identifying a user who has performed an input operation,

when a body is recognized by the object recognition unit, the function setting unit presents to a user candidates of operation and processing that can be set to the type of body, sets the operation and processing selected by the user as the function of the body, and associates the set function with the user identifier and retains the same, and

the processing determination unit determines the processing for the operation determined by the operation determination unit among the operations configuring the function set to the user about which the function setting unit holds the user identifier.

4. The function extension device according to claim 2, wherein, when a body is recognized by the object recognition unit, the function setting unit determines the operation and processing that can be set to the body among the operations and processing that can be set to the type of body, and sets the determined operation and processing as the function of the body.

5. The function extension device according to claim 4, wherein

the object recognition unit recognizes a shape and a size of a body, and

the function setting unit determines the operation and processing that can be set to the body based on at least one of the shape and the size of the body recognized by the object recognition unit.

6. The function extension device according to claim 4, wherein the function setting unit determines the operation and processing that can be set to the body recognized by the object recognition unit based on a capability of the function extension device and a capability of a peripheral device connected to the function extension device.

7. The function extension device according to claim 4,

wherein, when the function setting unit is connected to an external determination device via a predetermined communication network and a body is recognized by the object recognition unit, the function setting unit requests the external determination device to perform determination processing for determining the operation and processing that can be set to the body among the operations and processing that can be set to the type of the body, and sets the function to the body according to a determination result of the external determination device.

8. The function extension device according to claim 1, wherein the function setting unit acquires the function information from a server connected via a predetermined communication network.

9. The function extension device according to claim 1, wherein the execution unit includes a video output control unit which superimposes graphics on video data acquired by the video input device and displays the same on a video output device,

the function extension device further comprising:

a graphics generation unit which outputs, to the video output control unit, a drawing request of generating graphics to be superimposed on the selected body and superimposing the generated graphics on the selected body when the processing determined by the processing determination unit is the processing of superimposing graphics on the selected body.

10. The function extension device according to claim 9, wherein

the execution unit includes a tactile output control unit which controls a tactile output device which provides a tactile sense to a user, and

the tactile output control unit drives the tactile output device for providing tactile sense to the user, incidental to the superimposition of graphics on the selected body.

11. The function extension device according to claim 1, further comprising:

a 3D model management unit which assigns a body identifier to each body recognized by the object recognition unit, generates a 3D stereoscopic model of each body including the body identifier, the type of the body, and the position of the body in the real space, and notifies the function setting unit of the generated 3D stereoscopic model of each body as a object recognition notice, wherein

when the object recognition notice is notified from the 3D model management unit, the function setting unit determines that the body indicated by the notified object recognition notice has been recognized by the object recognition unit, and refers to the function information in order to set from the type of recognized body the function to the recognized body.

12. The function extension device according to claim 11, wherein

the 3D model management unit notifies the selection determination unit of body information which associates position data and a body identifier of each body recognized by the object recognition unit, and

the selection determination unit recognizes the position, in the real space, of each body recognized by the object recognition unit based on the body information notified by the 3D model management unit, and determines, as the selected body, a body positioned within a predetermined range relative to the input position recognized by the input recognition unit.

13. A function extension method of incorporating a body in a real space into an extended real space, and setting a function to the body,

the method comprising:

an object recognition step of a function extension device recognizing, from real-space video data acquired by a video input device, a body included in the video data, and recognizing a type of recognized body and a position of the recognized body in the real space;

a function setting step of a function extension device retaining function information in which is prescribed a function configured from a pair of operation and processing that can be set for each type of body, and referring to the function information when a body is recognized in the object recognition step, and then setting a function to the recognized body based on the type of recognized body;

an input recognition step of a function extension device recognizing an input operation that has been performed by a user in the real space and an input position in the real space where the input operation has been performed;

a selection determination step of a function extension device determining a selected body that has been selected by the user as the body to be operated, based on the position of the body in the real space recognized by the object recognition step and the input position in the real position recognized in the input recognition step;

an operation determination step of a function extension device determining the operation of the user by using the input operation recognized in the input recognition step;

a processing determination step of a function extension device determining the processing for the operation that has been determined in the operation determination step among the operations configuring the function set in the function setting unit to the selected body selected by the selection determination step; and

an execution step of a function extension device executing the processing determined in the processing determination step.

14. A computer-readable recording medium which stores a function extension program for causing a computer to function as a function extension device for incorporating a body in a real space into an extended real space, and setting a function to the body,

the program causing the computer to function as:

an object recognition unit which recognizes, from real-space video data acquired by a video input device, a body included in the video data, and recognizes a type of recognized body and a position of the recognized body in the real space;

a function setting unit which retains function information in which is prescribed a function configured from a pair of operation and processing that can be set for each type of body, refers to the function information when a body is recognized by the object recognition unit, and sets a function to the recognized body based on the type of recognized body;

an input recognition unit which recognizes an input operation performed by a user in the real space and an input position in the real space where the input operation has been performed;

a selection determination unit which determines a selected body selected by the user as the body to be operated, based on the position of the body in the real space recognized by the object recognition unit and the input position in the real position recognized by the input recognition unit;

an operation determination unit which determines the operation of the user by using the input operation recognized by the input recognition unit;

a processing determination unit which determines the processing for the operation determined by the operation determination unit among the operations configuring the function set by the function setting unit to the selected body selected by the selection determination unit; and

an execution unit which executes the processing determined by the processing determination unit.

15. An integrated circuit of a function extension device for incorporating a body in a real space into an extended real space, and setting a function to the body,

the integrated circuit comprising:

an object recognition unit which recognizes, from real-space video data acquired by a video input device, a body included in the video data, and recognizes a type of recognized body and a position of the recognized body in the real space;

a function setting unit which retains function information in which is prescribed a function configured from a pair of operation and processing that can be set for each type of body, refers to the function information when a body is recognized by the object recognition unit, and sets a function to the recognized body based on the type of recognized body;

an input recognition unit which recognizes an input operation performed by a user in the real space and an input position in the real space where the input operation has been performed;

a selection determination unit which determines a selected body selected by the user as the body to be operated, based on the position of the body in the real space recognized by the object recognition unit and the input position in the real position recognized by the input recognition unit;

an operation determination unit which determines the operation of the user by using the input operation recognized by the input recognition unit;

a processing determination unit which determines the processing for the operation determined by the operation determination unit among the operations configuring the function set by the function setting unit to the selected body selected by the selection determination unit; and

an execution unit which executes the processing determined by the processing determination unit.