Three-dimensional space interface apparatus and method

• US 20110096072A1
• Filed: 10/14/2010
• Published: 04/28/2011
• Est. Priority Date: 10/27/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jung Bae, Bang, Won-Chul, Kim, Chang Yeong
• Granted Patent: US 9,377,858 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts

Abstract

A three-dimensional (3D) space interface apparatus and method are provided. The 3D space interface apparatus provides an interface in which a user may directly touch and transform a 3D virtual object of a 3D space, which is shown in various ways depending on a position of the user. The 3D space interface apparatus may provide for manipulation of the 3D virtual object by the user using a sensing signal that senses the position and movement of a manipulation object the user. Specifically, an interface unit of the apparatus verifies, using the sensing signal, whether a predetermined movement of the manipulation object has occurred or whether a contact between the 3D virtual object and the manipulation object has been made, and generates a predetermined reaction corresponding to the predetermined movement or the contact.

Claims

1. A three-dimensional (3D) space interface apparatus comprising:

a 3D display unit to output a 3D virtual object;

a sensing unit to generate a sensing signal for sensing a position and a movement of a user and for sensing a position and a movement of a manipulation object; and

an interface unit to verify, using the sensing signal, whether a predetermined movement of the manipulation object has occurred or whether a contact between the 3D virtual object and the manipulation object has been made, and to generate a predetermined reaction corresponding to the predetermined movement or the contact.

2. The 3D space interface apparatus of claim 1, wherein the sensing unit comprises at least one of a color camera, a depth camera, and a motion sensor.

3. The 3D space interface apparatus of claim 1, wherein the interface unit comprises:

a space determination unit to determine a rendering space in which the 3D virtual object is output and a sensing space in which a manipulation of the 3D virtual object is sensed, using the sensing signal;

a 3D rending unit to 3D-render the 3D virtual object in accordance with the position of the user to output the rendered 3D virtual object in the rendering space;

a manipulation object-detection unit to detect the manipulation object positioned in the sensing space using the sensing signal and to trace a movement of the manipulation object;

a contact sensing unit to sense whether contact between the manipulation object and the 3D virtual object has been made;

a motion recognition unit to trace the movement of the manipulation object to verify whether the predetermined movement has occurred; and

a reaction generation unit to generate the predetermined reaction corresponding to one or more of the predetermined movement and the contact.

4. The 3D space interface apparatus of claim 3, wherein the space determination unit traces the position of the user and a face of the user using the sensing signal, determines, as the rendering space, an entire space between a 3D display unit and the face of the user, and determines, as the sensing space, a predetermined sized space between the 3D display unit and the face of the user.

5. The 3D space interface apparatus of claim 3, wherein the manipulation object-detection unit detects, as the manipulation object, any one or more of hands of the user within the sensing space, a body of the user within the sensing space, and an object within the sensing space.

6. The 3D space interface apparatus of claim 3, wherein the contact sensing unit corrects a distortion of the rendering and sensing spaces, each space varying depending on the position of the user, aggregates the corrected rendering and sensing spaces, and verifies whether contact has been made between the 3D virtual object output in the corrected rendering space and the manipulation object detected in the corrected sensing space.

7. The 3D space interface apparatus of claim 3, wherein the predetermined reaction corresponding to the predetermined movement and the contact is at least one of a transformation expanding or compressing the 3D virtual object, a rotation of the 3D virtual object, a movement of the 3D virtual object, a selection of a portion of the 3D virtual object where the contact has been made, a performance of a predetermined command corresponding to the predetermined movement or the contact, a creation of a new 3D virtual object, and a removal of the 3D virtual object.

8. The 3D space interface apparatus of claim 3, wherein the reaction generation unit further includes an ultrasonic wave-generation unit to generate a contact signal enabling a contact sensation by sending ultrasonic waves to the manipulation object when contact has been made with the 3D virtual object.

9. A 3D space interface method for interfacing with a 3D virtual object, the method comprising:

generating a sensing signal for sensing a position and a movement of a user and for sensing a position and a movement of a manipulation object;

verifying, using the sensing signal, whether a predetermined movement of the manipulation object has occurred or whether a contact between the 3D virtual object and the manipulation object has been made; and

generating a predetermined reaction corresponding to the predetermined movement and the contact.

10. The 3D space interface method of claim 9, wherein the sensing signal is generated by at least one of a color camera, a depth camera, and a motion sensor.

11. The 3D space interface method of claim 9, wherein the interfacing includes:

determining a rendering space in which the 3D virtual object is output and a sensing space in which a manipulation of the 3D virtual object is sensed, using the sensing signal;

3D-rendering the 3D virtual object in accordance with the position of the user to output the rendered 3D virtual object in the rendering space;

detecting the manipulation object positioned in the sensing space using the sensing signal and tracing a movement of the manipulation object;

sensing whether contact between the manipulation object and the 3D virtual object has been made;

verifying whether the predetermined movement of the manipulation object has occurred by tracing the movement of the manipulation object; and

generating the predetermined reaction corresponding to one or more of the predetermined movement and the contact.

12. The 3D space interface method of claim 9, wherein the determining of the rendering space and the sensing space includes:

tracing the position of the user and a face of the user using the sensing signal;

determining, as the rendering space, an entire space between a 3D display unit and the face of the user; and

determining, as the sensing space, a predetermined sized space between the 3D display unit and the face of the user.

13. The 3D space interface method of claim 11, wherein the object of the manipulation object detects, as the manipulation object, any one or more of hands of the user within the sensing space, a body of the user within the sensing space, and an object within the sensing space.

14. The 3D space interface method of claim 11, wherein the sensing of whether the contact has been made includes:

correcting a distortion of the rendering and sensing spaces varying depending on the position of the user; and

aggregating the corrected rendering and sensing spaces to verify whether contact between the 3D virtual object output in the corrected rendering space and the manipulation object detected in the corrected sensing space has been made.

15. The 3D space interface method of claim 11, wherein the predetermined reaction corresponding to the predetermined movement and the contact is at least one of a transformation expanding or compressing the 3D virtual object, a rotation of the 3D virtual object, a movement of the 3D virtual object, a selection of a portion of the 3D virtual object where the contact has been made, a performance of a predetermined command corresponding to the predetermined movement or the contact, a creation of a new 3D virtual object, and a removal of the 3D virtual object.

16. The 3D space interface method of claim 11, further comprising:

sending ultrasonic waves to the manipulation object when contact has been made with the 3D virtual object to generate a contact signal enabling a contact sensation.

17. An apparatus for interfacing with a rendered three-dimensional (3D) virtual object in a 3D space, the apparatus comprising:

a sensing unit to generate a sensing signal for tracking a position and a movement of a manipulation object of a user in a sensing space of the 3D virtual object;

an interface unit to verify, using the sensing signal, whether a predetermined movement of the manipulation object with respect to the 3D object has occurred in the sensing space; and

a reaction generation unit to generate a predetermined reaction corresponding to the predetermined movement and to output the predetermined reaction such that it may be perceived by the user.

18. The apparatus of claim 17, wherein the predetermined movement comprises movement that results in contact between the manipulation object and the 3D virtual object.

19. The apparatus of claim 18, wherein the reaction generation unit further includes an ultrasonic wave-generation unit to generate a contact signal enabling the user to perceive a contact sensation by sending ultrasonic waves to the manipulation object when contact has been made with the 3D virtual object.

20. The apparatus of claim 17, wherein the reaction generation unit alters an appearance of the 3D virtual object in response to the predetermined movement.

21. The 3D space interface apparatus of claim 17, wherein, in response to the predetermined movement, the reaction generation unit performs at least one of:

a transformation expanding or compressing the 3D virtual object, a rotation of the 3D virtual object, a movement of the 3D virtual object, a selection of a portion of the 3D virtual object where the contact has been made, a performance of a predetermined command corresponding to the predetermined movement or the contact, a creation of a new 3D virtual object, and a removal of the 3D virtual object.

22. A method for interfacing with a rendered three-dimensional (3D) virtual object in a 3D space, the method comprising:

generating a sensing signal to track a position and a movement of a manipulation object of a user in a sensing space of the 3D virtual object;

verifying, using the sensing signal, whether a predetermined movement of the manipulation object with respect to the 3D object has occurred in the sensing space; and

generating a predetermined reaction corresponding to the predetermined movement and outputting the predetermined reaction such that it may be perceived by the user.

23. The method of claim 22, wherein the predetermined movement comprises movement that results in contact between the manipulation object and the 3D virtual object.

24. The method of claim 23, further comprising:

generating a contact signal enabling the user to perceive a contact sensation by sending ultrasonic waves to the manipulation object when contact has been made with the 3D virtual object.

25. A non-transitory computer-readable storage medium encoded with computer readable code comprising a program for implementing the method of claim 22.