APPARATUS AND METHOD OF USER INTERFACE FOR MANIPULATING MULTIMEDIA CONTENTS IN VEHICLE

• US 20110260965A1
• Filed: 10/06/2010
• Published: 10/27/2011
• Est. Priority Date: 04/22/2010
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Jin Woo, LEE, Jung Hee
• Application Number:
  US12/898,990
• Publication Number:
  US 20110260965A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Disclosed are provided an apparatus and a method of a user interface for manipulating multimedia contents for a vehicle. An apparatus of a user interface for manipulating multimedia contents for a vehicle according to an embodiment of the present invention includes: a transparent display module displaying an image including one or more multimedia objects; an ultrasonic detection module detecting a user indicating means by using an ultrasonic sensor in a 3D space close to the transparent display module; an image detection module tracking and photographing the user indicating means; and a head unit judging whether or not any one of the multimedia objects is selected by the user indicating means by using information received from at least one of the image detection module and the ultrasonic detection module and performing a control corresponding to the selected multimedia object.

Claims

1. An apparatus of a user interface for manipulating multimedia contents for a vehicle, comprising:

a transparent display module displaying an image including one or more multimedia objects;

an ultrasonic detection module detecting a user indicating means by using an ultrasonic sensor in a 3D space close to the transparent display module;

an image detection module tracking and photographing the user indicating means; and

a head unit judging whether or not any one of the multimedia objects is selected by the user indicating means by using information received from at least one of the image detection module and the ultrasonic detection module and performing a control corresponding to the selected multimedia object.

2. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein the transparent display module include a thin film transistor.

3. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein the image detection module is rotatable and includes a stereo camera tracking the user indicating means.

4. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein the user indicating means is a user'"'"'s hand and the image detection module photographs an image including the hand and a pupil of the user'"'"'s.

5. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 4, wherein the head unit judges whether or not any one of the multimedia objects is selected by the user indicating means on the basis of a vector component acquired by using the position of an end point of the hand and the position of the pupil.

6. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein the ultrasonic detection module detects a 3D shape, a position, and a motion of the user indicating means by configuring n volume elements detected by the plurality of ultrasonic sensors in the 3D space.

7. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein arrangement or display or not of one or more multimedia objects displayed in the transparent display module are changeable depending on a travelling environment of the vehicle and user'"'"'s selection

8. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 1, wherein the transparent display module displays one or more multimedia objects in a 3D.

9. An apparatus of a user interface for manipulating multimedia contents for a vehicle, comprising:

a transparent display displaying an image including one or more multimedia objects;

an ultrasonic sensor detecting an object in a 3D space close to the transparent display;

a stereo camera stereo-photographing the 3D space;

a motion tracker judging whether or not the detected object is a hand and when the object is the hand in accordance with the judgment result, tracking a motion of the hand;

a first coordinate detector detecting a first coordinate corresponding to a 3D position of an end point of the hand;

a second coordinate detector acquiring 3D coordinates of both user'"'"'s pupils from the image photographed by the stereo camera and detecting a second coordinate corresponding to a point where an indication vector linking the first coordinate with a center position of the both pupils meets the transparent display;

a motion analyzer acquiring a user'"'"'s gesture from a motion of the hand;

an integrator acquiring a final intention of the user by integrating the gesture, the first coordinate, and the second coordinate; and

a controller performing predetermined control depending on the acquired final intention.

10. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the plurality of ultrasonic sensors are arranged to form n volume elements in the 3D space.

11. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the motion tracker judges whether or not the detected object is the hand by analyzing the image photographed by the stereo camera or shape information sensed by the ultrasonic sensor.

12. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the motion analyzer acquires the gesture by comparing a previous frame and a current frame of the shape information acquired by the ultrasonic sensor.

13. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the integrator does not consider an item set to be excluded by the user among the gesture, the first coordinate, and the second coordinate at the time of acquiring the final intention.

14. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the integrator integrates the gesture, the first coordinate, and the second coordinate detected at the same time.

15. The apparatus of a user interface for manipulating multimedia contents for a vehicle according to claim 9, wherein the predetermined control is related to at least one of reproduction of multimedia contents provided in the vehicle, navigation control, vehicle information monitoring, and wired/wireless communication control in the vehicle.

16. A method of a user interface for manipulating multimedia contents for a vehicle, comprising:

when an object is detected in a 3D space, verifying whether or not the detected object is a user'"'"'s hand;

detecting a first coordinate which is a 3D coordinate corresponding to an end point of the hand when the object is the hand;

detecting 3D coordinates corresponding to both pupils of the user'"'"'s and detecting a second coordinate corresponding to a point where an indication vector linking the first coordinate with a center position of the both pupils meets a transparent display;

acquiring a user'"'"'s gesture by tracking the hand;

acquiring a final intention of the user by integrating the gesture, the first coordinate, and the second coordinate; and

performing predetermined control depending on the acquired final intention.

17. The method of a user interface for manipulating multimedia contents for a vehicle according to claim 16, wherein at the acquiring a final intention, the gesture, the first coordinate, and the second coordinate detected at the same time are integrated to acquire the final intention of the user.

18. The method of a user interface for manipulating multimedia contents for a vehicle according to claim 16, wherein at the acquiring a final intention, an item set to be excluded by the user is not considered among the gesture, the first coordinate, and the second coordinate.

19. The method of a user interface for manipulating multimedia contents for a vehicle according to claim 16, wherein at the verifying the hand or not, the hand or not is verified by using a stereo image photographed by a stereo camera or shape information sensed by an ultrasonic sensor.

20. The method of a user interface for manipulating multimedia contents for a vehicle according to claim 16, wherein at the acquiring the gesture, the gesture is acquired by comparing a previous frame and a current frame of the shape information acquired by the ultrasonic sensor.