Three-dimensional menu system using manual operation tools

• US 9,218,062 B2
• Filed: 11/11/2010
• Issued: 12/22/2015
• Est. Priority Date: 01/26/2010
• Status: Active Grant

• Current Assignee: Intellectual Discovery Co., Ltd.
• Original Assignee: Gwangju Institute of Science and Technology
• Inventors: Woo, Woontack, Lee, Hyeong Mook
• Primary Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US13/575,571
• Publication Number:
  US 20120304128A1
• Time in Patent Office: 1,867 Days
• Field of Search: 715/848, 715/852, 715/850, 715/836
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts

Abstract

Disclosed is an augmented reality-based three-dimensional menu system using manual operation tools. According to the present invention, the three-dimensional menu system comprises: a display device; at least one pair of manual operation tools which are manually operated by the user, and are in a hexahedral shape; an image acquisition device which acquires images for the manual operation tools; and a menu augmentation unit which tracks the manual operation tools from the acquired images, and augments menu items in the vicinity of the manual operation tools of the acquired images, thereby outputting the augmented menu items to the display device.

Claims

1. A three-dimensional menu system comprising:

a display device;

at least a pair of hexahedral manual operation tools configured to be manually manipulated by a user;

an image acquisition device configured to acquire images of the manual operation tools; and

a menu augmenting unit configured to track the manual operation tools using the acquired images, add menu items to the acquired images at positions around the manual operation tools, and output the images to the display device,

wherein the menu augmenting unit comprising;

a tracking unit configured to track the manual operation tools in the acquired images;

a menu organizing unit configured to organize menu items from previously stored menu information in response to manipulation of the manual operation tools; and

a rendering unit configured to add the organized menu items to the acquired images at positions around the manual operation tools and output the images to the display device.

2. The three-dimensional menu system of claim 1, wherein the rendering unit adds the organized menu items around the manual operation tools at regular intervals according to the number of the organized menu items.

3. The three-dimensional menu system of claim 1, wherein a marker is provided on each surface of the hexahedral manual operation tools, and the tracking unit tracks the marker in the acquired images so as to track the manual operation tools.

4. The three-dimensional menu system of claim 1, wherein the hexahedral manual operation tools comprise central magnets at centers of surfaces thereof, respectively.

5. The three-dimensional menu system of claim 4, wherein auxiliary magnets are arranged on each surface of the hexahedral manual operation tools around the center magnet at angular intervals.

6. The three-dimensional menu system of claim 5, wherein the manual operation tools comprise at least two hall sensors between two neighboring auxiliary magnets of the auxiliary magnets.

7. The three-dimensional menu system of claim 6, wherein a hall sensor detecting unit is disposed in the manual operation tools to detect outputs of the hall sensors.

8. The three-dimensional menu system of claim 6, further comprising a rotation detecting unit, wherein when one of the manual operation tools is rotated in a state where the manual operation tools are in contact with each other, the rotation detecting unit detects a rotation direction of the rotated manual operation tool based on an order of outputs of the hall sensors.

9. The three-dimensional menu system of claim 3, wherein the manual operation tools comprise a manipulation button on at least one corner thereof and a manipulation button control unit therein, and the menu augmenting unit comprises a manual operation tool input unit configured to receive a manipulation signal from the manipulation button control unit.

10. Manual operation tools manually operable for use with a three-dimensional menu system in which images of the manual operation tools are acquired so as to track the manual operation tools in the acquired images, add menu items to the acquired images around the manual operation tools, and output the images to a displace device, the manual operation tools being characterized in that:

the manual operation tools have a hexahedral shape and constitute at least one pair; and

the manual operation tools comprises a marker on each surface thereof, a central magnet at a center of each surface thereof, a rotation detecting unit and at least two hall sensors between two neighboring auxiliary magnets of the auxiliary magnets,

wherein auxiliary magnets are arranged on each surface of the hexahedral manual operation tools around the center magnet at regular intervals,

wherein a hall sensor detecting unit is disposed in the manual operation tools to detect outputs of the hall sensors, and

wherein when one of the manual operation tools is rotated in a state where the manual operation tools are in contact with each other, the rotation detecting unit detects a rotation direction of the rotated manual operation tool based on an order of outputs of the hall sensors.

11. The manual operation tools of claim 10, further comprising:

a manipulation button on at least one corner thereof; and

a manipulation button control unit therein to transmit a manipulation signal of the manipulation button.