Interface apparatus, method, and recording medium

• US 8,884,985 B2
• Filed: 09/22/2011
• Issued: 11/11/2014
• Est. Priority Date: 02/01/2011
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Nishiyama, Masashi, Watanabe, Tomoki, Shibata, Tomoyuki, Yamauchi, Yasunobu
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): BUTTRAM, ALAN T
• Application Number:
  US13/240,390
• Publication Number:
  US 20120194545A1
• Time in Patent Office: 1,146 Days
• Field of Search: 345/158, 345/173, 345/629, 715/762
• US Class Current: 345/629
• CPC Class Codes: G06F 2203/04807 Pen manipulated menu
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04N 9/3179 Video signal processing therefor

Abstract

According to one embodiment, an interface apparatus for displaying a GUI object according to a position and a direction of a pointing object comprises a display unit, an image capturing unit, a detection unit, and an additional display unit. The display unit displays a movie based on a first image frame group. The image capturing unit obtains a second image frame group by capturing an image of a surface on which the movie is displayed. The detection unit detects a position and a direction of the pointing object based on the first image frame group and the second image frame group. The additional display unit displays a specific shape which is superimposed to the first image frame group, so that the detected position of the pointing object is irradiated with light of no less than a predetermined illuminance and the light attenuates according to a distance from the position.

Claims

1. An interface apparatus for displaying a GUI object according to a position and a direction of a pointing object having an arc shape, comprising:

a display unit configured to display a movie based on a first image frame group;

an image capturing unit configured to obtain a second image frame group by capturing an image of a surface on which the movie is displayed;

a detection unit configured to detect a position and a direction of the pointing object based on the arc shape of the pointing object, the first image frame group, and the second image frame group; and

an additional display unit configured to display a specific shape which is superimposed to the first image frame group, so that the detected position of the pointing object is irradiated with light of no less than a predetermined illuminance and the light attenuates according to a distance from the position, wherein the detection unit is configured to detect a position and a direction of the pointing object by using inner product values of gradient vectors at respective pixels of a template.

2. The apparatus according to claim 1, wherein the detection unit is configured to detect a position and a direction of the pointing object in a region corresponding to the specific shape.

3. The apparatus according to claim 1, wherein the display unit is configured to display a GUI object which is laid out around the pointing object, and has an arc shape defined by partially notching an oval so as not to overlap the pointing object.

4. The apparatus according to claim 1, wherein the additional display unit is configured to display a GUI object which serves as a menu pointable by the pointing object, and has a plurality of icons or text items laid out within the menu, the icons or text items being associated with commands.

5. The apparatus according to claim 3, wherein the additional display unit is configured to display the interface so as to locate a center of the arc shape in a direction of the pointing object.

6. The apparatus according to claim 1, wherein the additional display unit is configured to begin to display the interface according to stoppage of a position of the pointing object over a predetermined time period or a specific locus of the position.

7. The apparatus according to claim 1, wherein the additional display unit is configured to end display of the interface according to a moving distance of not less than a given amount of the pointing object.

8. The apparatus according to claim 4, wherein the additional display unit is configured to highlight an icon or text item corresponding to a direction of the pointing object with respect to other icons or text items.

9. The apparatus according to claim 4, wherein the additional display unit is configured to highlight an icon or text item selected by the pointing object with respect to non-selected icons or text items.

10. The apparatus according to claim 4, wherein the additional display unit is configured to lay out some of the plurality of icons or text items within the menu, and to display a button used to change icons or text items to be displayed.

11. The apparatus according to claim 10, wherein the additional display unit is configured to change the icons or text items to be displayed according to a specific moving locus of the pointing object.

12. The apparatus according to claim 4, wherein the additional display unit is configured to display an icon or text item which overlaps the specific shape in an enlarged scale or to display an icon or text item which overlaps the specific shape together with a balloon.

13. The apparatus according to claim 1, further comprising a fixed position lightness adjustment unit configured to change a brightness of a specific region at a fixed position in each individual frame of the first image frame group.

14. The apparatus according to claim 13, wherein the specific region is at least a part of a periphery of the surface on which the movie is displayed.

15. The apparatus according to claim 13, wherein the detection unit is configured to detect a position of the pointing object or a variation of the second image frame group inside the specific region using a background difference or an inter-frame difference.

16. The apparatus according to claim 13, further comprising a surrounding brightness detection unit configured to detect a brightness of environmental light by comparing a pixel value of a position where a movie is not displayed by the display unit in the second image frame group with a threshold, which is given in advance,

wherein the fixed position lightness adjustment unit changes a movie to be displayed by the additional display unit according to the brightness of the environmental light.

17. An interface method for displaying a GUI object according to a position and a direction of a pointing object having an arc shape, comprising:

controlling a display unit to display a movie based on a first image frame group;

controlling an image capturing unit to obtain a second image frame group by capturing an image of a surface on which the movie is displayed;

controlling a detection unit to detect a position and a direction of the pointing object based on the arc shape of the pointing object, the first image frame group, and the second image frame group;

controlling an additional display unit to display a specific shape which is superimposed to the first image frame group, so that the detected position of the pointing object is irradiated with light of no less than a predetermined illuminance and the light attenuates according to a distance from the position; and

detecting a position and a direction of the pointing object by using inner product values of gradient vectors at respective pixels of a template.

18. A non-transitory storage medium storing a program for displaying a GUI object according to a position and a direction of a pointing object having an arc shape, the program comprising:

displaying a movie based on a first image frame group;

obtaining a second image frame group by capturing an image of a surface on which the movie is displayed;

detecting a position and a direction of the pointing object based on the arc shape of the pointing object, the first image frame group, and the second image frame group;

displaying a specific shape which is superimposed to the first image frame group, so that the detected position of the pointing object is irradiated with light of no less than a predetermined illuminance and the light attenuates according to a distance from the position; and

detecting a position and a direction of the pointing object by using inner product values of gradient vectors at respective pixels of a template.