USER INTERFACE DEVICE, USER INTERFACE METHOD, PROGRAM, AND COMPUTER-READABLE INFORMATION STORAGE MEDIUM

• US 20160147307A1
• Filed: 10/03/2013
• Published: 05/26/2016
• Est. Priority Date: 10/03/2012
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Masuko, Soh, Hagiwara, Masato
• Granted Patent: US 10,591,998 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/279 Recognition of textual entities

Abstract

To allow for easy entry of a plurality of characters by handwriting gestures in the air, a user interface device includes template data storage means for storing template data indicating changes in a predetermined writing position when a gesture to write each of a plurality of characters in the air is made, position obtaining means for sequentially obtaining the predetermined writing position when a user makes gestures to sequentially writing characters in the air, similarity evaluation information output means, every time the predetermined writing position is obtained by the position obtaining means, for sequentially outputting similarity evaluation information indicating a similarity between data to be evaluated including a predetermined number of the predetermined writing positions taken in order from newly obtained data and the template data related to each of the plurality of characters, and character string determination means for determining a character string related to the gestures of the user based on the sequentially output similarity evaluation information related to each of the plurality of characters.

Claims

1. A user interface device comprising:

a template data storage unit configured to store template data indicating changes in a predetermined writing position when a gesture to write each of a plurality of characters in the air is made;

a position obtaining unit configured to sequentially obtain the predetermined writing position when a user makes gestures to sequentially writing characters in the air;

a similarity evaluation information output unit configured to sequentially output similarity evaluation information indicating a similarity between data to be evaluated including a predetermined number of the predetermined writing positions taken in order from newly obtained data and the template data related to each of the plurality of characters every time the predetermined writing position is obtained by the position obtaining unit; and

a character string determination unit configured to determine a character string related to the gestures of the user based on the sequentially output similarity evaluation information related to each of the plurality of characters.

2. The user interface device according to claim 1, further comprising

a period determination unit configured to determine a character determination period based on the sequentially output similarity evaluation information related to each of the plurality of characters, wherein

the character string determination unit determines, as characters included in the character string related to the gestures of the user, one or more of the characters related to the similarity evaluation information meeting a predetermined criterion, the similarity evaluation information being output during the character determination period.

3. The user interface device according to claim 2, wherein,

when only a single character is related to the similarity evaluation information meeting the predetermined criterion, the similarity evaluation information being output two or more times in a row during the character determination period, the character string determination unit determines that the single character is included in the character string related to the gestures of the user.

4. The user interface device according to claim 2, wherein,

when it is determined that two or more characters are related to the similarity evaluation information meeting the predetermined criterion, the similarity evaluation information being consecutively output overlapping in time with each other during the character determination period, the character string determination unit determines that any of the two or more characters is alternatively included in the character string related to the gestures of the user.

5. The user interface device according to claim 4, wherein

the character string determination unit stores related characters related to each other, and, when it is determined that two or more characters are related to the similarity evaluation information meeting the predetermined criterion, the similarity evaluation information being consecutively output overlapping in time with each other during the character determination period, and if the two or more characters are the related characters, the character string determination unit determines that a predetermined one of the two or more characters is included in the character string related to the gestures of the user.

6. The user interface device according to claim 1, wherein

the position obtaining unit sequentially obtains three-dimensional coordinates of the predetermined writing position, and sequentially obtains two-dimensional coordinates indicating a projection position of the writing position on a plane determined by at least three of the three-dimensional coordinates.

7. The user interface device according to claim 1, wherein

the character string determination unit comprises:

a unit configured to determine a plurality of primary character strings obtained by joining characters that are selected based on the sequentially output similarity evaluation information;

a unit configured to determine one or more candidates for the character string related to the gestures of the user based on each of the plurality of primary character strings;

a unit configured to calculate a candidate evaluation value that is an evaluation value of each of the candidates based on similarity evaluation information related to the characters included in the primary character string corresponding to the candidate; and

a unit configured to output one or more character strings related to the gestures of the user based on the calculated candidate evaluation values.

8. A user interface method comprising:

sequentially obtaining a predetermined writing position when a user makes gestures to sequentially write characters in the air;

referring to template data storage means that stores template data indicating changes in the predetermined writing position when a gesture to write each of a plurality of characters in the air is made every time the predetermined writing position is obtained,

sequentially outputting similarity evaluation information indicating a similarity between data to be evaluated including a predetermined number of the predetermined writing positions taken in order from newly obtained data and the template data related to each of the plurality of characters, every time the predetermined writing position is obtained; and

determining a character string related to the gestures of the user based on the sequentially output similarity evaluation information related to each of the plurality of characters.

9. A non-transitory computer-readable information storage medium storing a program for causing a computer to function as:

a template data storage unit configured to store template data indicating changes in a predetermined writing position when a gesture to write each of a plurality of characters in the air is made;

a position obtaining unit configured to sequentially obtain the predetermined writing position when a user makes gestures to sequentially writing characters in the air;

a similarity evaluation information output unit configured to sequentially output similarity evaluation information indicating a similarity between data to be evaluated including a predetermined number of the predetermined writing positions taken in order from newly obtained data and the template data related to each of the plurality of characters, every time the predetermined writing position is obtained by the position obtaining unit; and

a character string determination unit configured to determine a character string related to the gestures of the user based on the sequentially output similarity evaluation information related to each of the plurality of characters.

10. (canceled)