Informing a user of gestures made by others out of the user's line of sight

• US 7,725,547 B2
• Filed: 09/06/2006
• Issued: 05/25/2010
• Est. Priority Date: 09/06/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Goldman, Steven D., Sessa, Anthony J., Arnold, Kenneth C., Paolini, Michael A., Albertson, Jacob C.
• Primary Examiner(s): NEURAUTER, GEORGE C
• Application Number:
  US11/470,421
• Publication Number:
  US 20080059578A1
• Time in Patent Office: 1,357 Days
• Field of Search: 709/206, 715/863, 382/107, 382/154
• US Class Current: 709/206
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture-enabled electronic communication system informs users of gestures made by other users participating in a communication session. The system captures a three-dimensional movement of a first user from among the multiple users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user. The system identifies a three-dimensional object properties stream using the captured movement and then identifies a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with multiple electronic communication gesture definitions. In response to identifying the particular electronic communication gesture from among the multiple electronic communication gesture definitions, the system transmits, to the users participating in the electronic communication session, an electronic object corresponding to the identified electronic communication gesture.

Claims

1. A method for an electronic communication service that supports a plurality of electronic communication sessions to inform a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, comprising:

capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions;

displaying within a user interface, the captured movement;

prompting the first user, within the user interface, to verify the captured movement is intended to communicate the particular electronic communication gesture or to select to assign another electronic communication gesture to the captured movement;

responsive to the first user verifying the captured movement is intended to communicate the particular communication gesture, updating a record of the particular electronic communication gesture within the plurality of electronic communication gesture definitions with the verification;

responsive to the first user selecting to assign another electronic communication gesture to the capture movement, setting the particular electronic communication gesture to the another electronic communication gesture; and

in response to the first user verifying the identified particular electronic communication gesture from among the plurality of electronic communication gesture definitions, transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture by transmitting the electronic object as a command to a tactile detectable device to output a particular tactile detectable output pattern representative of the identified electronic communication gesture and sending the command to output the particular tactile detectable output pattern at a level of tactile pulse intensity set to a percentage of certainty.

2. The method according to claim 1, wherein capturing a three-dimensional movement of a first user further comprises:

prompting the first user, within with the user interface, to perform at least one particular type of gesture from among a plurality of types of gestures and displaying a selectable option to record within the user interface;

responsive to detecting the first user select the selectable option to record, capturing a three-dimensional movement of the first user, wherein the three-dimensional movement is determined using the at least one image capture device aimed at the first user;

identifying a three-dimensional object properties stream using the captured movement; and

updating the plurality of electronic communication gesture definitions with the identified three-dimensional object properties stream indentified as said at least one particular type of gesture for said first user.

3. The method according to claim 1, wherein capturing a three-dimensional movement of a first user further comprises capturing the three-dimensional movement using at least one stereoscopic video capture device and at least one sensor enabled device to detect a depth of a detected moving object in the three-dimensional movement.

4. The method according to claim 1, wherein capturing a three-dimensional movement of the first user further comprises capturing the three-dimensional movement of the first user when the first user is actively engaged in the electronic communication session by at least one of actively speaking and actively typing.

5. The method according to claim 1, wherein identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises:

calculating a percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture;

accessing a minimum prediction percentage threshold set for the particular gesture;

comparing the percentage of certainty with a minimum percentage of certainty with the calculated percentage of certainty; and

responsive to the calculated percentage of certainty not exceeding the minimum percentage of certainty, prompting the user to confirm whether the particular gesture matches an intended communication by the three-dimensional movement of the user prior to transmitting the electronic object.

6. The method according to claim 5, further comprising:

adjusting a graphical transparency of the electronic object to a transparency percentage equal to the percentage of certainty.

7. The method according to claim 1, wherein transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture further comprises transmitting the electronic object as a communication entry by the first user in the electronic communication session.

8. The method according to claim 1, wherein transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture further comprises:

accessing, for each of the plurality of users, a separate user profile, wherein each separate user profile comprises at least one preference for a particular category of gesture object for the electronic communication session from among a plurality of categories of gesture objects, wherein the plurality of categories of gesture objects comprise at least one of text, avatar, graphic, audio, and tactile detectable feedback, wherein the preference is based on at least one factor from among a particular communicative device used by the user for the communication session, a type of the communication session, and the identities of the other users in the communication session;

selecting, from a database of a plurality of gesture objects for the identified electronic communication gesture, for each user, a separate gesture object assigned to the particular category of gesture object designated by the user in the separate user profile; and

transmitting to each of the plurality of users a separate gesture object selected for each user.

9. A system for inform a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, comprising:

a gesture processing system comprising at least one computer system communicatively connected to a network;

said gesture processing system further comprising;

means for capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

means for identifying a three-dimensional object properties stream using the captured movement;

means for identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions;

means for displaying within a user interface, the captured movement;

means for prompting the first user, within the user interface, to verify captured movement is intended to communicate the particular electronic communication gesture or to select to assign another electronic communication gesture to the captured movement;

means, responsive to the first user verifying the captured movement is intended to communicate the particular communication gesture, for updating a record of the particular electronic communication gesture within the plurality of electronic communication gesture definitions with the verification;

means, responsive to the first user selecting to assign another electronic communication gesture to the captured movement, for setting the particular electronic communication gesture to the another electronic communication gesture; and

means, in response to the first user verifying the indentified particular electronic communication gesture from among the plurality of electronic communication gesture definitions, transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture; and

at least one electronic communication service provider server that comprises;

means for transmitting to at least one of the plurality of users participating in the electronic communication session the electronic object corresponding to the identified electronic communication gesture, wherein the electronic object is a command to a tactile detectable device to output a particular tactile detectable output pattern representative of the identified electronic communication gesture and sending the command to output the particular tactile detectable output pattern at a level of tactile pulse intensity set to a percentage of certainty.

10. The system according to claim 9, wherein said means for capturing a three-dimensional movement of a first user further comprises:

means for promoting the first user, within with the user interface, to perform at least one particular type of gesture from among a plurality of types of gestures and displaying a selectable option to record within the user interface;

means, responsive to detecting the first user select the selectable option to record, for capturing a three-dimensional movement of the first user, wherein the three-dimensional movement is determined using the at least one image capture device aimed at the first user;

means for identifying a three-dimensional object properties stream using the captured movement; and

means for updating the plurality of electronic communication gesture definitions with the identified three-dimensional object properties stream identified as said at least one particular type of gesture for said first user.

11. The system according to claim 9, wherein said means for capturing a three-dimensional movement of a first user further comprises means for capturing the three-dimensional movement using at least one stereoscopic video capture device and at least one sensor enabled device to detect a depth of a detected moving object in the three-dimensional movement.

12. The system according to claim 9, wherein said means for means for capturing a three-dimensional movement of the first user further comprises means for capturing the three-dimensional movement of the first user when the first user is actively engaged in the electronic communication session by at least one of actively speaking and actively typing.

13. The system according to claim 9, wherein said means for identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises:

means for calculating the percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture;

means for accessing a minimum prediction percentage threshold set for the particular gesture;

means for comparing the percentage of certainty with a minimum percentage of certainty with the calculated percentage of certainty; and

means, responsive to the calculated percentage of certainty not exceeding the minimum percentage of certainty, for prompting the user to confirm whether the particular gesture matches an intended communication by the three-dimensional movement of the user prior to transmitting the electronic object.

14. The system according to claim 13, further comprising means for adjusting a graphical transparency of the electronic object to a transparency percentage equal to the percentage of certainty.

15. The system according to claim 9, wherein said at least one electronic communication service provider server comprises means for transmitting to at least one of the plurality of users participating in the electronic communication session the electronic object corresponding to the identified electronic communication gesture, wherein the electronic object is a communication entry by the first user in the electronic communication session.

16. The system according to claim 9, further comprising:

said at least one electronic communication service provider server comprising means for accessing, for each of the plurality of users, a separate user profile, wherein each separate user profile comprises at least one preference for a particular category of gesture object for the electronic communication session from among a plurality of categories of gesture objects, wherein the plurality of categories of gesture objects comprise at least one of text, avatar, graphic, audio, and tactile detectable feedback, wherein the preference is based on at least one factor from among a particular communicative device used by the user for the communication session, a type of the communication session, and the identities of the other users in the communication session;

said at least one electronic communication service provider server further comprising means for selecting, from a database of a plurality of gesture objects for the identified electronic communication gesture, for each user, a separate gesture object assigned the particular category designated by the user in the separate user profile; and

said at least one electronic communication service provider server further comprising means for transmitting to each of the plurality of users a separate gesture object selected for each user.

17. A computer program product for informing a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, said program embodied in a volatile or non-volatile computer-readable medium, said program comprising computer-executable instructions which cause at least one computer to perform the steps of:

capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions;

displaying within a user interface, the captured movement;

prompting the first user, within the user interface, to verify the captured movement is intended to communicate the particular electronic communication gesture or to select to assign another electronic communication gesture to the captured movement;

responsive to the first user verifying the captured movement is intended to communicate the particular communication gesture, updating a record of the particular electronic communication gesture within the plurality of electronic communication gesture definitions with the verification;

responsive to the first user selecting to assign another electronic communication gesture to the captured movement, setting the particular electronic communication gesture to the another electronic communication gesture; and

in response to the first user verifying the identified particular electronic communication gesture from among the plurality of electronic communication gesture definitions, transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture by transmitting the electronic object as a command to a tactile detectable device to output a particular tactile detectable output pattern representative of the identified electronic communication gesture and sending the command to output the particular tactile detectable output pattern at a level of tactile pulse intensity set to a percentage of certainty.

18. The program according to claim 17, wherein identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises:

calculating the percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture; and

adjusting at least one output characteristic of the electronic object to represent the percentage of certainty.