INFORMING A USER OF GESTURES MADE BY OTHERS OUT OF THE USER'S LINE OF SIGHT

• US 20080059578A1
• Filed: 09/06/2006
• Published: 03/06/2008
• Est. Priority Date: 09/06/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: SESSA, ANTHONY J., GOLDMAN, STEVEN D., ARNOLD, KENNETH C., ALBERTSON, JACOB C, PAOLINI, MICHAEL A.
• Granted Patent: US 7,725,547 B2
• Time in Patent Office: Days
• US Class Current: 709/204
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture-enabled electronic communication system informs users of gestures made by other users participating in a communication session. The system captures a three-dimensional movement of a first user from among the multiple users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user. The system identifies a three-dimensional object properties stream using the captured movement and then identifies a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with multiple electronic communication gesture definitions. In response to identifying the particular electronic communication gesture from among the multiple electronic communication gesture definitions, the system transmits, to the users participating in the electronic communication session, an electronic object corresponding to the identified electronic communication gesture.

Claims

1. A method for an electronic communication service that supports a plurality of electronic communication sessions to inform a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, comprising:

capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions; and

in response to identifying the particular electronic communication gesture from among the plurality of electronic communication gesture definitions, transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture.

2. The method according to claim 1, wherein capturing a three-dimensional movement of a first user further comprises capturing the three-dimensional movement using a stereoscopic video capture device to identify and track a particular three-dimensional movement.

3. The method according to claim 1, wherein capturing a three-dimensional movement of a first user further comprises capturing the three-dimensional movement using at least one stereoscopic video capture device and at least one sensor enabled device to detect a depth of a detected moving object in the three-dimensional movement.

4. The method according to claim 1, wherein capturing a three-dimensional movement of the first user further comprises capturing the three-dimensional movement of the first user when the first user is actively engaged in the electronic communication session by at least one of actively speaking and actively typing.

5. The method according to claim 1, wherein identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises calculating a percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture.

6. The method according to claim 5, further comprising adjusting at least one output characteristic of the electronic object to represent the percentage of certainty.

7. The method according to claim 1, wherein transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture further comprises transmitting the electronic object as a communication entry by the first user in the electronic communication session.

8. The method according to claim 1, wherein transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture further comprises transmitting the electronic object as a command to a tactile detectable device to output a particular tactile detectable output pattern representative of the identified electronic communication gesture.

9. The method according to claim 1, wherein transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture further comprises:

accessing, for each of the plurality of users, a separate user profile, wherein each separate user profile comprises at least one preference for a particular category of gesture object for the electronic communication session from among a plurality of categories of gesture objects, wherein the plurality of categories of gesture objects comprise at least one of text, avatar, graphic, audio, and tactile detectable feedback, wherein the preference is based on at least one factor from among a particular communicative device used by the user for the communication session, a type of the communication session, and the identities of the other users in the communication session;

selecting, from a database of a plurality of gesture objects for the identified electronic communication gesture, for each user, a separate gesture object assigned to the particular category of gesture object designated by the user in the separate user profile; and

transmitting to each of the plurality of users a separate gesture object selected for each user.

10. A system for inform a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, comprising:

a gesture processing system comprising at least one computer system communicatively connected to a network;

said gesture processing system further comprising;

means for capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

means for identifying a three-dimensional object properties stream using the captured movement;

means for identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions; and

means, in response to identifying the particular electronic communication gesture from among the plurality of electronic communication gesture definitions, for transmitting to at least one electronic communication service provider server facilitating the electronic communication session said particular electronic communication gesture, wherein said electronic communication service provider server transmits an electronic object corresponding to the identified electronic communication gesture to least one of the plurality of users.

11. The system according to claim 10, wherein said means for capturing a three-dimensional movement of a first user further comprises means for capturing the three-dimensional movement using a stereoscopic video capture device to identify and track a particular three-dimensional movement.

12. The system according to claim 10, wherein said means for capturing a three-dimensional movement of a first user further comprises means for capturing the three-dimensional movement using at least one stereoscopic video capture device and at least one sensor enabled device to detect a depth of a detected moving object in the three-dimensional movement.

13. The system according to claim 10, wherein said means for means for capturing a three-dimensional movement of the first user further comprises means for capturing the three-dimensional movement of the first user when the first user is actively engaged in the electronic communication session by at least one of actively speaking and actively typing.

14. The system according to claim 10, wherein said means for identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises means for calculating a percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture.

15. The system according to claim 14, further comprising means for adjusting at least one output characteristic of the electronic object to represent the percentage of certainty.

16. The system according to claim 10, wherein said at least one electronic communication service provider server comprises means for transmitting to at least one of the plurality of users participating in the electronic communication session the electronic object corresponding to the identified electronic communication gesture, wherein the electronic object is a communication entry by the first user in the electronic communication session.

17. The system according to claim 10, wherein said at least one electronic communication service provider server comprises means for transmitting to at least one of the plurality of users participating in the electronic communication session the electronic object corresponding to the identified electronic communication gesture, wherein the electronic object is a command to a tactile detectable device to output a particular tactile detectable output pattern representative of the identified electronic communication gesture.

18. The system according to claim 10, further comprising:

said at least one electronic communication service provider server comprising means for accessing, for each of the plurality of users, a separate user profile, wherein each separate user profile comprises at least one preference for a particular category of gesture object for the electronic communication session from among a plurality of categories of gesture objects, wherein the plurality of categories of gesture objects comprise at least one of text, avatar, graphic, audio, and tactile detectable feedback, wherein the preference is based on at least one factor from among a particular communicative device used by the user for the communication session, a type of the communication session, and the identities of the other users in the communication session;

said at least one electronic communication service provider server further comprising means for selecting, from a database of a plurality of gesture objects for the identified electronic communication gesture, for each user, a separate gesture object assigned the particular category designated by the user in the separate user profile; and

said at least one electronic communication service provider server further comprising means for transmitting to each of the plurality of users a separate gesture object selected for each user.

19. A program for informing a plurality of users participating in an electronic communication session of gestures made by at least one of the plurality of users, said program embodied in a computer-readable medium, said program comprising computer-executable instructions which cause a computer to perform the steps of:

capturing a three-dimensional movement of a first user from among a plurality of users participating in an electronic communication session, wherein the three-dimensional movement is determined using at least one image capture device aimed at the first user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular electronic communication gesture representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of electronic communication gesture definitions; and

in response to identifying the particular electronic communication gesture from among the plurality of electronic communication gesture definitions, transmitting to at least one of the plurality of users participating in the electronic communication session an electronic object corresponding to the identified electronic communication gesture.

20. The program according to claim 19, wherein identifying a particular electronic communication gesture representing the three-dimensional object properties stream further comprises:

calculating a percentage of certainty that the captured three-dimensional movement represents a particular gesture defined in the particular electronic communication gesture; and

adjusting at least one output characteristic of the electronic object to represent the percentage of certainty.