Enhanced interface for voice and video communications

• US 8,325,214 B2
• Filed: 09/23/2008
• Issued: 12/04/2012
• Est. Priority Date: 09/24/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan
• Primary Examiner(s): Kuntz, Curtis
• Assistant Examiner(s): ROBINSON, RYAN C
• Application Number:
  US12/235,831
• Publication Number:
  US 20090079813A1
• Time in Patent Office: 1,533 Days
• Field of Search: 348 1401- 1416, 715/852, 715/863
• US Class Current: 348/14.03
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04M 2203/1025 of avatars
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 2203/6054 Biometric subscriber identification
• CPC Class Codes: H04M 3/42 Systems providing special services or facilities to subscribers specially adapted for wireless communication networks H04W4/00
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

An enhanced interface for voice and video communications, in which a gesture of a user is recognized from a sequence of camera images, and a user interface is provided include a control and a representation of the user. The process also includes causing the representation to interact with the control based on the recognized gesture, and controlling a telecommunication session based on the interaction.

Claims

1. A computer-implemented method comprising:

recognizing an engagement gesture of a user from a sequence of camera images, wherein

a plurality of persons are present in at least one image of the sequence of camera images, the plurality of persons including the user;

focusing one or more camera images on the user using a user reference location identified based on the recognized engagement gesture and a face of the user;

providing a user interface comprising a control and a representation of the user;

causing the representation to interact with the control based on the recognized gesture; and

controlling a telecommunication session based on the interaction.

2. The method of claim 1, wherein:

controlling the telecommunication session further comprises changing the telecommunication session from a first state to a second, different state, and

the first or second states comprise a standby state, a notification state, an outgoing call state, a connecting state, a mute state, a menu state, a connection options state, a direct dial state, a redial state, a search state, a browse state, a message menu state, a message playback state, or a configuration state.

3. The method of claim 1, wherein the telecommunication session further comprises a voice and video telecommunication session.

4. The method of claim 1, wherein controlling the telecommunication session further comprises changing the telecommunication session from a standby state to a call or menu state based on recognizing an engagement gesture.

5. The method of claim 1, further comprising:

determining an identity of the user; and

enabling the control based on the determined identity.

6. The method of claim 5, further comprising changing the telecommunication session from a connecting state to a call state only if the determined identity of the user matches an intended recipient of a call.

7. The method of claim 1, further comprising:

determining identities of users within the image; and

enabling or disabling the control based on at least one of the determined identities.

8. The method of claim 1, wherein controlling the telecommunication session further comprises changing the telecommunication session to a standby state based on recognizing a disengagement gesture.

9. The method of claim 1, wherein focusing the one of the camera images further comprises maintaining focus of a camera on the user, who is moving.

10. The method of claim 1, wherein focusing the one of the camera images further comprises centering the one of the camera images on the user.

11. The method of claim 1, further comprising localizing audio to focus on the user.

12. The method of claim 1, wherein:

recognizing the engagement gesture of the user further comprises recognizing that a second user has performed a second engagement gesture after a first user has performed a first engagement gesture,

controlling the telecommunication session further comprises changing a focus to the second user based on recognizing that the second user has performed the second engagement gesture.

13. The method of claim 12, further comprising:

determining whether the first user has relinquished the focus,

wherein the focus is changed only if the first user has relinquished the focus.

14. The method of claim 13, wherein determining whether the first user has relinquished the focus further comprises determining whether the first user has finished speaking.

15. The method of claim 13, wherein determining whether the first user has relinquished the focus further comprises determining whether the first user has performed a relinquishment gesture.

16. The method of claim 1, further comprising:

determining an identity of the user,

wherein controlling the telecommunication session further comprises displaying, in a browse state, a set of contacts associated with the determined identity.

17. The method of claim 1, further comprising:

determining an identity of the user based on face recognition,

wherein controlling the telecommunication session further comprises transmitting the determined identity of the user to a recipient of a call placed by the user.

20. The method of claim 1, further comprising:

identifying a user reference position of the user based at least in part on the engagement gesture and a face position of the user of the plurality of persons.

21. The method of claim 20, wherein identifying the user reference position of the user based at least in part on the engagement gesture and the face position of the user of the plurality of persons is at least partially based on a distance between the engagement gesture and the face position.

22. The method of claim 21, wherein the distance is a horizontal distance.

18. A device comprising a processor configured to:

recognize an engagement gesture of a user from a sequence of camera images, wherein;

a plurality of persons are present in at least one image of the sequence of camera images, the plurality of persons including the user;

identify a user reference position of the user based at least in part on the engagement gesture and a face position of the user of the plurality of persons;

set the user as active using the user reference position;

focus one or more camera images on the user using the user reference position;

provide a user interface comprising a control and a representation of the user;

cause the representation to interact with the control based on the recognized gesture; and

control a telecommunication session based on the interaction.

19. A computer-readable storage medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

recognizing an engagement gesture of a user from a sequence of camera images, wherein;

a plurality of persons are present in at least one image of the sequence of camera images, the plurality of persons including the user;

identifying a user reference position of the user based at least in part on the engagement gesture and a face position of the user of the plurality of persons;

setting the user as active using the user reference position;

focusing one or more camera images on the user using the user reference position;

providing a user interface comprising a control and a representation of the user;

causing the representation to interact with the control based on the recognized gesture; and

controlling a telecommunication session based on the interaction.

23. An apparatus comprising:

means for recognizing an engagement gesture of a user from a sequence of camera images, wherein

a plurality of persons are present in at least one image of the sequence of camera images, the plurality of persons including the user;

means for focusing one or more camera images on the user using a user reference location identified based on the recognized engagement gesture and a face of the user;

means for providing a user interface comprising a control and a representation of the user;

means for causing the representation to interact with the control based on the recognized gesture; and

means controlling a telecommunication session based on the interaction.