Image analysis and communication device control technology

• US 9,384,382 B1
• Filed: 01/02/2015
• Issued: 07/05/2016
• Est. Priority Date: 09/29/2009
• Status: Active Grant

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Hudson River, Series 77 of Allied Security Trust I (Allied Security Trust)
• Inventors: Denise, Jason Adam
• Primary Examiner(s): Bekele, Mekonen
• Application Number:
  US14/588,576
• Time in Patent Office: 550 Days
• Field of Search: 382/190, 382/199, 382/195, 382/110, 345/786, 345/784, 345/688, 379/265.01, 379/265.02, 379/265.11
• US Class Current: 1/1
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72463 to restrict the functionality of the device
• CPC Class Codes: H04M 1/724631 by limiting the access to the user interface, e.g. locking a touch-screen or a keypad
• CPC Class Codes: H04M 1/72484 wherein functions are triggered by incoming communication events
• CPC Class Codes: H04M 19/04 the ringing-current being generated at the substations
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04M 3/02 Calling substations, e.g. by ringing selective calling H04Q
• CPC Class Codes: H04W 4/12 Messaging; Mailboxes; Announcements

Abstract

Techniques are described for controlling communication devices using image analysis. For instance, when a communication is received by a communication device, the communication device outputs an alert to notify users of the received communication and accesses one or more images of an area proximate to the communication device. The one or more images cover an area proximate to the communication device at a time during which the communication device is outputting the alert. The communication device analyzes the one or more images to determine whether a user is present in the one or more images and in a position to perceive the received communication. The communication device handles at least one aspect of the received communication based on the determination of whether a user is in a position to perceive the received communication.

Claims

1. A method of using image analysis to control an electronic communication device, the method comprising:

receiving, using the electronic communication device, the electronic communication; and

providing, using the electronic communication device, an alert in response to receiving the electronic communication;

capturing, using at least one image capture device, one or more images of an area proximate to an electronic communication device;

based on receipt of an electronic communication by the electronic communication device, analyzing, using at least one processing device, the one or more captured images to determine whether a user is in a position that enables perception of the electronic communication, wherein the step of analyzing comprises;

while providing the alert, detecting user activity indicative of whether the user perceived the electronic communication, and

based on the detected user activity, determining whether a user perceived and intentionally ignored the electronic communication or whether the electronic communication was not perceived; and

handling, using at least one processing device, at least one aspect of the electronic communication based on the determination of whether the user is in a position that enables perception of the electronic communication, wherein the step of handling comprises;

handling, using at least one processing device, the at least one aspect of the electronic communication in a first manner based on a determination that a user perceived and intentionally ignored the electronic communication, and

handling, using at least one processing device, the at least one aspect of the electronic communication in a second manner based on a determination that the electronic communication was not perceived by a user, wherein the second manner of handling the at least one aspect of the electronic communication is different than the first manner of handling the at least one aspect of the electronic communication.

2. The method of claim 1 wherein:

analyzing, using at least one processing device, the one or more captured images to determine whether a user is in a position that enables perception of the electronic communication comprises determining whether a face is present within the one or more captured images; and

handling, using at least one processing device, at least one aspect of the electronic communication comprises;

in response to a determination that a face is not present within the one or more captured images, handling the electronic communication as not being perceived by a user;

in response to a determination that a face is present within the one or more captured images, handling the electronic communication as being perceived by a user.

3. The method of claim 2 wherein handling the electronic communication as being perceived by a user comprises:

determining whether an identity of the face detected within the one or more captured images is known or unknown;

in response to a determination that the identity of the face detected within the one or more captured images is unknown, handling the electronic communication as being perceived by an unknown user; and

in response to a determination that the identity of the face detected within the one or more captured images is known;

accessing, from electronic storage, personalized electronic communication handling settings for the identified user, and

handling the electronic communication based on the personalized electronic communication handling settings for the identified user.

4. The method of claim 1 wherein:

receiving, using the electronic communication device, the electronic communication comprises receiving, using the electronic communication device, a phone call;

providing, using the electronic communication device, an alert in response to receiving the electronic communication comprises providing an alert in response to receiving the phone call;

detecting user activity indicative of whether the user perceived the electronic communication comprises detecting user activity indicative of whether a user perceived the phone call during provision of the alert;

determining whether the user perceived and intentionally ignored the electronic communication or whether the electronic communication was not perceived comprises ending provision of the alert without establishing the phone call and determining, based on the detected user activity, whether a user was in a position that enables perception of the phone call during a time between beginning provision of the alert in response to receiving the phone call and ending provision of the alert;

handling, using at least one processing device, the at least one aspect of the electronic communication in a first manner based on a determination that a user perceived and intentionally ignored the electronic communication comprises handling, using at least one processing device, the at least one aspect of the electronic communication in a first manner based on a determination that a user was in a position that enables perception of the phone call during a time between beginning provision of the alert in response to receiving the phone call and ending provision of the alert, and

handling, using at least one processing device, the at least one aspect of the electronic communication in a second manner based on a determination that the electronic communication was not perceived by a user comprises handling, using at least one processing device, the at least one aspect of the electronic communication in a second manner based on a determination that a user was not in a position that enables perception of the phone call during a time between beginning provision of the alert in response to receiving the phone call and ending provision of the alert.

5. The method of claim 1 wherein handling, using at least one processing device, the at least one aspect of the electronic communication in a first manner based on a determination that a user perceived and intentionally ignored the electronic communication comprises:

in response to a determination that a user perceived and intentionally ignored the electronic communication, determining one or more types of user activity that led to the determination that a user perceived and intentionally ignored the electronic communication; and

handling the electronic communication based on the one or more determined types of user activity.

6. The method of claim 1 further comprising:

tracking electronic communications determined to be perceived and intentionally ignored;

identifying a source of electronic communications that are relatively frequently perceived and intentionally ignored; and

performing an operation related to device settings of the electronic communication device based on the identified source.

7. The method of claim 1 further comprising:

tracking electronic communications determined to be perceived and intentionally ignored;

identifying a type of electronic communications that are relatively frequently perceived and intentionally ignored; and

performing an operation related to device settings of the electronic communication device based on the identified type of electronic communications.

8. The method of claim 1 further comprising:

tracking electronic communications determined to be perceived and intentionally ignored;

identifying a time period during which a user relatively frequently perceives and intentionally ignores electronic communications; and

performing an operation related to device settings of the electronic communication device based on the identified time period.

9. The method of claim 1 further comprising:

monitoring action of a user subsequent to perceiving and intentionally ignoring an electronic communication;

based on the monitored action, determining how the user handles intentionally ignored electronic communications from a particular source; and

performing an operation based on the determination of how the user handles intentionally ignored electronic communications from the particular source.

10. The method of claim 9 wherein:

determining how the user handles intentionally ignored electronic communications from the particular source comprises determining whether the user returns intentionally ignored electronic communications from the particular source; and

performing the operation based on the determination of how the user handles intentionally ignored electronic communications from the particular source comprises;

in response to a determination that the user returns intentionally ignored electronic communications from the particular source, performing an operation related to automatically ignoring future electronic communications received from the particular source; and

in response to a determination that the user does not return intentionally ignored electronic communications from the particular source, performing an operation related to automatically blocking future electronic communications received from the particular source.

11. The method of claim 1 further comprising:

monitoring action of a user subsequent to perceiving and intentionally ignoring an electronic communication;

based on the monitored action, determining how the user handles intentionally ignored electronic communications of a particular type; and

performing an operation based on the determination of how the user handles intentionally ignored electronic communications of the particular type.

12. The method of claim 11 wherein:

determining how the user handles intentionally ignored electronic communications of a particular type comprises determining whether the user returns intentionally ignored electronic communications of a particular type; and

performing the operation based on the determination of how the user handles intentionally ignored electronic communications of a particular type comprises;

in response to a determination that the user returns intentionally ignored electronic communications of the particular type, performing an operation related to automatically ignoring future electronic communications received from the particular source; and

in response to a determination that the user does not return intentionally ignored electronic communications of the particular type, performing an operation related to automatically blocking future electronic communications of the particular type.

13. The method of claim 1 further comprising:

monitoring action of a user subsequent to perceiving and intentionally ignoring an electronic communication;

based on the monitored action, determining how the user handles intentionally ignored electronic communications received during a particular time period; and

performing an operation based on the determination of how the user handles intentionally ignored electronic communications received during the particular time period.

14. The method of claim 13 wherein:

determining how the user handles intentionally ignored electronic communications received during a particular time period comprises determining whether the user returns intentionally ignored electronic communications received during a particular time period; and

performing the operation based on the determination of how the user handles intentionally ignored electronic communications received during a particular time period comprises;

in response to a determination that the user returns intentionally ignored electronic communications received during a particular time period, performing an operation related to automatically ignoring future electronic communications received during a particular time period; and

in response to a determination that the user does not return intentionally ignored electronic communications received during a particular time period, performing an operation related to automatically blocking future electronic communications received during a particular time period.

15. The method of claim 1 further comprising:

determining whether a user returns perceived and intentionally ignored electronic communications from a particular source;

in response to a determination that the user does not return perceived and intentionally ignored electronic communications from the particular source, handling a perceived and intentionally ignored electronic communication from the particular source without setting a reminder related to returning the perceived and intentionally ignored electronic communication; and

in response to a determination that the user returns perceived and intentionally ignored electronic communications from the particular source;

determining timing information related to when the user returns perceived and intentionally ignored electronic communications from the particular source, and

based on the determined timing information, providing a reminder related to returning a perceived and intentionally ignored electronic communication from the particular source.

16. The method of claim 1 further comprising:

determining whether a user returns perceived and intentionally ignored electronic communications of a particular type;

in response to a determination that the user does not return perceived and intentionally ignored electronic communications of the particular type, handling a perceived and intentionally ignored electronic communication of the particular type without setting a reminder related to returning the perceived and intentionally ignored electronic communication; and

in response to a determination that the user returns perceived and intentionally ignored electronic communications of the particular type;

determining timing information related to when the user returns perceived and intentionally ignored electronic communications of the particular type, and

based on the determined timing information, providing a reminder related to returning a perceived and intentionally ignored electronic communication of the particular type.

17. The method of claim 1 further comprising:

determining whether a user returns perceived and intentionally ignored electronic communications received during a particular time period;

in response to a determination that the user does not return perceived and intentionally ignored electronic communications received during the particular time period, handling a perceived and intentionally ignored electronic communication received during the particular time period without setting a reminder related to returning the perceived and intentionally ignored electronic communication; and

in response to a determination that the user returns perceived and intentionally ignored electronic communications received during the particular time period;

determining timing information related to when the user returns perceived and intentionally ignored electronic communications received during the particular time period, and

based on the determined timing information, providing a reminder related to returning a perceived and intentionally ignored electronic communication received during the particular time period.

18. The method of claim 1 further comprising:

detecting motion of the electronic communication device;

detecting a direction of motion of the electronic communication device;

detecting a speed of motion of the electronic communication device;

controlling communication settings of the electronic communication device based on the detected motion, the detected direction of motion, and the detected speed of motion.

19. A method of using image analysis to control an electronic communication device, the method comprising:

capturing, using at least one image capture device, one or more images of an area proximate to an electronic communication device;

based on receipt of an electronic communication by the electronic communication device, analyzing, using at least one processing device, the one or more captured images to determine whether a user is in a position that enables perception of the electronic communication; and

handling, using at least one processing device, at least one aspect of the electronic communication based on the determination of whether the user is in a position that enables perception of the electronic communication,

wherein analyzing, using at least one processing device, the one or more captured images to determine whether a user is in a position that enables perception of the electronic communication comprises;

monitoring for a user in images captured after receipt of an incoming electronic communication, and

determining whether a user is detected in the images captured after receipt of the incoming electronic communication based on the monitoring; and

handling, using at least one processing device, at least one aspect of the electronic communication comprises;

in response to a determination that a user is detected in the images captured after receipt of the incoming electronic communication, handling at least one aspect of the incoming electronic communication based on the detected user;

in response to a determination that a user is not detected in the images captured after receipt of the incoming electronic communication;

determining whether a threshold amount of time has passed from receipt of the incoming electronic communication;

in response to a determination that a threshold amount of time has not passed from receipt of the incoming electronic communication, maintaining communication settings of the electronic communication device and continuing to monitor for a user in images captured after receipt of the incoming electronic communication; and

in response to a determination that a threshold amount of time has passed from receipt of the incoming electronic communication, updating electronic communication handling settings of the electronic communication device and handling the incoming electronic communication based on the updated communication handling settings.

20. The method of claim 19 further comprising:

detecting motion of the electronic communication device;

detecting a direction of motion of the electronic communication device;

detecting a speed of motion of the electronic communication device;

controlling communication settings of the electronic communication device based on the detected motion, the detected direction of motion, and the detected speed of motion.