User detection by a computing device

• US 9,766,702 B2
• Filed: 11/03/2014
• Issued: 09/19/2017
• Est. Priority Date: 06/19/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cieplinski, Avi E., Bernstein, Jeffrey Traer, Missig, Julian, Khoe, May-Li, Costanzo, Bianca Cheng, Haggerty, Myra Mary, Kerr, Duncan Robert, Ording, Bas, Chen, Elbert D.
• Primary Examiner(s): DANIELSEN, NATHAN ANDREW
• Application Number:
  US14/531,874
• Publication Number:
  US 20150370323A1
• Time in Patent Office: 1,051 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/485 End-user interface for client configuration
• CPC Class Codes: H04N 21/4852 for modifying audio parameters, e.g. switching between mono and stereo

Abstract

In some embodiments, an electronic device optionally identifies a person'"'"'s face, and optionally performs an action in accordance with the identification. In some embodiments, an electronic device optionally determines a gaze location in a user interface, and optionally performs an action in accordance with the determination. In some embodiments, an electronic device optionally designates a user as being present at a sound-playback device in accordance with a determination that sound-detection criteria and verification criteria have been satisfied. In some embodiments, an electronic device optionally determines whether a person is further or closer than a threshold distance from a display device, and optionally provides a first or second user interface for display on the display device in accordance with the determination. In some embodiments, an electronic device optionally modifies the playing of media content in accordance with a determination that one or more presence criteria are not satisfied.

Claims

1. A first electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while respective activity of a respective type of activity is being performed on a second electronic device, detecting that the second electronic device satisfies presence criteria;

in response to detecting that the second electronic device satisfies presence criteria;

in accordance with a determination that the respective type of activity being performed on the second electronic device is not a type of activity that is specified for transfer to a display device associated with the first electronic device, forgoing transferring display of the respective activity from the second electronic device to the display device associated with the first electronic device; and

in accordance with a determination that the respective type of activity being performed on the second electronic device is a type of activity that is specified for transfer to the display device associated with the first electronic device, transferring display of the respective activity from the second electronic device to the display device associated with the first electronic device, including playing media content on the first electronic device, wherein the media content is included in a user interface for display on the display device;

while playing the media content on the first electronic device, determining whether one or more presence criteria are satisfied, wherein the one or more presence criteria include a criterion that is satisfied while a first person is present at the display device; and

in accordance with a determination that the one or more presence criteria are not satisfied, modifying the playing of the media content at the first electronic device in a first manner.

2. The first electronic device of claim 1, wherein the one or more programs further include instructions for, in accordance with a determination that the one or more presence criteria are satisfied, maintaining the playing of the media content in the user interface.

3. The first electronic device of claim 1, wherein the one or more programs further include instructions for:

in accordance with the determination that the one or more presence criteria are not satisfied, determining a third electronic device associated with the first person, wherein modifying the playing of the media content in the first manner comprises transferring the playing of the media content to the third electronic device.

4. The first electronic device of claim 1, wherein the one or more programs further include instructions for, in accordance with a determination that the one or more presence criteria are satisfied:

determining a relative location of the first person with respect to the display device, and

modifying the playing of the media content in a second manner based on the relative location of the first person, wherein the second manner is different from the first manner.

5. The first electronic device of claim 1, wherein the one or more programs further include instructions for, in accordance with a determination that the one or more presence criteria are satisfied:

determining a distance of the first person from the display device, and

modifying the playing of the media content in a second manner based on the distance of the first person from the display device, wherein the second manner is different from the first manner.

6. The first electronic device of claim 1, wherein the one or more presence criteria include a criterion that is satisfied when the first person is oriented towards the display device.

7. The first electronic device of claim 1, wherein the one or more presence criteria include a criterion that is satisfied when the first person'"'"'s gaze is directed at the display device.

8. The first electronic device of claim 1, wherein the one or more programs further include instructions for:

while displaying the media content to a single person, detecting that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device, modifying the playing of the media content in a second manner by increasing a visibility of the media content in the user interface.

9. The first electronic device of claim 1, wherein the one or more programs further include instructions for:

while displaying the media content to a single person, detecting that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device;

determining whether the media content is designated as private media content, and

in accordance with a determination that the media content is designated as private media content, modifying the playing of the media content in a second manner by reducing a visibility of the media content in the user interface.

10. The first electronic device of claim 9, wherein the one or more programs further include instructions for:

in accordance with a determination that the media content is not designated as private media content, modifying the playing of the media content in a third manner by increasing the visibility of the media content in the user interface.

11. The first electronic device of claim 1, wherein

detecting that the second electronic device satisfies the presence criteria comprises determining that the second electronic device is present at the display device.

12. The first electronic device of claim 1, wherein determining whether the one or more presence criteria are satisfied is based on information received from one or more optical sensors.

13. The first electronic device of claim 1, wherein:

at a first point in time, the one or more presence criteria are not satisfied, and the playing of the media content is modified in the first manner, and the one or more programs further include instructions for;

at a second point in time, after the modification of the playing of the media content in the first manner, determining that the one or more presence criteria are satisfied; and

in response to determining that the one or more presence criteria are satisfied, reversing the modification of the playing of the media content in the first manner.

14. A method comprising:

at a first electronic device with one or more processors and memory;

while respective activity of a respective type of activity is being performed on a second electronic device, detecting that the second electronic device satisfies presence criteria;

in response to detecting that the second electronic device satisfies presence criteria;

in accordance with a determination that the respective type of activity being performed on the second electronic device is not a type of activity that is specified for transfer to a display device associated with the first electronic device, forgoing transferring display of the respective activity from the second electronic device to the display device associated with the first electronic device; and

in accordance with a determination that the respective type of activity being performed on the second electronic device is a type of activity that is specified for transfer to the display device associated with the first electronic device, transferring display of the respective activity from the second electronic device to the display device associated with the first electronic device, including playing media content on the first electronic device, wherein the media content is included in a user interface for display on the display device;

while playing the media content on the first electronic device, determining whether one or more presence criteria are satisfied, wherein the one or more presence criteria include a criterion that is satisfied while a first person is present at the display device; and

in accordance with a determination that the one or more presence criteria are not satisfied, modifying the playing of the media content at the first electronic device in a first manner.

15. The method of claim 14, further comprising:

in accordance with the determination that the one or more presence criteria are not satisfied, determining a third electronic device associated with the first person, wherein modifying the playing of the media content in the first manner comprises transferring the playing of the media content to the third electronic device.

16. The method of claim 14, wherein

detecting that the second electronic device satisfies the presence criteria comprises determining that the second electronic device is present at the display device.

17. The method of claim 14, further comprising:

in accordance with a determination that the one or more presence criteria are satisfied, maintaining the playing of the media content in the user interface.

18. The method of claim 14, further comprising:

in accordance with a determination that the one or more presence criteria are satisfied;

determining a relative location of the first person with respect to the display device, and

modifying the playing of the media content in a second manner based on the relative location of the first person, wherein the second manner is different from the first manner.

19. The method of claim 14, further comprising:

in accordance with a determination that the one or more presence criteria are satisfied;

determining a distance of the first person from the display device, and

modifying the playing of the media content in a second manner based on the distance of the first person from the display device, wherein the second manner is different from the first manner.

20. The method of claim 14, wherein the one or more presence criteria include a criterion that is satisfied when the first person is oriented towards the display device.

21. The method of claim 14, wherein the one or more presence criteria include a criterion that is satisfied when the first person'"'"'s gaze is directed at the display device.

22. The method of claim 14, further comprising:

while displaying the media content to a single person, detecting that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device, modifying the playing of the media content in a second manner by increasing a visibility of the media content in the user interface.

23. The method of claim 14, further comprising:

while displaying the media content to a single person, detecting that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device;

determining whether the media content is designated as private media content, and

in accordance with a determination that the media content is designated as private media content, modifying the playing of the media content in a second manner by reducing a visibility of the media content in the user interface.

24. The method of claim 14, further comprising:

in accordance with a determination that the media content is not designated as private media content, modifying the playing of the media content in a third manner by increasing the visibility of the media content in the user interface.

25. The method of claim 14, wherein determining whether the one or more presence criteria are satisfied is based on information received from one or more optical sensors.

26. The method of claim 14, wherein at a first point in time, the one or more presence criteria are not satisfied, and the playing of the media content is modified in the first manner, further comprising:

at a second point in time, after the modification of the playing of the media content in the first manner, determining that the one or more presence criteria are satisfied; and

in response to determining that the one or more presence criteria are satisfied, reversing the modification of the playing of the media content in the first manner.

27. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a first electronic device with one or more processors and memory, cause the first electronic device to:

while respective activity of a respective type of activity is being performed on a second electronic device, detect that the second electronic device satisfies presence criteria;

in response to detecting that the second electronic device satisfies presence criteria;

in accordance with a determination that the respective type of activity being performed on the second electronic device is not a type of activity that is specified for transfer to a display device associated with the first electronic device, forgo transferring display of the respective activity from the second electronic device to the display device associated with the first electronic device; and

in accordance with a determination that the respective type of activity being performed on the second electronic device is a type of activity that is specified for transfer to the display device associated with the first electronic device, transfer display of the respective activity from the second electronic device to the display device associated with the first electronic device, including playing media content on the first electronic device, wherein the media content is included in a user interface for display on the display device;

while playing the media content on the first electronic device, determine whether one or more presence criteria are satisfied, wherein the one or more presence criteria include a criterion that is satisfied while a first person is present at the display device; and

in accordance with a determination that the one or more presence criteria are not satisfied, modify the playing of the media content at the first electronic device in a first manner.

28. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

in accordance with the determination that the one or more presence criteria are not satisfied, determine a third electronic device associated with the first person, wherein modifying the playing of the media content in the first manner comprises transferring the playing of the media content to the third electronic device.

29. The non-transitory computer readable storage medium of claim 27, wherein

detecting that the second electronic device satisfies the presence criteria comprises determining that the second electronic device is present at the display device.

30. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

in accordance with a determination that the one or more presence criteria are satisfied, maintain the playing of the media content in the user interface.

31. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

in accordance with a determination that the one or more presence criteria are satisfied;

determine a relative location of the first person with respect to the display device, and

modify the playing of the media content in a second manner based on the relative location of the first person, wherein the second manner is different from the first manner.

32. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

in accordance with a determination that the one or more presence criteria are satisfied;

determine a distance of the first person from the display device, and

modify the playing of the media content in a second manner based on the distance of the first person from the display device, wherein the second manner is different from the first manner.

33. The non-transitory computer readable storage medium of claim 27, wherein the one or more presence criteria include a criterion that is satisfied when the first person is oriented towards the display device.

34. The non-transitory computer readable storage medium of claim 27, wherein the one or more presence criteria include a criterion that is satisfied when the first person'"'"'s gaze is directed at the display device.

35. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

while displaying the media content to a single person, detect that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device, modify the playing of the media content in a second manner by increasing a visibility of the media content in the user interface.

36. The non-transitory computer readable storage medium of claim 27, wherein the one or more programs further comprise instructions that cause the first electronic device to:

while displaying the media content to a single person, detect that two or more people are present at the display device; and

in accordance with a determination that two or more people are present at the display device;

determine whether the media content is designated as private media content, and

in accordance with a determination that the media content is designated as private media content, modify the playing of the media content in a second manner by reducing a visibility of the media content in the user interface.

37. The non-transitory computer readable storage medium of claim 36, wherein the one or more programs further comprise instructions that cause the first electronic device to:

in accordance with a determination that the media content is not designated as private media content, modify the playing of the media content in a third manner by increasing the visibility of the media content in the user interface.

38. The non-transitory computer readable storage medium of claim 27, wherein determining whether the one or more presence criteria are satisfied is based on information received from one or more optical sensors.

39. The non-transitory computer readable storage medium of claim 27, wherein:

at a first point in time, the one or more presence criteria are not satisfied, and the playing of the media content is modified in the first manner, and the one or more programs further comprise instructions that cause the first electronic device to;

at a second point in time, after the modification of the playing of the media content in the first manner, determine that the one or more presence criteria are satisfied; and

in response to determining that the one or more presence criteria are satisfied, reverse the modification of the playing of the media content in the first manner.