Shared attention detection system and method

• US 20050004496A1
• Filed: 04/29/2004
• Published: 01/06/2005
• Est. Priority Date: 04/29/2003
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Cheatle, Stephen Phillip, Grosvenor, David Arthur, Pilu, Maurizio
• Granted Patent: US 7,106,204 B2
• Time in Patent Office: Days
• US Class Current: 600/595
• CPC Class Codes: G03B 29/00 Combinations of cameras, projectors or photographic printing apparatus with non-photographic non-optical apparatus, e.g. clocks or weapons; Cameras having the shape of other objects combinations with flash apparatus G03B15/03
• CPC Class Codes: H04N 23/61 based on recognised objects

Abstract

An attention detection system detects a condition of shared attention of plural persons. The system includes plural body language detectors each associated with a different person for detecting body language of the associated person. An analyzer receives body language information from the body language detectors, analyzes the body language of the persons and determines when said body language information indicates shared attention between the persons. The attention detection system generates a signal that captures an image of the shared attention.

Claims

1. An attention detection system for detecting a condition of shared attention of first and second animate objects, said system comprising:

a first body language detector for detecting body language of a first of the objects;

a second body language detector for detecting body language of a second of the objects;

an analyzer arranged for receiving body language information from said first and second body language detectors and for analyzing said body language of said first and second objects for determining that said body language information indicates a situation of shared attention between said first and second objects.

2. The attention detection system as claimed in claim 1 wherein the analyzer is arranged for generating an image capture signal for capturing an image in response to the determination of the shared attention between said first and second objects.

3. The attention detection system as claimed in claim 2, further comprising at least one image capture device arranged for capturing the image in response to the signal being generated by said analyzer.

4. The attention detection system as claimed in claim 3 wherein the image capture device includes an optical detector.

5. The attention detection system as claimed in claim 1 wherein said analyzer is arranged for analyzing information exchanged between said first and second objects.

6. The attention detection system as claimed in claim 1, wherein said analyzer is arranged for (a) analyzing said body language of said first object from a first object perspective and (b) analyzing said body language of said second object from an observer perspective.

7. The attention detection system as claimed in claim 1 wherein the objects are persons.

8. The attention detection system as claimed in claim 7, wherein said body language detector comprises a sensor arrangement for at least one of:

eye gaze direction sensor for detecting the direction of gaze of an optical detector of the person;

a body direction sensor for sensing the direction of movement of the body of the person;

a head motion sensor for sensing the direction of motion of a structure of the person carrying an optical detector;

a limb orientation sensor for detecting the orientation direction of a limb of the person in three dimensions;

a facial expression detector for detecting a facial expression of the person; and

a sonic detector for detecting aspects of a noise originating from the person and which indicates a raised level of attention of said person.

9. The attention detection system as claimed in claim 7 wherein said body language detector comprises a sensor arrangement for all of:

eye gaze direction sensor for detecting the direction of gaze of an optical detector of the person;

a body direction sensor for sensing the direction of movement of the body of the person;

a head motion sensor for sensing the direction of motion of a structure of the person carrying an optical detector;

a limb orientation sensor for detecting the orientation direction of a limb of the person in three dimensions;

a facial expression detector for detecting a facial expression of the person; and

a sonic detector for detecting aspects of a noise originating from the person and which indicates a raised level of a attention of said person.

10. The attention detection system as claimed in claim 7 wherein said body language detector comprises a sensor arrangement for a plurality of:

eye gaze direction sensor for detecting the direction of gaze of an optical detector of the person;

a body direction sensor for sensing the direction of movement of the body of the person;

a head motion sensor for sensing the direction of motion of a structure of the person carrying an optical detector;

a limb orientation sensor for detecting the orientation direction of a limb of the person in three dimensions;

a facial expression detector for detecting a facial expression of the person; and

a sonic detector for detecting aspects of a noise originating from the person and which indicates a raised level of a attention of said person.

11. The attention detection system as claimed in claim 1, wherein said analyzer is capable of detecting a mutual presence between said first and second objects.

12. The attention detection system as claimed in claim 1, wherein said analyzer is adapted to be carried by one of said objects.

13. The attention detection system as claimed in claim 12 wherein the objects are persons.

14. A method of enabling shared attention to be detected between first and second animate objects, said method comprising:

detecting attention information of said first object using a first device;

detecting attention information of said second object using a second device; and

exchanging said attention information between said first and second devices.

15. The method of claim 14 further comprising indicating that said exchanged information is about the same attention getting phenomenon.

16. The method as claimed in claim 14 wherein the objects are persons.

20. The method as claimed in claim 14, wherein the detecting step includes detecting a shared static point of common attention between said first and second objects, from an analysis of said shared information.

21. The method as claimed in claim 14, wherein the detecting step includes detecting a single final direction of attention shared by said first and second objects from a determination of detected joint ballistic direction information detected for each of said first and second objects.

22. The method as claimed in claim 14, further comprising:

determining a common change of direction of attention of said first and second objects.

23. The method as claimed in claim 14, wherein the detecting step includes detecting a direction shift of a joint smooth gaze by said objects as a result of said objects following a moving object of shared attention.

24. The method as claimed in claim 14, further comprising:

determining, by at least one of said objects, whether or not to send attention information to an attention information detecting device.

25. The method as claimed in claim 24 wherein the attention information detecting device is at a location different from the object that performs the determining step.

26. The method as claimed in claim 14, wherein each of said objects determines with whom it communicates said exchangeable information.

27. The method as claimed in claim 14, wherein each of said devices determines whether to exchange attention information with another of said devices, based upon an analysis of contextual information concerning the environment where said device is located.

28. The method as claimed in claim 14, further including determining whether or not to send said exchange information by analyzing information based upon the type of detected attention clue.

17. The method as claimed in 16, wherein said exchanged information comprises information including at least one of the following:

direction and gaze of optical detectors of the persons;

direction of movement of the body of the persons;

direction of movement of a structure of a body of each of the persons carrying optical detectors;

orientation direction of a limb of each person in three dimensions;

facial expression of the persons;

noises originating from the persons; and

indicating a raised level of attention of the persons.

18. The method as claimed in 16, wherein said exchanged information comprises information including all of the following:

direction and gaze of optical detectors of the persons;

direction of movement of the body of the persons;

direction of movement of a structure of a body of each of the persons carrying optical detectors;

orientation direction of a limb of each person in three dimensions;

facial expression of the persons;

noises originating from the persons; and

indicating a raised level of attention of the persons.

19. The method as claimed in 16, wherein said exchanged information comprises information including a plurality of the following:

direction and gaze of optical detectors of the persons;

direction of movement of the body of the persons;

direction of movement of a structure of a body of each of the persons carrying optical detectors;

orientation direction of a limb of each person in three dimensions;

facial expression of the persons;

noises originating from the persons; and

indicating a raised level of attention of the persons.

29. A method of capturing at least one image, in a situation of shared attention involving first and second animate objects, said method comprising:

monitoring at least one parameter associated with each of said objects;

determining from the monitored parameters that said objects have a shared attention situation; and

activating an image capture device in response to said determination of a shared attention situation.

30. The method of claim 29 wherein the first and second objects are respectively first and second persons.

31. The method as claimed in claim 30 wherein the image capture device captures a replica of the shared attention situation.

32. The method as claimed in claim 30, wherein said parameter comprises:

movement of a limb of said persons.

33. The method as claimed in claim 30, wherein said parameter comprises eye gaze direction of said persons.

34. The method as claimed in claim 30, wherein said parameter comprises an utterance by said persons.

35. The method as claimed in claim 30, wherein said parameter comprises head movement of said persons.

36. The method as claimed in claim 30 wherein said parameter comprises the pointing direction of a limb of said persons.

37. The method as claimed in claims 30 wherein said determined shared attention situation comprises a shared static attention point of said persons.

38. The method as claimed in claim 30 wherein said determined situation of shared attention comprises a shift of head direction for both of said persons, to a common attention point.

39. The method as claimed in claim 30 wherein said determined situation of shared attention comprises:

detecting a shift in head direction of both of said persons; and

detecting a moving object within a field of view of each of said persons.

40. The method as claimed in claim 39, wherein said shift in head direction comprises a smooth and continuous change of head direction of each of said persons.

41. An image capture system capable of capturing images under conditions of shared attention between a plurality of animate objects, said system comprising:

a wearable image capture device adapted to be worn by a first of the objects;

a first set of sensors for detecting behavioral aspects of said first object;

a second set of sensors for detecting behavioral aspects of a second of the objects;

an analyzer for responding to the detected behavioral aspects sensed by the first and second set of body sensors and for determining a situation of shared attention between said first and second objects in response to the detected behavioral aspects sensed by the first and second set of body sensors.

42. The method as claimed in claim 41 wherein the first and second sets of sensors are adapted to sense body activity of the first and second objects, respectively.

43. The method as claimed in claim 42 wherein the objects are persons and the sensors are adapted to sense for each person at least one of limb movement, eye gaze direction, utterance, head movement, limb pointing direction and head pointing direction.

44. The method as claimed in claim 42 wherein the objects are persons and the sensors are adapted to sense for each person all of limb movement, eye gaze direction, utterance, head movement, limb pointing direction and head pointing direction.

48. The system of claim 44 wherein the image capture device includes an optical detector.

45. A portable image capture system adapted to be worn by a host animate object, said device comprising:

a host portable image capture device;

a sensor arrangement for sensing at least one behavior aspect of said object;

an analyzer for analyzing outputs of said sensor arrangement and for monitoring aspects of behavior of said object;

said analyzer being arranged for receiving a second set of sensor signals for describing behavior aspects of a second object, wherein said analyzer is arranged for analyzing said first set of sensor signals and said second set of sensor signals and determining from said first and second sets of sensor signals a situation in which said first and second sets of sensor signals exhibit a correlation of behavior between said first and second objects.

46. The image capture system of claim 45 wherein the object is a person.

47. The system of claim 46 wherein the image capture device includes an optical detector.

49. An attention detection system for detecting a shared attention between two or more animate objects, said system comprising:

a plurality of sensor systems, each of said sensor systems being associated with one of said objects and being arranged for sensing information concerning a focus of attention of the sensor associated with said object; and

at least one data processing component, said data processing component being arranged for receiving information from at least two of said sensor systems and for determining conditions under which at least said objects associated with the at least two of said sensor systems share the same focus of attention.

50. The system of claim 49 wherein the objects are persons.

51. An image capture system comprising:

at least one image capture device;

a first detector for detecting body language of a first animate object;

a second detector for detecting body language of a second animate object;

an analyzer for analyzing said body language of said first and second animate objects and for deriving a signal in response to said first and second detectors determining a situation of shared attention between said first and second animate objects;

said image capture device being arranged for capturing an image of the shared attention between said first and second objects in response to the signal from said analyzer.

52. The system of claim 51 wherein the objects are persons.

53. The system of claim 52 wherein the image capture device includes an optical detector.

54. The system of claim 51 wherein the image capture device includes an optical detector.