Attention detection

• US 7,633,527 B2
• Filed: 04/13/2004
• Issued: 12/15/2009
• Est. Priority Date: 04/15/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Pilu, Maurizio
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US10/822,696
• Publication Number:
  US 20040208496A1
• Time in Patent Office: 2,072 Days
• Field of Search: 348/211.4, 455/566.1
• US Class Current: 348/211.4
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

An attention detection system comprises a portable attention detection module for receiving attention clues of a host person. The attention clues are generated from a self perspective of the host. Attention clues of the host person generated from an observer perspective are received. A sensor device generates signals relating to the attention clue signals of the self perspective. At least one sensor generates signals relating to the attention clue signals of the observer perspective. The attention detection module determines a situation of raised attention of the host wearer from the self perspective attention clues and the received observer perspective attention clues.

Claims

1. An attention detection system comprising:

at least one first sensor device, located at a host wearer, for generating a host perspective signal relating to attention clues indicative of the attention of the host wearer to an observer; and

at least one second sensor device, located at the observer, for generating an observer perspective signal relating to attention clues indicative of the attention of the observer to the host wearer; and

a portable attention detector for receiving the host perspective and the observer perspective signals and for determining a situation of raised attention of said host wearer from said received host perspective signal and said received observer perspective signal.

2. The attention detection system as claimed in claim 1, wherein said at least one first sensor device comprises:

an image capture device for capturing an image from the self-perspective of said host wearer in response to the determined situation of raised attention.

3. The system as claimed in claim 1, wherein said at least one first sensor device is adapted to be worn by said host wearer.

4. The system as claimed in claim 1, wherein said at least one second sensor device is adapted to be worn by the observer.

5. The system as claimed in claim 1, wherein said at least one second sensor device is adapted to be located in a place where said host wearer is likely to be.

6. The system as claimed in claim 1, further comprising a people-observing device for communicating with said attention detector.

7. The system as claimed in claim 6, further comprising a plurality of the people-observing devices, each of which is capable of:

communicating with said attention detector; and

communicating with each other.

8. The system as claimed in claim 7, wherein at least one of the people-observing devices comprises a camera device.

9. The system as claimed in claim 1 further comprising first and second user-observing devices, including beacons for locating and detecting the host wearer and the observer.

10. The system as claimed in claim 1, wherein the portable attention detector and at least one first sensor device are integrated into a host wearable device.

11. The system as claimed in claim 1, further comprising a people-observing device for communicating with said attention detector, wherein said people-observing device is configurable for cooperating with at least one other people-observing device for communicating information with said at least one other people-observing device.

12. The system as claimed in claim 1, further comprising a people-observing device for communicating with said attention detector, wherein said people-observing device is configurable for recognizing at least one other people-observing device to form a group of people-observing devices capable of recognizing each other.

13. The system as claimed in claim 12, wherein said people-observing device can be set to communicate or not communicate with at least one other people-observing device based on an analysis of contextual information relating to the host wearer.

14. The system as claimed in claim 1, further comprising a people-observing device configured to be set to selectively communicate with at least one other people-observing device based on an analysis of a type of attention clue detected.

15. The system as claimed in claim 1, further comprising at least one people-observing device configured to be set to selectively communicate with said attention detector depending upon the type of attention clue detected.

16. The system as claimed in claim 1, wherein at least one of the first sensor device and the second sensor device comprises a digital camera device for capturing a digital image.

17. The system as claimed in claim 1, wherein the at least one first sensor device is arranged for detecting a facial expression of said host wearer and for generating the host perspective signal based on the detected facial expression of said host wearer.

18. The system as claimed in claim 1, wherein the at least one first sensor device is arranged for detecting an eye direction of said host wearer and for generating the host perspective signal based on the detected eye direction of said host wearer.

19. The system as claimed in claim 1, wherein the at least one first sensor device is arranged for detecting body language of said host wearer and for generating the host perspective signal based on the detected body language of said host wearer.

20. The system as claimed in claim 1, wherein the at least one first sensor device is arranged for detecting body posture of the host wearer and for generating the host perspective signal based on the detected body posture of the host wearer.

21. The system as claimed in claim 1, wherein the first sensor device or the second sensor device or both sensor devices include a detector for the context of an environment where said host wearer is located, the host perspective signal or the observer perspective signal or both, as appropriate, being dependent on the context of the environment where said host wearer is located.

22. The system as claimed in claim 1 wherein the first sensor device or the second sensor device or both sensor devices include a vocal utterance detector of the host wearer, the host perspective signal or the observer perspective signal or both, as appropriate, being dependent on the vocal utterance detector of the host wearer.

23. A method of capturing images using at least one camera device, said method comprising:

detecting an attention clue exhibited by at least one first animate object from the perspective of a host second animate object carrying said at least one camera device, said attention clue being captured by said at least one camera device and indicating that the first animate object pays attention to the second animate object;

detecting an attention clue of said second animate object from an observer perspective of the at least one first animate object external of said second animate object;

activating said at least one camera device so it captures an image of a subject related to the at least one first animate object in response to detection of said attention clues of the first and second animate objects.

24. The method as claimed in claim 23, wherein said step of detecting an attention clue exhibited by the at least one first animate object comprises detecting a facial expression of said at least one first animate object.

25. The method as claimed in claim 23, wherein said step of detecting an attention clue exhibited by the at least one first animate object comprises detecting an eye direction of said at least one first animate object.

26. The method as claimed in claim 23, wherein said step of detecting an attention clue exhibited by the at least one first animate object comprises detecting body language of said at least one first animate object.

27. The method as claimed in claim 23, wherein said step of detecting an attention clue exhibited by the at least one first animate object comprises detecting body posture of said at least one first animate object.

28. The method as claimed in claim 23, wherein said step of detecting an attention clue exhibited by the at least one first animate object comprises detecting a vocal utterance of said at least one first animate object.

29. The method as claimed in claim 23, wherein the detecting step of the attention clue of said second animate object comprises:

capturing an image from the perspective of the at least one first animate object external to the second animated object; and

performing image processing of said image so there is detected an attention clue of said second animate object, the detected image of the attention clue of the second animate object being selected from the set including;

a facial expression;

an eye direction;

a body movement; and

a body posture.

30. A device for observing at least one first animate object comprising:

an interface for interfacing with at least one sensor device for deriving a first signal indicative of an attention state of the first animate object;

a receiver for a sensor signal representing aspects of body language of the first animate object, the aspects being observed from a position external of said first animate object; and

an analyzer for determining from said first signal and said sensor signal at least one attention clue related to a second animate object observing the first animate object; and

a transmitter for transmitting the attention clue signals.

31. The device as claimed in claim 30, further comprising:

a transponder device adapted to be carried by the first animate object for transmitting said sensor signals to a location displaced from the first animate object.

32. The device as claimed in claim 30, wherein the sensor signal receiver includes:

an image capture device for capturing image frames including aspects of the body language of the first animate object.

33. An attention detection component for determining a level of attention of a host wearer, said component comprising:

a self perspective attention analyzer for analyzing attention clues of the host wearer toward an observer; and

an observer perspective attention analyzer for analyzing attention clues of the observer toward the host wearer from an observed perspective of the host wearer;

wherein the self and observer perspective attention analyzers determine from the attention clues a level of interest of the host wearer, the attention detection component being operable for analyzing said attention clues in a self perspective mode, in which said attention clues relate to the host wearer.

34. The attention detection component as claimed in claim 33, the component being operable in an observer perspective mode, in which said attention clues represent signals describing behavior of the observer observed by the host wearer from a remote location.

35. The attention detection component as claimed in claim 33, comprising a transponder device for receiving said attention clue signals from a remote sender device.

36. A computer readable storage medium storing a computer program for causing a computer to perform steps comprising:

analyzing a plurality of sensor signals representing attention clues collected from a self perspective of a first animate object toward a second animate object, and attention clues from the second animate object toward the first animate object collected from an observed perspective of said first animate object, and determining from said sensor signals and collected attention clues, a behavioral mode of the first animate object; and

generating an image capture Trigger signal for triggering an image capture device to capture image data, in response to said sensed behavioral mode of said first animate object.

37. An attention detection system comprising:

a portable attention detector for receiving attention clues toward an observer generated from a self perspective of a host wearer of said attention detector;

an animate object observing device for observing said host wearer from an observer perspective of the observer external of said host wearer and determining attention clues of said host wearer from said observer perspective externally of said host wearer;

said attention detector being capable of determining a situation of raised attention of said host wearer from said self perspective attention clues, and said received observer perspective attention clues.

38. A system for detecting the attention level of a first animate object comprising:

a first sensor for generating a first signal relating to the attention level of the first animate object to a second animate object from the perspective of the first animate object;

a second sensor for generating a second signal relating to the attention level toward the first animate object from a perspective of the second animate object; and

a processor for determining that the first animate object has a raised attention level in response to the first and second signals.

39. The system of claim 38 wherein the second sensor is arranged to respond to a parameter indicative of the attention being paid to the first animate object by the second animate object.

40. The system of claim 39 wherein the first and second sensors are adapted to be carried by the first animate object.

41. The system of claim 40 wherein the second sensor comprises an image detector adapted to be responsive to at least a portion of an image of the second animate object.

42. The system of claim 39 wherein the first and second sensors are respectively adapted to be carried by the first and second animate objects.

43. The system of claim 42 wherein the second sensor is connected to a transmitter (a) adapted to be carried by the second animate object, and (b) arranged to transmit the second signal from the second animate object to the first animate object.

44. The system of claim 38 wherein the first sensor is arranged to respond to a parameter indicative of the reaction of the first animate object to the environment where the first animate object is located.

45. The system of claim 44 wherein the second sensor is adapted to be located at a position removed from the first animate object, and further comprising a transmitter arranged to transmit the second signal from said position to a receiver (a) arranged to receive the second signal, and (b) adapted to be carried by the first animate object.

46. A method of detecting an attention level of a first animate object comprising:

sensing a first attention level of the first animate object toward a second animate object from the perspective of the first animate object;

sensing a second attention level to the first animate object from a perspective of the second animate object; and

determining that the first animate object has a raised attention level in response to the sensed first and second attention levels.

47. The method of claim 46 wherein the second attention level is generated in response to a parameter indicative of the attention being paid to the first animate object by a second animate object.

48. The method of claim 47 wherein the sensing of the first attention level is by a first sensor carried by the first animate object.

49. The method of claim 48 wherein the sensor comprises an image detector responsive to at least a portion of an image of the second animate object.

50. The method of claim 48 wherein the sensing of the second attention level is by a second sensor carried by the second animate object.

51. The method of claim 50 further comprising transmitting the second signal from the second animate object to the first animate object.

52. The method of claim 46 wherein the sensing of the first attention level responds to a parameter indicative of the reaction of the first animate object to the environment where the first animate object is located.

53. The method of claim 50 wherein the second sensor is located at a position removed from the first animate object, and further comprising transmitting the second signal from said position to a receiver that receives the second signal and is carried by the first animate object.