Automatic photography

• US 20040186726A1
• Filed: 10/31/2003
• Published: 09/23/2004
• Est. Priority Date: 10/05/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Co. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Co. (HP Inc.)
• Inventors: Grosvenor, David Arthur
• Granted Patent: US 7,619,660 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

In automatic photographic or electronic camera apparatus, sounds from a person are preferentially received or identified to provide an audio signal, used to produce a saliency signal (indicative of the person'"'"'s response to circumstances) to control the camera or an image signal produced therefrom. The sounds are detected by a sound conduction microphone, e.g. on the head or throat of the person, or by voice identification circuitry used with a less specific microphone not necessarily mounted on the person, who may or may not be wearing the camera. Control can be in real time, or the sound or saliency signal can be recorded with an image signal for subsequent control of the latter. The audio signal is categorised by waveform analysis, e.g. for the recognition of sounds, such as speech or non-speech vocal sounds, and/or non-vocal sounds; signal amplitude may also be taken into account.

Claims

1. Automatic imaging apparatus comprising a camera, sound detection means capable of preferentially receiving or identifying sounds from a predetermined person and producing an audio signal in response thereto, audio means for providing a saliency signal in response to the audio signal, and control means responsive to the saliency signal for controlling the camera, wherein the audio means is arranged to provide said saliency signal by responding to at least one of (a) the prosody of speech, (b) non-speech vocal sounds and (c) non-vocal sounds.

2. Apparatus according to claim 1 wherein the camera is arranged for image recordal on a photosensitive medium.

3. Apparatus according to claim 1 wherein the camera is an electronic camera providing an image signal.

9. Apparatus according to any one of claims 3 to 8 and including recording means for recording the controlled image signal

10. Apparatus according to any preceding claim wherein said audio analysis means is also arranged to provide a said saliency signal by word recognition.

11. Apparatus according to any preceding claim wherein the audio means comprises audio analysis means for analysing the waveform of the audio signal to categorise it as falling within one of a set of predetermined categories.

12. Apparatus according to claim 11 wherein the audio analysis contains a series of reference waveforms against which to perform the categorisation.

13. Apparatus according to any preceding claim wherein the audio analysis is arranged so that it can be trained to recognise new sounds.

14. Apparatus according to any one of claims 11 to 13 wherein the audio analysis means is arranged to take account of the overall signal amplitude or envelope when assigning a category to the audio signal.

15. Apparatus according to any one of claims 11 to 14 wherein the audio analysis means is arranged to provide a saliency signal having a value determined at least in part by the assigned category of the audio signal.

16. Apparatus according to claim 15 wherein the audio analysis means provides a binary said saliency signal according to the assigned category of the audio signal.

17. Apparatus according to claim 15 wherein the audio analysis means provides a saliency signal having one of a plurality of values according to the assigned category of the audio signal.

18. Apparatus according to claim 15 wherein the audio analysis means is arranged to provide a saliency signal which is discretely variable as a function of the assigned category and continuously variable as a function of the overall signal amplitude or envelope of the audio signal.

19. Apparatus according to any preceding claim wherein said sound detection means comprises a sound conduction microphone adapted to be worn on the head or throat region of a said predetermined person for preferentially detecting sounds emitted from the wearer'"'"'s head and producing said audio signal in response thereto.

20. Apparatus according to claim 19 and further comprising a conventional microphone for receiving sounds via the air.

21. Apparatus according to claim 20 and including means for recording the signal from the conventional microphone.

22. Apparatus according to claim 20 or claim 21 wherein the audio analysis means is arranged to take account of the signal from the conventional microphone when categorising said audio signal as falling within one of a set of predetermined categories.

23. Automatic imaging apparatus according to any preceding claim wherein said camera is provided with fastening means for fastening it to a said predetermined person.

24. Automatic imaging apparatus according to any one of claims 1 to 22 wherein said camera is provided with fastening means for fastening at least the imaging optics thereof to the head of a said predetermined person.

25. Apparatus according to any one of claims 1 to 22 wherein said camera is mounted at a camera location other than on the said predetermined person.

26. Apparatus according to claim 1 or claim 21 wherein the recording means is provided with securing means for securing it to a said predetermined person.

27. Apparatus according to claim 1 or claim 21 wherein said recording means is located at a recorder location remote from sound detection means, and comprising means for wirelessly transmitting said audio signal from said sound detection means to said recorder location.

28. Apparatus according to any preceding claim wherein said audio means is mounted at an audio means location remote from the sound detection means, and comprising means for wirelessly transmitting said audio signal to said audio means location.

29. Apparatus according to any preceding claim wherein said control means is located at a control location remote from said audio means, and comprising means for wirelessly transmitting said saliency signal from said audio means to said control location.

4. Automatic imaging apparatus comprising an electronic camera providing an image signal, sound detection means capable of preferentially receiving or identifying sounds from a predetermined person and producing an audio signal in response thereto, audio means for providing a saliency signal in response to the audio signal, and control means responsive to the saliency signal for controlling the image signal, wherein the audio means is arranged to provide said saliency signal by responding to at least one of (a) the prosody of speech, (b) non-speech vocal sounds and (c) non-vocal sounds.

5. Apparatus according to claim 4 and further including recording means for recording said audio signal together with said image signal and playback means for playing back the audio and image signals, wherein said audio means is arranged to provide said saliency signal in response to the replayed audio signal, and said control means is responsive to the saliency signal for controlling the replayed image signal.

6. Apparatus according to claim 4 or claim 5 wherein said control means comprises a gate for selectively transmitting the image signal.

7. Apparatus according to claim 6 wherein said control means includes combination means for combining said saliency signal with other control signals to produce a gate control output for controlling said gate.

8. Apparatus according to claim 6 wherein said gate is arranged to be controlled directly by a binary said saliency signal.

30. A method of imaging comprising the steps of operating a camera to produce a image signal, preferentially receiving or identifying sounds from a predetermined person and producing an audio signal in response thereto, producing a saliency signal in response to the audio signal, and controlling the camera in response to the saliency signal, wherein said saliency signal is determined by at least one of (a) the prosody of speech, (b) non-speech vocal sounds and (c) non-vocal sounds.

31. A method according to claim 30 wherein the camera is an electronic camera providing an image signal.

33. A method according to any one of claims 30 to 32 wherein said step of preferentially receiving or identifying is performed using a sound conduction microphone fitted to the head or throat region of said predetermined person for preferentially detecting sounds emitted from the person'"'"'s head.

34. A method according to any one of claims 30 to 33 wherein the step of producing said saliency signal comprises analysing the waveform of the audio signal to categorise it as falling within one of a set of predetermined categories.

35. A method according to any one of claims 30 to 34 including the step of recording and replaying said audio and video signals prior to producing said saliency signal.

32. A method of imaging comprising the steps of operating a camera to produce an image signal, preferentially receiving or identifying sounds from a predetermined person and producing an audio signal in response thereto, producing a saliency signal in response to the audio signal, and controlling the image signal in response to the saliency signal, wherein said saliency signal is determined by at least one of (a) the prosody of speech, (b) non-speech vocal sounds and (c) non-vocal sounds.