Recall device

• US 8,886,298 B2
• Filed: 03/01/2004
• Issued: 11/11/2014
• Est. Priority Date: 03/01/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Williams, Lyndsay, Wood, Kenneth Robert, Schofield, Kevin Michael
• Primary Examiner(s): Bertram, Eric D.
• Application Number:
  US10/790,602
• Publication Number:
  US 20050203430A1
• Time in Patent Office: 3,907 Days
• Field of Search: 600/513, 396 52- 55, 396/69, 396/98, 396/420
• US Class Current: 600/513
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: G01S 19/34 Power consumption
• CPC Class Codes: G03B 2217/005 Blur detection
• CPC Class Codes: G06F 11/1625 in communications, e.g. transmission, interfaces
• CPC Class Codes: G06F 11/2005 using redundant communication controllers
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2141 Access rights, e.g. capability lists, access control lists, access tables, access matrices
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 20/1235 with control of digital rights management [DRM]
• CPC Class Codes: G06Q 20/385 using an alias or single-use codes
• CPC Class Codes: G06Q 20/425 using two different networks, one for transaction and one for security confirmation
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06Q 30/0609 Buyer or seller confidence or verification
• CPC Class Codes: G06Q 50/188 Electronic negotiation
• CPC Class Codes: G07F 17/16 : for devices exhibiting advertisements, announcements, pictures or the like
• CPC Class Codes: G08B 13/19621 : Portable camera
• CPC Class Codes: G08B 13/19628 : of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19695 : Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
• CPC Class Codes: G08B 15/004 : using portable personal devices hand-held or body-worn self-defence devices using repellent gases or chemicals F41H9/10
• CPC Class Codes: H01M 16/006 : of fuel cells with rechargeable batteries
• CPC Class Codes: H01M 2250/30 : Fuel cells in portable systems, e.g. mobile phone, laptop
• CPC Class Codes: H04B 1/0483 : Transmitters with multiple parallel paths
• CPC Class Codes: H04B 1/40 : Circuits
• CPC Class Codes: H04B 7/0604 : with predefined switching scheme
• CPC Class Codes: H04B 7/084 : Equal gain combining, only phase adjustments antenna beam scanning or forming by phase or amplitude control H01Q3/26, e.g. phased arrays
• CPC Class Codes: H04B 7/0894 : using different delays between antennas
• CPC Class Codes: H04B 7/15507 : Relay station based processing for cell extension or control of coverage area, network planning with network coordinated processing with regard to cell extension H04W16/26; network topologies using dedicated repeater stations H04W84/047; terminal devices adapted for relaying to or from an other terminal H04W88/04
• CPC Class Codes: H04B 7/15535 : Control of relay amplifier gain amplifier gain control in general H03G3/00; gain control reducing self - or loop interference H04B7/15578
• CPC Class Codes: H04J 13/12 : Generation of orthogonal codes
• CPC Class Codes: H04L 1/0041 : Arrangements at the transmitter end
• CPC Class Codes: H04L 1/0045 : Arrangements at the receiver end
• CPC Class Codes: H04L 1/0066 : Parallel concatenated codes
• CPC Class Codes: H04L 1/0069 : Puncturing patterns
• CPC Class Codes: H04L 1/0071 : Use of interleaving interleaving per se H03M13/27
• CPC Class Codes: H04L 1/06 : using space diversity
• CPC Class Codes: H04L 1/08 : by repeating transmission, e.g. Verdan system H04L1/1858 and H04L1/189 take precedence
• CPC Class Codes: H04L 1/1819 : with retransmission of additional or different redundancy
• CPC Class Codes: H04L 1/1841 : Resequencing
• CPC Class Codes: H04L 1/1845 : Combining techniques, e.g. code combining
• CPC Class Codes: H04L 1/1848 : Time-out mechanisms
• CPC Class Codes: H04L 12/12 : Arrangements for remote connection or disconnection of substations or of equipment thereof
• CPC Class Codes: H04L 12/2803 : Home automation networks
• CPC Class Codes: H04L 12/2809 : indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/2856 : Access arrangements, e.g. Internet access asynchronous transfer mode networks H04L12/5601; broadband local area networks H04L12/2801; optical access or distribution networks H04Q11/0067; access to open networks H04L12/5691; digital subscriber line end-user equipment and bit-level processing of data on a PSTN-based network H04M11/00; home network gateways H04L12/2834; wireless access networks H04W
• CPC Class Codes: H04L 12/2874 : Processing of data for distribution to the subscribers
• CPC Class Codes: H04L 12/40078 : Bus configuration home automation networks H04L12/2803; arrangements for maintenance or administration H04L41/00
• CPC Class Codes: H04L 12/40195 : by using a plurality of nodes
• CPC Class Codes: H04L 12/44 : Star or tree networks
• CPC Class Codes: H04L 12/56 : Packet switching systems
• CPC Class Codes: H04L 12/6418 : Hybrid transport
• CPC Class Codes: H04L 2001/0096 : Channel splitting in point-to-point links
• CPC Class Codes: H04L 2012/40241 : Flexray
• CPC Class Codes: H04L 2012/40273 : the transportation system being a vehicle
• CPC Class Codes: H04L 2012/6462 : Movable boundaries in packets or frames
• CPC Class Codes: H04L 2209/127 : Trusted platform modules [TPM]
• CPC Class Codes: H04L 41/12 : Discovery or management of network topologies
• CPC Class Codes: H04L 47/10 : Flow control; Congestion control
• CPC Class Codes: H04L 47/22 : Traffic shaping
• CPC Class Codes: H04L 47/2441 : relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/28 : in relation to timing considerations
• CPC Class Codes: H04L 47/34 : ensuring sequence integrity, e.g. using sequence numbers
• CPC Class Codes: H04L 47/50 : Queue scheduling
• CPC Class Codes: H04L 47/621 : Individual queue per connection or flow, e.g. per VC
• CPC Class Codes: H04L 5/0023 : Time-frequency-space
• CPC Class Codes: H04L 5/0042 : intra-user or intra-terminal allocation
• CPC Class Codes: H04L 5/0044 : allocation of payload
• CPC Class Codes: H04L 5/0083 : symbol-by-symbol
• CPC Class Codes: H04L 61/4541 : Directories for service discovery
• CPC Class Codes: H04L 63/0428 : wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 : for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/08 : for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0807 : using tickets, e.g. Kerberos cryptographic mechanisms or cryptographic arrangements for entity authentication using tickets or tokens H04L9/3213
• CPC Class Codes: H04L 63/102 : Entity profiles
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 65/103 : in the network
• CPC Class Codes: H04L 65/104 : in the network
• CPC Class Codes: H04L 65/1043 : Gateway controllers, e.g. media gateway control protocol [MGCP] controllers
• CPC Class Codes: H04L 65/1101 : Session protocols
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/14 : Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/306 : User profiles
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 69/16 : Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/168 : specially adapted for link layer protocols, e.g. asynchronous transfer mode [ATM], synchronous optical network [SONET] or point-to-point protocol [PPP]
• CPC Class Codes: H04L 69/324 : in the data link layer [OSI layer 2], e.g. HDLC
• CPC Class Codes: H04L 69/326 : in the transport layer [OSI layer 4] H04L69/16 takes precedence
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/32 : including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
• CPC Class Codes: H04M 1/0214 : Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/0225 : Rotatable telephones, i.e. the body parts pivoting to an open position around an axis perpendicular to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/0235 : Slidable or telescopic telephones, i.e. with a relative translation movement of the body parts; Telephones using a combination of translation and other relative motions of the body parts
• CPC Class Codes: H04M 1/2535 : adapted for voice communication over an Internet Protocol [IP] network Voice over Internet Protocol (VoIP) network equipment and services H04M7/006; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16
• CPC Class Codes: H04M 3/22 : Arrangements for supervision, monitoring or testing
• CPC Class Codes: H04M 3/42 : Systems providing special services or facilities to subscribers specially adapted for wireless communication networks H04W4/00
• CPC Class Codes: H04M 3/42102 : Making use of the called party identifier
• CPC Class Codes: H04M 7/0057 : Services where the data services network provides a telephone service in addition or as an alternative, e.g. for backup purposes, to the telephone service provided by the telephone services network
• CPC Class Codes: H04M 7/0069 : comprising a residential gateway, e.g. those which provide an adapter for POTS or ISDN terminals
• CPC Class Codes: H04N 1/00127 : Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00132 : in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
• CPC Class Codes: H04N 1/00137 : Transmission
• CPC Class Codes: H04N 1/00148 : Storage information retrieval G06F16/00
• CPC Class Codes: H04N 1/00342 : with a radio frequency tag transmitter or receiver storage of additional information in an RFID tag attached to an image bearing paper sheet H04N1/32138
• CPC Class Codes: H04N 1/2133 : Recording or reproducing at a specific moment, e.g. time interval or time-lapse
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/64738 : Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64784 : Data processing by the network data processing in packet switching systems H04L12/56; flow control in packet networks H04L47/10; intermediate storage or scheduling H04L49/90; provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 2201/0017 : Notifying a communication result
• CPC Class Codes: H04N 2201/0084 : Digital still camera
• CPC Class Codes: H04N 2201/3205 : of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 2201/3226 : of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
• CPC Class Codes: H04N 23/68 : for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/772 : the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/0112 : one of the standards corresponding to a cinematograph film standard
• CPC Class Codes: H04N 7/08 : Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band , e.g. by time division H04N7/007 takes precedence
• CPC Class Codes: H04N 7/148 : Interfacing a video terminal to a particular transmission medium, e.g. ISDN
• CPC Class Codes: H04N 7/188 : Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253
• CPC Class Codes: H04W 24/00 : Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 28/14 : using intermediate storage
• CPC Class Codes: H04W 48/16 : Discovering, processing access restriction or access information
• CPC Class Codes: H04W 52/143 : Downlink power control
• CPC Class Codes: H04W 52/225 : Calculation of statistics, e.g. average, variance
• CPC Class Codes: H04W 52/24 : using SIR [Signal to Interference Ratio] or other wireless path parameters
• CPC Class Codes: H04W 52/241 : taking into account channel quality metrics, e.g. SIR, SNR, CIR, Eb/lo
• CPC Class Codes: H04W 52/242 : taking into account path loss
• CPC Class Codes: H04W 52/245 : taking into account received signal strength
• CPC Class Codes: H04W 52/46 : in multi hop networks, e.g. wireless relay networks
• CPC Class Codes: H04W 52/48 : during retransmission after error or non-acknowledgment
• CPC Class Codes: H04W 72/54 : based on quality criteria
• CPC Class Codes: H04W 74/02 : Hybrid access
• CPC Class Codes: H04W 76/10 : Connection setup
• CPC Class Codes: H04W 8/005 : Discovery of network devices, e.g. terminals
• CPC Class Codes: H04W 8/04 : Registration at HLR or HSS [Home Subscriber Server]
• CPC Class Codes: H04W 80/10 : adapted for application session management, e.g. SIP [Session Initiation Protocol] connection management H04W76/00; arrangements for session management H04L67/14
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/16 : Gateway arrangements
• CPC Class Codes: Y02B 90/10 : Applications of fuel cells in buildings
• CPC Class Codes: Y02E 60/10 : Energy storage using batteries
• CPC Class Codes: Y02E 60/50 : Fuel cells
• CPC Class Codes: Y10S 707/99933 : Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99939 : Privileged access

Abstract

A small wearable recall device is provided to capture images triggered by a combination of a detection of a capture condition (e.g., changes in motion, temperature or light level) followed by a relatively stable period, as detected by an accelerometer. By triggering on the combination of a detected capture condition followed by a detected stability condition, a clearer image of the environment of an interesting event is expected to be captured. The small size of the recall device makes it possible to integrate it into common portable consumer products, such as MP3 players, purses, clothing, hats, backpacks, necklaces, collars, and other human-wearable products.

Claims

1. A portable recall device configured to be carried by a wearer comprising:

a camera;

at least one accelerometer connected to the camera that detects an acceleration of the camera;

one or more environmental sensors adapted to monitor ambient light level, in an environment external to the wearer; and

a controller operably connected to the camera, to the at least one accelerometer, and to the one or more environmental sensors, the controller being configured to;

determine that a change in the monitored ambient light level is above a first threshold; and

in response to the determination, trigger image capture by automatically sending a signal to capture an image using the camera, if the acceleration of the camera is below a second threshold;

wherein the controller is configured to trigger image capture using the camera in response to detecting a change in average light level within a given time period.

2. The portable recall device of claim 1 further comprising:

an audio recording circuit adapted to record ambient sounds,

wherein the controller is operably connected to the audio recording circuit and is adapted to determine whether to record ambient sounds based at least in part on the monitored ambient light level.

3. The portable recall device of claim 1 wherein the plurality of environmental sensors is also configured to monitor for a change in ambient sound.

4. The portable recall device of claim 1 wherein the one or more environmental sensors include a temperature sensor, a light sensor, and a passive infrared radiation detector.

5. The portable recall device of claim 1 wherein the at least one accelerometer comprises:

a plurality of accelerometers, each accelerometer oriented to detect acceleration along a different axis,

wherein the controller is adapted to determine whether the acceleration of the camera is below the second threshold based at least in part on a signal from each accelerometer.

6. The portable recall device of claim 1 further comprising:

a gyroscope,

wherein the controller is operably connected to the gyroscope and is further adapted to instruct the camera to capture the image when a signal from the gyroscope indicates that yawing movement of the camera is below a threshold yawing value.

7. The portable recall device of claim 1 wherein the controller is further adapted to control the camera to capture the image at least a predefined delay period after the determination.

8. The portable recall device of claim 1 further comprising:

a passive infrared detector that monitors the ambient infrared radiation, and

wherein the controller is operably connected to the passive infrared detector.

9. The portable recall device of claim 1, wherein the one or more environmental sensors includes a chemical sensor.

10. The portable recall device of claim 1, further comprising:

at least one interface to play back at least one image captured by the camera to aid the wearer in remembering at least one activity in which the wearer engaged.

11. The portable recall device of claim 1, wherein the controller further determines whether to capture the image using the camera based at least in part on movement of the wearer.

12. The portable recall device of claim 1, wherein the controller triggers image capture by automatically sending a signal to a shutter control line of the camera, in response to the determination that the change in the monitored ambient light level is about the first threshold.

13. The portable recall device of claim 1, wherein the one or more environmental sensors monitor at least one of ambient temperature and ambient infrared radiation.

14. The portable recall device of claim 13, the controller being configured to:

determine that monitored ambient temperature in the environment surrounding the wearer has increased over a third threshold, or that a threshold change in the monitored ambient temperature has occurred within a given time period, and, in response, trigger image capture by automatically sending a signal to capture an image using the camera, if the acceleration of the camera is below the second threshold.

15. A method comprising:

monitoring acceleration of a camera along at least one axis using an accelerometer;

monitoring ambient temperature, ambient light level, and ambient infrared radiation of an environment of the camera with a plurality of environmental sensors;

comparing acceleration of the camera in a current monitoring interval to acceleration of the camera in a previous monitoring interval to determine whether a stable condition is satisfied, the stable condition being satisfied by a difference between the acceleration of the camera in the current monitoring interval and the acceleration of the camera in the previous monitoring interval being less than a first threshold value;

repeating the acceleration monitoring and comparing until the stable condition is satisfied;

comparing a change in the ambient temperature to a second threshold value;

comparing a change in the ambient light level to a third threshold value;

comparing a change in the ambient infrared radiation to a fourth threshold value;

determining that a capture condition is satisfied based on the change in the ambient temperature, the change in the ambient light level, or the change in the ambient infrared radiation being greater than its corresponding threshold value;

determining whether to capture an image based at least in part on whether the stable condition and the capture condition are satisfied; and

when it is determined that an image should be captured, sending a signal to a shutter control line to capture the image by the camera.

16. The method of claim 15 further comprising:

recording ambient sounds responsive to detection of the capture condition.

17. The method of claim 15 wherein the camera includes a wide-angle lens.

18. The method of claim 15 wherein detecting whether the stable condition is satisfied further comprises:

detecting a signal from a gyroscope that indicates that yawing movement of the camera is below a defined threshold.

19. The method of claim 15 wherein capturing the image by the camera comprises:

delaying at least a predefined delay period after determining that the capture condition is satisfied; and

following the predefined delay period, capturing the image.

20. The method of claim 15 further comprising:

reviewing in sequence a plurality of captured images downloaded from the portable recall device.

21. The method of claim 15, wherein the plurality of environmental sensors comprises a light level sensor.

22. The method of claim 15, and further comprising:

monitoring an ambient sound level.

23. The method of claim 15, wherein the ambient temperature is monitored utilizing a temperature sensor.

24. The method of claim 15, wherein the camera is carried or worn by a person while the person engages in at least one activity, and wherein the method further comprises playing back a sequence of one or more images captured to aid the person in remembering the at least one activity in which the person engaged.

25. A method comprising:

monitoring acceleration of a camera along at least one axis using an accelerometer;

monitoring ambient temperature, ambient light level, and ambient infrared radiation with a plurality of environmental sensors;

comparing changes in the ambient temperature, the ambient light level, and the ambient infrared radiation to threshold values;

determining that the capture condition is satisfied upon one of the changes being greater than one of the threshold values;

detecting whether a stable condition is satisfied by comparing acceleration of the camera in a current monitoring interval to acceleration of the camera in a previous monitoring interval, the stable condition being satisfied by a difference between the acceleration of the camera in the current monitoring interval and the acceleration of the camera in the previous monitoring interval being less than an acceleration threshold value;

determining whether to capture an image based at least in part on whether the capture condition is satisfied; and

when it is determined that an image is to be captured;

determining when to capture an image based at least in part on repeating the acceleration monitoring and comparing until the stable condition is satisfied;

sending a signal to a shutter control line to capture an image by the camera at least a predefined delay period after detection of the capture condition;

capturing the image utilizing a wide-angle lens of the camera; and

removing radial distortion from the captured image to generate a corrected image.

26. A digital media player configured to be carried by a wearer comprising:

a camera that is configured to automatically continuously capture images utilizing a wide-angle lens;

a plurality of environmental sensors that monitor ambient temperature, ambient light level, and ambient infrared radiation; and

a controller operably connected to the camera and to the plurality of environmental sensors, the controller sending signals to a shutter control line to capture the images, the controller being configured to compare changes in the ambient temperature, the ambient light level, and the ambient infrared radiation to threshold values, the controller being configured to automatically save a portion of the images that corresponds to at least one of the changes being greater than one of the threshold values, the controller being configured to automatically delete another portion of the images that corresponds to none of the changes being greater than the threshold values, and the controller being configured to automatically remove radial distortion from the saved portion of the images to generate corrected images.

27. The digital media player of claim 26, wherein at least one of the plurality of environmental sensors comprises a light level sensor.