WEARABLE SMART DEVICE FOR HAZARD DETECTION AND WARNING BASED ON IMAGE AND AUDIO DATA

• US 20160210834A1
• Filed: 01/21/2015
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Dayal, Rajiv
• Granted Patent: US 9,576,460 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/0184 by raising an alarm
• CPC Class Codes: A61H 2201/1609 Neck
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5012 using the internet
• CPC Class Codes: A61H 2201/5046 Touch screens
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5064 Position sensors
• CPC Class Codes: A61H 2201/5082 Temperature sensors
• CPC Class Codes: A61H 2201/5089 Gas sensors, e.g. for oxygen or CO2
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/12 executed aboard the object being navigated; Dead reckoning
• CPC Class Codes: G01C 21/3691 Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions arrangements for giving variable traffic instructions G08G1/09
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons

Abstract

A wearable smart device for providing hazard warning information to a user. The wearable smart device includes a microphone configured to detect audio data associated with a potential hazard. The wearable smart device also includes a camera configured to detect image data associated with the potential hazard. The wearable smart device also includes a processor coupled to the microphone and the camera and configured to determine whether the potential hazard presents a real hazard based on the detected audio data and the detected image data.

Claims

1. A wearable smart device for providing hazard warning information comprising:

a microphone configured to detect audio data associated with a potential hazard;

a camera configured to detect image data associated with the potential hazard; and

a processor coupled to the microphone and the camera and configured to determine whether the potential hazard presents a real hazard based on at least one of the detected audio data or the detected image data.

2. The wearable smart device of claim 1 further comprising a memory coupled to the processor and configured to store memory audio data and memory image data associated with at least one hazard and wherein the processor is configured to determine whether the potential hazard presents the real hazard by comparing at least one of the detected audio data or the detected image data to at least one of the memory audio data or the memory image data.

3. The wearable smart device of claim 1 further comprising an inertial measurement unit configured to detect motion data and a global positioning system configured to detect location data and wherein the processor is further configured to determine a current location of the wearable smart device based on at least two of the detected motion data, the detected location data, the detected audio data or the detected image data.

4. The wearable smart device of claim 3 further comprising a memory coupled to the processor and configured to store a stored location and at least one hazard associated with the stored location and wherein the processor is further configured to determine whether the at least one hazard presents a danger based on a comparison of the current location to the stored location.

5. The wearable smart device of claim 1 wherein the microphone includes two microphones spaced apart and the processor is further configured to determine a direction of the potential hazard based on the detected audio data.

6. The wearable smart device of claim 5 wherein the processor is further configured to instruct the camera to focus towards the direction of the potential hazard in response to determining the direction of the potential hazard.

7. The wearable smart device of claim 1 wherein the processor is further configured to determine a distance to the potential hazard based on a volume of the detected audio data.

8. The wearable smart device of claim 1 wherein the processor is further configured to determine a severity level of the potential hazard based on at least one factor.

9. The wearable smart device of claim 8 wherein the at least one factor includes a distance to the potential hazard, whether the potential hazard is approaching, a size of the potential hazard or a volume of the potential hazard.

10. A smart necklace for providing hazard warnings comprising:

an upper portion having a first end and a second end;

a first lower portion coupled to the first end of the upper portion;

a second lower portion coupled to the second end of the upper portion;

at least one microphone positioned on the first lower portion, the second lower portion or the upper portion and configured to detect audio data associated with a potential hazard;

a camera positioned on the first lower portion or the second lower portion and configured to detect image data associated with the potential hazard; and

a processor coupled to the camera and the at least one microphone and configured to determine whether the potential hazard presents a real hazard based on the detected audio data and the detected image data.

11. The smart necklace of claim 10 further comprising a memory coupled to the processor and configured to store memory audio data and memory image data associated with at least one hazard and wherein the processor is configured to determine whether the potential hazard presents the real hazard by comparing the detected audio data and the detected image data to the memory audio data and the memory image data.

12. The smart necklace of claim 10 wherein the at least one microphone includes two microphones and the processor is further configured to determine a direction of the potential hazard based on the detected audio data.

13. The smart necklace of claim 12 wherein the processor is further configured to cause the camera to focus towards the direction of the potential hazard in response to determining the direction of the potential hazard.

14. The smart necklace of claim 10 wherein the processor is further configured to determine a distance to the potential hazard based on a volume of the detected audio data.

15. The smart necklace of claim 10 wherein the processor is further configured to determine a severity level of the potential hazard based on at least one of a distance to the potential hazard, whether the potential hazard is approaching, a size of the potential hazard or a volume of the potential hazard.

16. A method for providing hazard warnings to a user of a wearable smart device comprising:

detecting, by at least two microphones, audio data associated with a potential hazard and including volume information;

detecting, by a camera, image data associated with the potential hazard; and

determining, by a processor, whether the potential hazard presents a real hazard based on the detected audio data and the detected image data.

17. The method of claim 16 further comprising determining, by the processor, whether the detected image data or the detected audio data has a higher quality and wherein determining whether the potential hazard presents the real hazard is performed by providing more weight to which of the detected image data or the detected audio data has the higher quality.

18. The method of claim 16 further comprising storing, in a memory, stored audio data and stored image data associated with at least one hazard and wherein determining whether the potential hazard presents the real hazard is based on a comparison of the detected audio data and the detected image data to the stored audio data and the stored image data.

19. A method for providing hazard warnings to a user of a wearable smart device comprising:

storing, in a memory, a location of a hazard;

detecting, by at least two of a GPS, an IMU, a camera or a microphone, location data;

determining, by a processor, a current location of the wearable smart device;

determining, by the processor, that the current location is within a predetermined distance of the location of the hazard; and

determining, by the processor, feedback data to be provided to the user in response to the current location being within the predetermined distance of the location of the hazard.

20. The method of claim 19 further comprising generating, by a pair of vibration units, haptic feedback based on the feedback data.