SMART NECKLACE WITH STEREO VISION AND ONBOARD PROCESSING

• US 20150196101A1
• Filed: 12/05/2014
• Published: 07/16/2015
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Moore, Douglas A., Ota, Yasuhiro, Djugash, Joseph M.A., Chen, Tiffany L., Dayal, Rajiv
• Granted Patent: US 9,629,774 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A44C 15/0015 Illuminated or sound-producing jewellery
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 3/06 Walking aids for blind persons replacing direct visual perception by another kind of perception A61F9/08
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G08B 13/19621 Portable camera
• CPC Class Codes: G08B 21/12 responsive to undesired emission of substances, e.g. pollution alarms
• CPC Class Codes: G08B 3/00 Audible signalling systems; Audible personal calling systems
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

A smart necklace includes a body defining at least one cavity and having a neck portion and first and a second side portions. The necklace includes a pair of stereo cameras that is configured to detect image data including depth information corresponding to a surrounding environment of the smart necklace. The necklace further includes a positioning sensor configured to detect positioning data corresponding to a positioning of the smart necklace. The necklace includes a non-transitory memory positioned in the at least one cavity and configured to store map data and object data. The smart necklace also includes a processor positioned in the at least one cavity, coupled to the pair of stereo cameras, the positioning sensor and the non-transitory memory. The processor is configured to determine output data based on the image data, the positioning data, the map data and the object data.

Claims

1. A smart necklace to be worn around a neck of a user and for providing navigation, environmental awareness and social interaction to a user, the smart necklace comprising:

a body defining at least one cavity and having a neck portion having a first end and a second end, a first side portion coupled to the first end and a second side portion coupled to the second end;

a pair of stereo cameras positioned on a front surface of the first side portion or the second side portion and configured to detect image data including depth information corresponding to a surrounding environment of the smart necklace;

a positioning sensor coupled to the smart necklace and configured to detect positioning data corresponding to a positioning of the smart necklace;

a non-transitory memory positioned in the at least one cavity and configured to store map data and object data; and

a processor positioned in the at least one cavity, coupled to the pair of stereo cameras, the positioning sensor and the non-transitory memory and configured to determine output data based on the image data, the positioning data, the map data and the object data.

2. The smart necklace of claim 1 wherein:

the first side portion includes a first lower portion being rigid and substantially straight and having an upper end and a lower end and a first middle portion having an upper end coupled to the first end of the neck portion and a lower end coupled to the upper end of the first lower portion, the first middle portion being flexible such that the first lower portion can become flush with the user'"'"'s body when the smart necklace is worn; and

the second side portion includes a second lower portion being rigid and substantially straight and having an upper end and a lower end and a second middle portion having an upper end coupled to the second end of the neck portion and a lower end coupled to the upper end of the second lower portion, the second middle portion being flexible such that the second lower portion can become flush with the user'"'"'s body when the smart necklace is worn.

3. The smart necklace of claim 2 further comprising an insert having an upper end configured to attach to the first end of the upper portion, a lower end configured to attach to the upper end of the first middle portion and an electrical connection configured to electrically couple the first end of the upper portion to the upper end of the first middle portion.

4. The smart necklace of claim 2 wherein a width of the first lower portion near the lower end of the first lower portion is larger than a width of the first middle portion and a width of the second lower portion near the lower end of the second lower portion is larger than a width of the second middle portion.

5. The smart necklace of claim 2 further comprising four buttons, each corresponding to a mode of operation of the smart necklace and a button portion having a notch, a top and a bottom such that the top of the button portion is connected to the lower end of the first lower portion, the four buttons being positioned on a front surface of the button portion such that a first button and a second button are positioned above the front surface, a third button and a fourth button are positioned below the front surface of the button portion and the notch is positioned between the four buttons.

6. The smart necklace of claim 1 further comprising a plurality of buttons coupled to the processor and including a first button, a second button and a third button and wherein the processor is further configured to:

determine navigation instructions based on the map data and the positioning data and the image data in response to a depression of the first button;

determine the identity of an object based on the object data and the image data in response to a depression of the second button;

determine a first location of the smart necklace based on the image data, the positioning data and the map data in response to a first depression of the third button;

determine a second location of the smart necklace based on the image data, the positioning data and the map data in response to a second depression of the third button; and

determine navigation instructions from the second location to the first location based on the image data, the positioning data and the map data in response to the second depression of the third button.

7. The smart necklace of claim 1 further comprising:

a first speaker positioned on the first side portion and a second speaker positioned on the second side portion, the first speaker and the second speaker being coupled to the processor and configured to output stereo audio data based on the output data; and

a first vibration unit positioned on the first side portion and a second vibration unit positioned on the second side portion, the first vibration unit and the second vibration unit being coupled to the processor and configured to generate stereo haptic information based on the output data.

8. The smart necklace of claim 5 wherein the first side portion defines a first output cavity such that the first speaker and the first vibration unit are positioned in the first output cavity and the second side portion defines a second output cavity such that the second speaker and the second vibration unit are positioned in the second output cavity.

9. The smart necklace of claim 1 wherein the positioning sensor includes:

an inertial measurement unit (IMU) sensor coupled to the smart necklace and configured to detect inertial measurement data corresponding to a positioning, velocity, or acceleration of the smart necklace; and

a global positioning system (GPS) unit coupled to the smart necklace and configured to detect location data corresponding to a location of the smart necklace.

10. The smart necklace of claim 1 further comprising a battery configured to store energy and a power button coupled to the processor, wherein the processor is configured to switch the smart necklace between an on position and an off position based on a first type of depression of the power button and to determine a status of the battery in response to a second type of depression of the power button.

11. The smart necklace of claim 1 further comprising a light sensor coupled to at least one of the pair of stereo cameras and configured to detect an ambient light, wherein a sensitivity of the at least one of the pair of stereo cameras is configured to be adjusted based on the ambient light.

12. The smart necklace of claim 1 further comprising a necklace battery configured to store energy and a first charging contact configured to receive power via inductive charging, the first charging contact being positioned on a back surface of the first side portion or the second side portion and coupled to the necklace battery such that the power received by the first charging contact may transfer to the battery.

13. The smart necklace of claim 12 further comprising a portable charging unit having a charging battery configured to store energy, a strap and a second charging contact coupled to the charging battery via the strap, the portable charging unit being configured to provide power from the charging battery to the necklace battery via the first charging contact and the second charging contact.

14. The smart necklace of claim 1 further comprising a first magnet positioned near a back surface of the first side portion and a second magnet positioned near a back surface of the second side portion, the first magnet and the second magnet being configured to attach to a magnetic connector.

15. The smart necklace of claim 1 wherein a width of each of the neck portion, the first side portion and the second side portion is greater than a thickness of each of the neck portion, the first side portion and the second side portion such that the smart necklace is more comfortable when worn by a user.

16. A smart necklace to be worn by a user and to provide environmental awareness, social interaction and navigation instructions to a user, the smart necklace comprising:

an upper portion having a first end, a second end and a bottom surface configured to rest on a neck of the user, the upper portion defining an upper cavity and being rigid, substantially straight in a middle of the upper portion and curved towards the first end and the second end;

a first lower portion being rigid and substantially straight, the first lower portion defining a first cavity and having an upper end and a lower end;

a first middle portion having an upper end coupled to the first end of the upper portion and a lower end coupled to the upper end of the first lower portion, the first middle portion being flexible such that the first lower portion can become flush with the user'"'"'s body when the smart necklace is worn;

a second lower portion being rigid and substantially straight, the second lower portion defining a second cavity and having an upper end and a lower end;

a second middle portion having an upper end coupled to the second end of the upper portion and a lower end coupled to the upper end of the second lower portion, the second middle portion being flexible such that the second lower portion can become flush with the user'"'"'s body when the smart necklace is worn;

a pair of stereo cameras positioned on a front surface of the first lower portion and configured to detect image data including depth information corresponding to a surrounding environment of the smart necklace;

a positioning sensor configured to detect positioning data corresponding to a positioning of the smart necklace;

a plurality of buttons positioned on a front surface of the second lower portion, the plurality of buttons including a first button, a second button and a third button;

a non-transitory memory positioned in the upper cavity, the first cavity or the second cavity and configured to store map data and object data;

a processor positioned in the upper cavity, the first cavity or the second cavity and coupled to the non-transitory memory, the plurality of buttons, the pair of stereo cameras, the IMU and the GPS and configured to determine output data including;

navigation instructions based on the map data and the location data, the image data or the inertial measurement data in response to a depression of the first button,

the identity of an object based on the object data and the image data in response to a depression of the second button, and

an association a word or words detected by the microphone with the image data and store the word or words and the image data in memory in response to a depression of the third button; and

an output device coupled to the processor and configured to output the output data.

17. The smart necklace of claim 16 wherein the output device includes:

a first speaker positioned on the first middle portion and a second speaker positioned on the second middle portion, the first speaker and the second speaker being coupled to the processor and configured to output stereo audio data based on the output data; and

a first vibration unit positioned on the first middle portion and a second vibration unit positioned on the second middle portion, the first vibration unit and the second vibration unit being coupled to the processor and configured to generate stereo haptic information based on the navigation instructions.

18. The smart necklace of claim 16 further comprising a microphone positioned on the first lower portion or the second lower portion and configured to detect speech data and wherein the positioning sensor includes:

an inertial measurement unit (IMU) sensor positioned in the upper cavity, the first cavity or the second cavity and configured to detect inertial measurement data corresponding to a positioning, velocity, or acceleration of the smart necklace; and

a global positioning system (GPS) unit positioned in the upper cavity, the first cavity or the second cavity and configured to detect location data corresponding to a location of the smart necklace.

19. The smart necklace of claim 16 further comprising a light sensor coupled to at least one of the pair of stereo cameras and configured to detect an ambient light, wherein a sensitivity of the at least one of the pair of stereo cameras is configured to be adjusted based on the ambient light.

20. A smart necklace for providing environmental awareness and social interaction to a user, comprising:

an upper portion having a first end, a second end and a bottom surface configured to rest on a neck of the user, the upper portion defining an upper cavity and being rigid, substantially straight in a middle of the upper portion and curved towards the first end and the second end;

a button portion being rigid and substantially straight, the button portion having an upper end and a lower end;

a first lower portion being rigid and substantially straight, the first lower portion defining a first cavity and having an upper end and a lower end coupled to the upper end of the button portion;

a first middle portion having an upper end coupled to the first end of the upper portion and a lower end coupled to the upper end of the first lower portion, the first middle portion being flexible such that the button portion and the first lower portion can become flush with the user'"'"'s body when the smart necklace is worn;

a second lower portion being rigid and substantially straight, the second lower portion defining a second cavity and having an upper end and a lower end;

a second middle portion having an upper end coupled to the second end of the upper portion and a lower end coupled to the upper end of the second lower portion, the second middle portion being flexible such that the second lower portion can become flush with the user'"'"'s body when the smart necklace is worn;

a pair of stereo cameras positioned on a front surface of the first lower portion and configured to detect image data including depth information corresponding to a surrounding environment of the smart necklace;

an inertial measurement unit (IMU) sensor positioned in the upper cavity, the first cavity or the second cavity and configured to detect inertial measurement data corresponding to a positioning, velocity, or acceleration of the smart necklace;

a plurality of buttons positioned on a front surface of the button portion;

a non-transitory memory positioned in the upper cavity, the first cavity or the second cavity and configured to store map data and object data;

a processor positioned in the upper cavity, the first cavity or the second cavity and coupled to the non-transitory memory, the plurality of buttons, the pair of stereo cameras, the IMU and the GPS and configured to determine output data based on the inertial measurement data and the image data; and

an output device coupled to the processor and configured to output the output data.