Smart necklace with stereo vision and onboard processing

• US 9,578,307 B2
• Filed: 01/14/2014
• Issued: 02/21/2017
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Moore, Douglas A., Djugash, Joseph M. A., Ota, Yasuhiro
• Primary Examiner(s): Torrente, Richard
• Assistant Examiner(s): Matt, Marnie
• Application Number:
  US14/154,714
• Publication Number:
  US 20150201181A1
• Time in Patent Office: 1,134 Days
• US Class Current: 1/1
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/1609 Neck
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5007 computer controlled
• CPC Class Codes: A61H 2201/5043 Displays
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5058 Sensors or detectors
• CPC Class Codes: A61H 2201/5064 Position sensors
• CPC Class Codes: A61H 2201/5071 Pressure sensors
• CPC Class Codes: A61H 2201/5082 Temperature sensors
• CPC Class Codes: A61H 2201/5084 Acceleration sensors
• CPC Class Codes: A61H 2201/5089 Gas sensors, e.g. for oxygen or CO2
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A wearable neck device for providing environmental awareness to a user, the wearable neck device includes a flexible tube. A first stereo pair of cameras is encased in a left portion of the flexible tube and a second stereo pair of cameras is encased in a right portion of the flexible tube. A vibration motor within the flexible tube provides haptic and audio feedback to the user. A processor in the flexible tube recognizes objects from the first stereo pair of cameras and the second stereo pair of cameras. The vibration motor provides haptic and audio feedback of the items or points of interest to the user.

Claims

1. A wearable neck device for providing environmental awareness to a user, comprising:

a flexible tube for being worn around a neck of the user, defining a cavity, having a U-shape, and having a center portion configured to rest on a back of a neck of the user, a left portion and a right portion;

a first stereo pair of cameras positioned on the left portion of the flexible tube;

a second stereo pair of cameras positioned on the right portion of the flexible tube;

an inertial measurement unit (IMU) positioned within the cavity and configured to detect inertial measurement data;

a vibration unit positioned within the cavity and configured to provide haptic feedback to the user;

a speaker positioned along the flexible tube and configured to provide audio feedback to the user; and

a processor positioned within the cavity and configured to receive video frames from the first stereo pair of cameras and the second stereo pair of cameras, receive the detected inertial measurement data from the IMU, provide object recognition of items in the video frames, identify points of interest to the user based on the object recognition, determine navigation instructions based on the inertial measurement data and the received video frames, and control the vibration unit to provide haptic feedback and the speaker to provide audio feedback to the user based on the points of interest and the navigation instructions.

2. The wearable neck device of claim 1 further comprising a storage device configured to store a map data.

3. The wearable neck device of claim 1 further comprising an antenna configured for wireless communication.

4. The wearable neck device of claim 1 further comprising at least one of a radio frequency identification (RFID) reader for reading RFID tags or a near field communication (NFC) unit for reading NFC tags.

5. The wearable neck device of claim 1 further comprising a global position system (GPS) unit positioned in the cavity and configured to detect location data and wherein the processor is further configured to determine the navigation instructions based on the location data.

6. The wearable neck device of claim 1 further comprising a microphone for receiving voice commands.

7. The wearable neck device of claim 1 further comprising a battery capable of being wirelessly charged.

8. The wearable neck device of claim 1 further comprising an input device for adjusting parameters for haptic and audio feedback.

9. The wearable neck device of claim 1 further comprising at least one of a fifth camera positioned on the left portion of the flexible tube or a sixth camera positioned on the right portion of the flexible tube, the at least one of the fifth camera or the sixth camera being configured to increase a field of view for the wearable neck device.

10. The wearable neck device of claim 1 wherein the first stereo pair of cameras is removable from the left portion of the flexible tube and the second stereo pair of cameras is removable from the right portion of the flexible tube.

11. The wearable neck device of claim 1 wherein the first stereo pair of cameras is coupled to the flexible tube using at least one of a retractable tether, a string or a cable.

12. The wearable neck device of claim 10 wherein at least one of the first stereo pair of cameras or the second stereo pair of cameras are capable of isolated object recognition when placed near an item.

13. The wearable neck device of claim 1 further comprising a removable tube having a fifth camera, the removable tube being coupled to at least one of the left portion or the right portion using at least one of a magnetic attachment;

a clip;

a retractable tether, a string or a cable, wherein the fifth camera is capable of isolated object recognition when placed near an item.

14. A wearable neck device for providing environmental awareness to a user, comprising:

a band defining a cavity and having a center portion, a left portion and a right portion;

a first stereo pair of cameras positioned on the left portion of the band;

a first camera positioned to a side of the first stereo pair of cameras;

a second stereo pair of cameras positioned on the right portion of the band;

a second camera positioned to a side of the second stereo pair of cameras;

an inertial measurement unit (IMU) coupled to the band and configured to detect inertial measurement data;

a vibration unit positioned within the cavity and configured to provide haptic feedback to the user;

a speaker positioned along the band and configured to provide audio feedback to the user; and

a processor positioned within the cavity and configured to receive video frames from the first stereo pair of cameras, the first camera, the second stereo pair of cameras and the second camera, receive the inertial measurement data from the IMU, provide object recognition of items in the video frames, identify points of interest to the user based on the object recognition, determine navigation instructions based on the inertial measurement data and the video frames, and control the vibration unit to provide haptic feedback and the speaker to control the speaker to provide audio feedback to the user based on the items of interest and the navigation instructions.

15. The wearable neck device of claim 14 further comprising an overmold covering at least a portion of the band to provide cushioning and grip.

16. The wearable neck device of claim 14, further comprising:

a storage device configured to store a map data, and

a global position system (GPS) unit coupled to the band and configured to detect location data,

wherein the processor is further configured to determine the navigation instructions based on the location data.

17. The wearable neck device of claim 14 further comprising an antenna configured for wireless communication.

18. The wearable neck device of claim 14 wherein the first stereo pair of cameras is removable from the left portion of the band and the second stereo pair of cameras is removable from the right portion of the band to provide isolated object recognition when the first stereo pair of cameras or the second stereo pair of cameras is placed near an item.