SMART NECKLACE WITH STEREO VISION AND ONBOARD PROCESSING

• US 20150198455A1
• Filed: 12/05/2014
• Published: 07/16/2015
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Djugash, Joseph M.A., Chen, Tiffany L., Yamamoto, Kenichi
• Granted Patent: US 9,915,545 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/1609 Neck
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5007 computer controlled
• CPC Class Codes: A61H 2201/501 connected to external computer devices or networks
• CPC Class Codes: A61H 2201/5043 Displays
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5058 Sensors or detectors
• CPC Class Codes: A61H 2201/5064 Position sensors
• CPC Class Codes: A61H 2201/5071 Pressure sensors
• CPC Class Codes: A61H 2201/5082 Temperature sensors
• CPC Class Codes: A61H 2201/5084 Acceleration sensors
• CPC Class Codes: A61H 2201/5089 Gas sensors, e.g. for oxygen or CO2
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3617 using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3652 : Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
• CPC Class Codes: G06F 3/016 : Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 : using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor

Abstract

A method for providing directions to a blind user of a smart device is described. The method includes detecting, by at least two sensors and in response to a selection of a find mode of the smart device, image data corresponding to a surrounding environment of the smart device and positioning data corresponding to a positioning of the smart device. The method also includes receiving, by an input device, the desired object or the desired location. The method also includes determining, by a processor, the initial location of the smart device based on the image data, the positioning data and map data stored in a memory of the smart device. The method also includes providing, by the output device, the directions to the desired object based on the initial location of the smart device and the map data.

Claims

1. A method for providing directions to a blind user of a smart device, the directions being from an initial location of the smart device to a location of a desired object or a desired location, the method comprising:

detecting, by at least two sensors and in response to a selection of a find mode of the smart device, image data corresponding to a surrounding environment of the smart device and positioning data corresponding to a positioning of the smart device;

receiving, by an input device, the desired object or the desired location;

determining, by a processor, the initial location of the smart device based on the image data, the positioning data and map data stored in a memory of the smart device; and

providing, by the output device, the directions to the desired object based on the initial location of the smart device and the map data.

2. The method of claim 1 further comprising generating, by the processor, the map data based on the image data and the positioning data.

3. The method of claim 1 further comprising:

determining, by the processor, that the desired object or the desired location is not stored in the memory;

requesting, by the processor, additional data from the blind user identifying the desired object or the desired location; and

determining, by the processor, the directions to the desired object based on the initial location of the smart device, the map data and the additional data identifying the desired object or the desired location.

4. The method of claim 1 wherein the output device includes a first vibration unit positioned on a first side of the smart device and a second vibration unit positioned on a second side of the smart device and the directions include vibrational patterns output from the first vibration unit and the second vibration unit.

5. The method of claim 1 wherein the output device includes a speaker and the directions are provided by the speaker in speech output.

6. The method of claim 1 further comprising determining, by the processor, that an obstacle exists in a current path of the smart necklace based on the image data and wherein providing the directions to the desired object includes providing directions that cause the blind user to avoid the obstacle.

7. A method for describing at least one object or person within a predetermined distance and angle of a smart device to a blind user of the smart device, the method comprising:

storing, in a memory, stored image data associated with a plurality of stored objects and people and a plurality of identifiers such that each identifier is associated with a stored object or person;

detecting, by a camera and in response to a selection of an explore mode or a scan mode of the smart device, detected image data corresponding to the at least one object or person;

determining, by a processor, at least a first identifier from the plurality of identifiers corresponding to the at least one object or person within the predetermined distance and angle of the camera based on the detected image data and the stored image data; and

outputting, via a speaker, the at least first identifier.

8. The method of claim 7 wherein the predetermined distance and angle of the camera is less than a field of view of the camera.

9. The method of claim 7 further comprising:

determining, by the processor and in response to the selection of the explore mode, if the explore mode has been canceled;

detecting, by the camera and in response to the explore mode not being canceled, a second image data of at least another object or person after a predetermined period of time;

determining by the processor, at least a second identifier from the plurality of identifiers corresponding to the at least another object or person within the predetermined distance and angle of the camera based on the second image data and the stored image data; and

outputting, via the speaker, the at least second identifier.

10. The method of claim 7, further comprising determining, by the processor, whether a specific object or person is selected for identification based on the detected image data and wherein the first word or phrase includes a specific word or phrase associated with the specific object or person.

11. The method of claim 7, wherein:

the at least one object or person includes a first object or person, a second object or person and a third object or person;

the at least first identifier includes a first identifier corresponding to the first object or person, a second identifier corresponding to the second object or person and a third identifier corresponding to the third object or person; and

outputting the at least first identifier includes outputting the first identifier, the second identifier and the third identifier based on a directional order of the first object or person, the second object or person and the third object or person.

12. The method of claim 7, wherein the camera includes a pair of stereo cameras, the detected image data includes depth information and outputting the at least first identifier further includes outputting the depth information.

13. The method of claim 7 further comprising:

receiving, at the processor, a request for directions to the at least one object or person;

determining, by the processor, directions from a current location of the smart device to a location of the at least one object or person; and

outputting, via the speaker, the directions.

14. The method of claim 7, wherein the predetermined distance and the predetermined angle are different based on whether the explore mode or the scan mode were selected.

15. The method of claim 7 further comprising determining, by the processor, a granularity setting of the smart device wherein the image data corresponds to a first object and a second object, determining at least a first identifier includes determining a first identifier corresponding to the first object and a second identifier corresponding to the second object and outputting the at least first identifier includes outputting the first identifier only based on the granularity setting.

16. A method for storing a first location of a smart device and providing directions to a blind user from a second location of the smart device to the first location of the smart device, the method comprising:

detecting, by a positioning sensor, a first positioning data corresponding to a first location of the smart device and a second positioning data corresponding to a second location of the smart device;

storing, in a memory and in response to a first selection of the capture mode of the smart device, the first positioning data on a map;

determining, by a processor and in response to a second selection of a capture mode of the smart device, directions to the first location from the second location based on the first positioning data and the second positioning data; and

providing, by an output device, the directions.

17. The method of claim 16 further comprising:

detecting, by a camera, image data corresponding to an object or a person;

determining, by the processor, that the object is selected for labeling;

receiving, by an input device, a label for the object or person;

storing, in the memory, the image data and the label; and

associating, in the memory, the image data with the label.

18. The method of claim 16 further comprising detecting, by a camera and in response to the second selection of the capture mode, image data corresponding to an environment of the user and wherein determining the directions includes determining the directions based on the image data.

19. The method of claim 18 wherein determining the directions includes:

determining a route between the first location and the second location;

determining an obstacle along the route based on the image data; and

determining an alternate route based on the obstacle.

20. The method of claim 16, further comprising:

determining, by the processor, that the first location is selected for labeling;

receiving, at the processor, a label for the first location;

storing, in the memory, location data associated with the first location and the label; and

associating, in the memory, the location data with the label.