Wearable clip for providing social and environmental awareness

• US 10,024,678 B2
• Filed: 09/17/2014
• Issued: 07/17/2018
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Moore, Douglas A., Djugash, Joseph M. A., Ota, Yasuhiro
• Primary Examiner(s): Tran, Khoi H
• Assistant Examiner(s): WALLACE, DONALD JOSEPH
• Application Number:
  US14/489,420
• Publication Number:
  US 20160080897A1
• Time in Patent Office: 1,399 Days
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3617 using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3652 Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G08B 13/2462 Asset location systems combined with EAS inventory, tracking, logistic G06Q10/00; entrance control systems G07C9/00
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services

Abstract

A clip includes an IMU coupled to the clip and adapted to detect inertial measurement data and a GPS coupled to the device and adapted to detect location data. The clip further includes a camera adapted to detect image data and a memory adapted to store data. The clip further includes a processor adapted to recognize an object in the surrounding environment by analyzing the data. The processor can determine a desirable action based on the data and a current time or day. The processor can determine a destination based on the determined desirable action. The processor can determine a navigation path based on the determined destination and the data. The processor is further adapted to determine output based on the navigation path. The clip further includes a speaker adapted to provide audio information to the user.

Claims

1. An intelligent clip to be worn by a user, comprising:

a housing having a front portion, a back portion, a channel positioned on the back portion and configured to receive a connection to a fastening device, a top portion, and a bottom portion; and

one or more components encased within the housing and including;

an inertial measurement unit (IMU) sensor configured to detect inertial measurement data corresponding to a positioning, a speed, a direction of travel or an acceleration of the intelligent clip,

at least one camera including a first camera positioned on the front portion, the first camera being configured to detect image data corresponding to a surrounding environment,

a memory configured to store object data regarding previously determined objects and previously determined user data associated with the user,

a processor connected to the IMU sensor and the at least one camera and configured to;

recognize an object in the surrounding environment based on the detected image data and the stored object data,

determine a desirable event or action based on the recognized object, the previously determined user data, and a current time or day,

determine a destination based on the determined desirable event or action,

determine a plurality of navigation paths based on the determined destination,

filter the plurality of navigation paths based on the inertial measurement data including the speed and the direction of travel to determine a navigation path of the plurality of navigation paths for the user to travel, and

determine output data based on the determined navigation path; and

a speaker or a vibration unit configured to provide the output data to the user.

2. The intelligent clip of claim 1 wherein the vibration unit is

configured to provide haptic information to the user based on the output data.

3. The intelligent clip of claim 1 wherein the memory is configured to store map data and the processor is configured to determine the navigation path based on the stored map data.

4. The intelligent clip of claim 1 wherein the one or more components include:

a wireless communication antenna for establishing an audio or video communication with another portable electronic device or computer used by another person,

wherein the processor is further configured to establish the audio or video communication based on the determined desirable event or action.

5. The intelligent clip of claim 1 wherein the one or more components further include a microphone that is configured to detect a speech of the user or another person, wherein the processor is further configured to:

parse a conversation of the user or the another person into speech elements,

analyze the speech elements based on the previously determined user data, and

determine the desirable event or action further based on the analyzed speech elements.

6. The intelligent clip of claim 1 wherein the processor is further configured to determine the desirable event or action based on the detected inertial measurement data.

7. The intelligent clip of claim 1 wherein the at least one camera includes a second camera facing a second direction, wherein the first camera is facing a first direction.

8. The intelligent clip of claim 1 wherein the processor is configured to filter the detected image data and transfer the filtered image data to a remote processor via a wireless communication antenna.

9. The intelligent clip of claim 1 wherein the at least one camera includes a stereo pair of cameras that are further configured to detect depth information.

10. A method for providing continuous social and environmental awareness by an intelligent clip comprising:

detecting, via a camera or an inertial measurement unit (IMU) sensor, inertial measurement data corresponding to a positioning, a speed, a direction of travel, or an acceleration of the intelligent clip, or image data corresponding to a surrounding environment;

storing, in a memory, object data regarding previously determined objects and previously determined user data regarding a user;

recognizing, by a processor, an object in the surrounding environment based on the detected image data, the stored object data and the inertial measurement data including the speed and the direction of travel of the intelligent clip;

determining, by the processor, a desirable event or action based on the recognized object, the previously determined user data, and a current time or day;

determining, by the processor, a destination based on the determined desirable event or action;

determining, by the processor, a plurality of navigation paths based on the determined destination;

filtering, by the processor, the plurality of navigation paths based on the inertial measurement data including the speed and the direction of travel to determine a navigation path of the plurality of navigation paths for the user to travel;

determining, by the processor, output data based on the determined navigation path; and

providing, via a speaker or a vibration unit, the output data to the user.

11. The method of claim 10 wherein:

providing the output data includes providing audio or haptic information based on divergence data between the object data and the image data.

12. The method of claim 10 wherein the desirable event or action includes alerting the user of an approaching hazard.

13. The method of claim 10 further including transmitting, via an antenna, the image data and the inertial measurement data to a remote device for processing.

14. The method of claim 10 further comprising:

storing, in the memory, map data; and

determining the navigation path is based on the stored map data.

15. The method of claim 10 wherein storing the object data includes storing the object data in a remote database accessible by other intelligent devices.

16. An intelligent clip to be worn by a user, comprising:

a front;

a back;

a first side;

a second side;

an inertial measurement unit (IMU) sensor configured to detect inertial measurement data corresponding to a positioning, a speed, a direction of travel or an acceleration of the intelligent clip;

a first camera positioned on the front of the intelligent clip and configured to detect a first image data corresponding to a surrounding environment;

a second camera positioned on the first side or the second side and configured to detect a second image data corresponding to the surrounding environment;

a memory configured to store object data regarding previously determined objects and previously determined user data associated with the user;

a processor connected to the IMU sensor, the first camera and the second camera and configured to;

recognize an object in the surrounding environment based on the first image data, the second image data, the stored object data and the inertial measurement data including the speed and the direction of travel of the intelligent clip,

determine a desirable event or action based on the recognized object, the previously determined user data, and a current time or day,

determine a destination based on the determined desirable event or action,

determine a plurality of navigation paths based on the determined destination,

filter the plurality of navigation paths based on the inertial measurement data including the speed and the direction of travel to determine a navigation path of the plurality of navigation paths for the user to travel, and

determine output data based on the determined navigation path; and

a speaker or a vibration unit configured to provide the output data to the user.

17. The intelligent clip of claim 16 wherein the second camera has a lower focal length than the first camera and is configured to read fine print.

18. The intelligent clip of claim 16 further comprising a button configured to be used as an input device for selection of a mode of operation.

19. The intelligent clip of claim 16 further comprising a wireless communication antenna configured to communicate with a remote processor.

20. The intelligent clip of claim 16 wherein the memory is accessible remotely by multiple smart devices.