WEARABLE EYEGLASSES FOR PROVIDING SOCIAL AND ENVIRONMENTAL AWARENESS

• US 20160078278A1
• Filed: 09/17/2014
• Published: 03/17/2016
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Moore, Douglas A., Djugash, Joseph M.A., Ota, Yasuhiro
• Granted Patent: US 9,922,236 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Eyeglasses include a left lens, a right lens and an IMU sensor and a GPS unit. A camera and a memory are coupled to the eyeglasses. A processor is connected to the IMU, the GPS unit and the at least one camera and is adapted to recognize objects by analyzing image data based on the stored object data and inertial measurement data or location data. The processor is also adapted to determine a desirable event based on the object, previously determined user data, and a time. The processor is also adapted to determine a destination based on the determined desirable event and determine a navigation path for navigating the eyeglasses to the destination based on the determined destination, image data, and inertial measurement data or location data. The processor is also adapted to determine output data based on the determined navigation path. A speaker is also provided.

Claims

1. Eyeglasses to be worn by a user, comprising:

a left lens;

a right lens;

an inertial measurement unit (IMU) sensor coupled to the eyeglasses and configured to detect inertial measurement data corresponding to a positioning, velocity, or acceleration of the eyeglasses;

a global positioning system (GPS) unit coupled to the eyeglasses and configured to detect location data corresponding to a location of the eyeglasses;

at least one camera positioned on at least one of the left lens or the right lens and coupled to the eyeglasses, the at least one camera configured to detect image data corresponding to a surrounding environment of the eyeglasses;

a memory configured to store object data regarding previously determined objects and previously determined user data associated with the user;

a processor connected to the IMU, the GPS unit and the at least one camera and configured to;

recognize an object in the surrounding environment by analyzing the image data based on the stored object data and at least one of the inertial measurement data or the location data,

determine a desirable event or action based on the recognized object, the previously determined user data, and a current time or day,

determine a destination based on the determined desirable event or action,

determine a navigation path for navigating the eyeglasses to the destination based on the determined destination, the image data, and at least one of the inertial measurement data or the location data, and

determine output data based on the determined navigation path; and

a speaker configured to provide audio information to the user based on at least one of the recognized object, determined desirable event or action, or navigation path.

2. The eyeglasses of claim 1 further comprising:

a vibratory motor coupled to the eyeglasses configured to provide first haptic information to the user based on the determined output data.

3. The eyeglasses of claim 1 wherein the memory is configured to store map data and the processor is configured to determine the navigation path based on the image data, the map data, and at least one of the location data or the inertial measurement data.

4. The eyeglasses of claim 1 further comprising:

a wireless communication antenna for establishing an audio or video communication with another portable electronic device or computer used by another person,

wherein the processor is further configured to establish the audio or video communication based on the determined desirable event or action.

5. The eyeglasses of claim 1 further comprising a first connector adapted to electrically attach the left lens to the eyeglasses and a second connector adapted to electrically attach the right lens to the eyeglasses.

6. The eyeglasses of claim 1 further comprising a microphone configured to detect a speech of the user or another person, wherein the processor is further configured to:

parse a conversation of the user or the another person into speech elements,

analyze the speech elements based on the previously determined user data, and

determine the desirable event or action further based on the analyzed speech elements.

7. The eyeglasses of claim 1 further comprising a stereo camera pair configured to detect depth information regarding the surrounding environment.

8. The eyeglasses of claim 1 further comprising a wide angle camera configured to detect image data within a 120 degree field of view.

9. A method for providing continuous social and environmental awareness by eyeglasses comprising:

detecting, via a camera, a GPS unit or an IMU, inertial measurement data corresponding to a positioning, velocity, or acceleration of the eyeglasses, location data corresponding to a location of the eyeglasses or image data corresponding to a surrounding environment of the eyeglasses;

storing, in a memory, object data regarding previously determined objects and previously determined user data regarding a user;

recognizing, by a processor, an object in the surrounding environment by analyzing the image data based on the stored object data and at least one of the inertial measurement data or the location data;

determining, by the processor;

a desirable event or action based on the recognized object, the previously determined user data, and a current time or day,

a destination based on the determined desirable event or action,

a navigation path for navigating the eyeglasses to the destination based on the determined destination, the image data, and at least one of the inertial measurement data or the location data, or

output data based on the determined navigation path;

providing, via a speaker or a vibration unit, audio or haptic information to the user based on at least one of the recognized object, the determined desirable event or action, or the navigation path.

10. The method of claim 9 wherein:

detecting data includes detecting, by the camera, image data corresponding to the surrounding environment of the eyeglasses,

determining divergence data includes determining divergence data between the object data and the image data, and

providing audio or haptic information to the user further includes providing audio or haptic information based on the divergence data.

11. The method of claim 9 wherein providing audio or haptic information to the user includes providing stereo haptic information.

12. The method of claim 9 further including transmitting, via an antenna, the image data and at least one of the inertial measurement data or the location data to a remote device, where the processor is located on the remote device.

13. The method of claim 9 further comprising storing, in the memory, map data wherein determining the navigation path includes determining the navigation path based on the image data, the map data, and at least one of the location data or the inertial measurement data.

14. The method of claim 9 wherein storing object data includes storing object data in a remote database that can be accessed by other intelligent devices.

15. Eyeglasses to be worn by a user, comprising:

a right lens;

a left lens;

an inertial measurement unit (IMU) sensor coupled to the eyeglasses and configured to detect inertial measurement data corresponding to a positioning, velocity, or acceleration of the eyeglasses;

a global positioning system (GPS) unit coupled to the eyeglasses and configured to detect location data corresponding to a location of the eyeglasses;

at least one camera positioned on at least one of the right lens or the left lens and coupled to the eyeglasses, the at least one camera configured to detect image data corresponding to a surrounding environment of the eyeglasses;

a memory configured to store object data regarding previously determined objects and previously determined user data associated with the user;

an antenna configured to transmit the image data, the inertial measurement data, the location data and the object data to a remote processor and to receive processed data from the remote processor, the remote processor configured to;

recognize an object in the surrounding environment by analyzing the image data based on the stored object data and at least one of the inertial measurement data or the location data,

determine a desirable event or action based on the recognized object, the previously determined user data, and a current time or day,

determine a destination based on the determined desirable event or action,

determine a navigation path for navigating the eyeglasses to the destination based on the determined destination, the image data, and at least one of the inertial measurement data or the location data, and

determine output data based on the determined navigation path; and

a speaker configured to provide audio information to the user based on at least one of the recognized object, determined desirable event or action, or navigation path.

16. The eyeglasses of claim 15 further comprising a vibratory motor coupled to the eyeglasses configured to provide first haptic information to the user based on the determined output data.

17. The eyeglasses of claim 15 wherein the memory is configured to store map data and the remote processor is configured to determine the navigation path based on the image data, the map data, and at least one of the location data or the inertial measurement data.

18. The eyeglasses of claim 15 further comprising a first connector adapted to electrically attach the left lens to the eyeglasses and a second connector adapted to electrically attach the right lens to the eyeglasses.

19. The eyeglasses of claim 15, further comprising a microphone configured to detect a speech of the user or another person, wherein the remote processor is further configured to:

parse a conversation of the user or the another person into speech elements,

analyze the speech elements based on the previously determined user data, and

determine the desirable event or action further based on the analyzed speech elements.

20. The eyeglasses of claim 15 further comprising a stereo camera pair configured to detect depth information regarding the surrounding environment.