Smart necklace with stereo vision and onboard processing

• US 10,360,907 B2
• Filed: 10/13/2016
• Issued: 07/23/2019
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Dayal, Rajiv
• Primary Examiner(s): Lin, Abby Y
• Assistant Examiner(s): Wallace, Donald J
• Application Number:
  US15/292,987
• Publication Number:
  US 20170032787A1
• Time in Patent Office: 1,013 Days
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/1609 Neck
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/50 Control means thereof
• CPC Class Codes: A61H 2201/5012 using the internet
• CPC Class Codes: A61H 2201/5043 Displays
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5058 Sensors or detectors
• CPC Class Codes: A61H 2201/5064 Position sensors
• CPC Class Codes: A61H 2201/5082 Temperature sensors
• CPC Class Codes: A61H 2201/5084 Acceleration sensors
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 2230/06 Heartbeat rate
• CPC Class Codes: A61H 2230/08 Other bio-electrical signals
• CPC Class Codes: A61H 2230/202 Glucose
• CPC Class Codes: A61H 2230/25 Blood flowrate, e.g. by Doppler effect
• CPC Class Codes: A61H 2230/30 Blood pressure A61H2230/04 takes precedence
• CPC Class Codes: A61H 2230/50 Temperature
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/206 : specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3629 : Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3652 : Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
• CPC Class Codes: G06F 1/163 : Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 21/14 : Toxic gas alarms G08B21/16 takes precedence
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 : Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 : of application context
• CPC Class Codes: H04M 1/05 : specially adapted for use on head, throat or breast
• CPC Class Codes: H04M 1/724094 : Interfacing with a device worn on the user's body to provide access to telephonic functionalities, e.g. accepting a call, reading or composing a message
• CPC Class Codes: H04M 1/72412 : using two-way short-range wireless interfaces
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2213/001 : Constructional or mechanical details

Abstract

A wearable computing device includes an input/output port for communicating with an external mobile device, a microphone for receiving speech data, a speaker for outputting audio feedback data, and a mobile processor. The mobile processor is designed to receive detected speech data corresponding to a request to open an application on the external mobile device, transmit the request to the external mobile device, and receive a description of content within the application from the external mobile device. The mobile processor is also designed to transmit the description to the speaker to be output, receive detected speech data corresponding to a request for the external mobile device to perform an action within the application, and transmit the request to the external mobile device. The mobile processor is also designed to receive results of the action from the external mobile device and transmit the results to the speaker to be output.

Claims

1. A wearable computing device comprising:

a U-shaped housing configured to be worn around a neck of a user;

an input/output port attached to the U-shaped housing and configured to communicate with an external mobile device;

a microphone attached to the U-shaped housing and configured to receive speech data corresponding to instructions by the user;

a speaker attached to the U-shaped housing and configured to output audio feedback data to the user; and

a mobile processor positioned within the U-shaped housing, electrically coupled to the input/output port, the microphone, and the speaker, and configured to;

receive a first speech data from the microphone corresponding to a request to open an application on the external mobile device,

transmit the request to open the application to the external mobile device via the input/output port,

receive a description of content within the application from the external mobile device via the input/output port,

transmit the description of the content within the application to the speaker to be output,

receive a second speech data from the microphone corresponding to a request for the external mobile device to perform an action within the application,

transmit the request to perform the action within the application to the external mobile device via the input/output port,

receive a description of results of the action performed within the application from the external mobile device via the input/output port,

transmit the results of the action performed within the application to the speaker to be output,

receive a third speech data from the microphone corresponding to a request for the wearable computing device to perform a local action,

transmit a request to the external mobile device for sensor data via the input/output port,

receive the sensor data from the external mobile device via the input/output port,

perform the local action using the sensor data as input, and

transmit results of the local action to the speaker to be output.

2. The wearable computing device of claim 1 wherein:

the application is an internet browser;

the content within the internet browser includes text with selectable hyperlinks;

the action within the internet browser includes a selection of one of the selectable hyperlinks; and

the description of the results of the selection of the one of the selectable hyperlinks includes text from a webpage associated with the one of the selectable hyperlinks.

3. The wearable computing device of claim 2 wherein the mobile processor is further configured to:

receive a fourth speech data corresponding to a request to visit a specific webpage from the microphone;

transmit the request to visit the specific webpage to the external mobile device via the input/output port;

receive a description of the specific webpage from the external mobile device via the input/output port; and

transmit the description of the specific webpage to the speaker to be output.

4. The wearable computing device of claim 1 wherein

the application is a calendar application;

the content within the calendar application includes at least one calendar entry for a particular day;

the action within the calendar application includes opening one of the at least one calendar entry for the particular day; and

the description of the results of the opened calendar entry includes a description of the one of the at least one calendar entry.

5. The wearable computing device of claim 1 wherein

the application is a calendar application;

the content within the calendar application includes at least one calendar entry for a particular day;

the action within the calendar application includes adding a new calendar entry for the particular day; and

the description of the results of the opened calendar entry includes information indicating that the new calendar entry has been added.

6. The wearable computing device of claim 1 further comprising a sensor attached to the U-shaped housing and configured to detect detected data corresponding to an environment of the wearable computing device, wherein the mobile processor is further configured to:

receive a request for the detected data from the external mobile device via the input/output port, the detected data being used to perform the action; and

transmit the detected data to the external mobile device via the input/output port in response to receiving the request for the detected data.

7. The wearable computing device of claim 1 wherein the external mobile device includes at least one of another wearable computing device, a mobile telephone, a tablet, or a laptop computer.

8. A wearable computing device comprising:

a U-shaped housing configured to be worn around a neck of a user;

an input/output port attached to the U-shaped housing and configured to communicate with an external mobile device;

a sensor attached to the U-shaped housing and configured to detect detected data corresponding to an environment of the wearable computing device;

a microphone attached to the U-shaped housing and configured to receive input data corresponding to a request for the external mobile device to perform an action;

an output device attached to the U-shaped housing and configured to output data to the user; and

a mobile processor positioned within the U-shaped housing, electrically coupled to the input/output port, the sensor, the microphone, and the output device, and configured to;

receive the input data that corresponds to the request for the external mobile device to perform the action,

transmit the input data to the external mobile device via the input/output port,

receive a request for the detected data from the external mobile device via the input/output port, the detected data being used to perform the action,

transmit the detected data to the external mobile device via the input/output port in response to receiving the request for the detected data,

receive output data generated by the external mobile device via the input/output port,

cause the output device to output the output data,

receive a speech data from the microphone corresponding to a request for the wearable computing device to perform a local action,

transmit a request to the external mobile device for sensor data via the input/output port,

receive the sensor data from the external mobile device via the input/output port,

perform the local action using the sensor data as input, and

transmit results of the local action to the output device to be output.

9. The wearable computing device of claim 8 wherein the microphone is further configured to receive preference input data indicating a preference to output feedback data via the output device of the wearable computing device or a display of the external mobile device and wherein the mobile processor is further configured to transmit a message to the external mobile device via the input/output port instructing the external mobile device to:

transmit the feedback data to the wearable computing device when the preference input data indicates the preference to output the feedback data via the output device of the wearable computing device; and

output the feedback data via the display of the external mobile device when the preference input data indicates the preference to output the feedback data via the display of the external mobile device.

10. The wearable computing device of claim 8 wherein:

the action includes video communication with a third device;

the sensor is a camera configured to detect image data;

the image data detected by the camera of the wearable computing device is transmitted to the third device via the input/output port and the external mobile device; and

the output data includes at least one of image data or audio data detected by the third device.

11. The wearable computing device of claim 10 wherein the microphone is further configured to detect speech of the user and the speech of the user is transmitted to the third device along with the image data via the input/output port and the external mobile device.

12. The wearable computing device of claim 8 wherein:

the sensor is a biological sensor configured to detect health data corresponding to a health metric of the user;

the action includes analyzing the detected health data; and

the output data includes an analysis of the detected health data.

13. The wearable computing device of claim 12 wherein the biological sensor includes at least one of a blood pressure sensor, a blood flow sensor, a pulse sensor, a temperature sensor, a breathalyzer sensor, a blood alcohol sensor, or a glucose level sensor.

14. The wearable computing device of claim 8 wherein:

the microphone is further configured to detect audio data corresponding to speech of the user;

the action includes verification of an identity of the user;

the detected audio data is used by the external mobile device to verify the identity of the user; and

the output data indicates that the identity of the user has been verified.

15. A method for controlling an external mobile device by a wearable computing device comprising:

receiving, from a microphone, a first speech data corresponding to a request to open an application on the external mobile device;

transmitting, via an input/output port, the request to open the application to the external mobile device;

receiving, via the input/output port, a description of content within the application from the external mobile device;

transmitting, by a mobile processor, the content within the application to a speaker to be output;

receiving, from the microphone, a second speech data corresponding to a request for the external mobile device to perform an action within the application;

transmitting, via the input/output port, the request to perform the action within the application to the external mobile device;

receiving, via the input/output port, a description of results of the action performed within the application from the external mobile device;

instructing, by the mobile processor, the speaker to output the results of the action performed within the application;

receiving, from the microphone, a third speech data corresponding to a request for the wearable computing device to perform a local action;

transmitting, via the input/output port, a request to the external mobile device or sensor data;

receiving, via the input/output port, the sensor data from the external mobile device to me;

performing, by the mobile processor, the local action using the sensor data as input; and

transmitting, by the mobile processor, results of the local action to the speaker to be output.

16. The method of claim 15 wherein:

the application is an internet browser,

the content within the internet browser includes text with selectable hyperlinks,

the action within the internet browser includes a selection of one of the selectable hyperlinks, and

the description of the results of the selection of the one of the selectable hyperlinks includes text from a webpage associated with the one of the selectable hyperlinks.

17. The method of claim 16 further comprising:

receiving, from the microphone, a fourth speech data corresponding to a request to visit a specific webpage;

transmitting, via the input/output port, the request to visit the specific webpage to the external mobile device;

receiving, via the input/output port, a description of the specific webpage from the external mobile device; and

transmitting, by the mobile processor, the description of the specific webpage to the speaker to be output.

18. The method of claim 15 further comprising:

detecting, by a sensor attached to the U-shaped housing, detected data corresponding to an environment of the wearable computing device;

receiving, via the input/output port, a request for the detected data from the external mobile device, the detected data being used to perform the action; and

transmitting, via the input/output port, the detected data to the external mobile device in response to receiving the request for the detected data.