Alias resolving intelligent assistant computing device

• US 11,004,446 B2
• Filed: 06/30/2017
• Issued: 05/11/2021
• Est. Priority Date: 02/14/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Finkelstein, Erich-Soren, Fung, Han Yee Mimi, Solomon, Oz
• Primary Examiner(s): Geib, Benjamin P
• Application Number:
  US15/640,201
• Publication Number:
  US 20180232645A1
• Time in Patent Office: 1,411 Days
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/05 Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves impedance plethysmography A61B5/0295; measuring movement of the entire body or parts thereof A61B5/11; detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof A61B5/24
• CPC Class Codes: A61B 5/0507 using microwaves or terahertz waves
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/117 Identification of persons methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G01S 11/14 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 13/38 wherein more than one modulation frequency is used
• CPC Class Codes: G01S 13/726 Multiple target tracking
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/888 through wall detection
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 5/28 by co-ordinating position lines of different shape, e.g. hyperbolic, circular, elliptical or radial
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/324 by lowering clock frequency
• CPC Class Codes: G06F 1/329 by task scheduling
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 18/00 : Pattern recognition
• CPC Class Codes: G06F 18/251 : of input or preprocessed data
• CPC Class Codes: G06F 18/28 : Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 18/29 : Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06F 18/41 : Interactive pattern learning with a human teacher
• CPC Class Codes: G06F 21/00 : Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/32 : using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/35 : communicating wirelessly
• CPC Class Codes: G06F 2203/0381 : Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2221/2111 : Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2117 : User registration
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 : Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/211 : Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/35 : Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 3/044 : Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 5/025 : Extracting rules from data
• CPC Class Codes: G06N 5/047 : Pattern matching networks; Rete networks
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/20101 : Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/30196 : Human being; Person
• CPC Class Codes: G06T 2207/30201 : Face
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: G06V 10/94 : Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/40 : in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/653 : by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 2201/05 : Recognition of patterns representing particular kinds of hidden objects, e.g. weapons, explosives, drugs
• CPC Class Codes: G06V 30/274 : Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/166 : using acquisition arrangements
• CPC Class Codes: G06V 40/167 : using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G06V 40/173 : face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 : Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G07C 9/28 : the pass enabling tracking or indicating presence
• CPC Class Codes: G07C 9/32 : in combination with an identity check
• CPC Class Codes: G08B 13/1427 : with transmitter-receiver for distance detection
• CPC Class Codes: G08B 29/186 : Fuzzy logic; neural networks
• CPC Class Codes: G10L 15/00 : Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 : Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 : Training
• CPC Class Codes: G10L 15/08 : Speech classification or search
• CPC Class Codes: G10L 15/18 : using natural language modelling
• CPC Class Codes: G10L 15/1815 : Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 : Parsing for meaning understanding
• CPC Class Codes: G10L 15/19 : Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 : Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 : Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 : Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/32 : Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 17/00 : Speaker identification or verification techniques
• CPC Class Codes: G10L 17/04 : Training, enrolment or model building
• CPC Class Codes: G10L 17/06 : Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/08 : Use of distortion metrics or a particular distance between probe pattern and reference templates
• CPC Class Codes: G10L 17/22 : Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0635 : updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/088 : Word spotting
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 : Feedback of the input speech
• CPC Class Codes: G10L 2015/228 : of application context
• CPC Class Codes: G10L 25/51 : for comparison or discrimination
• CPC Class Codes: H04L 51/02 : using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 63/102 : Entity profiles
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04N 21/231 : Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/42203 : sound input device, e.g. microphone
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44222 : Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44224 : Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 : Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253
• CPC Class Codes: H04R 1/406 : microphones
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services
• CPC Class Codes: H04W 4/33 : for indoor environments, e.g. buildings
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Intelligent assistant systems, methods and computing devices are disclosed for resolving alias identifiers. A method comprises receiving and parsing data comprising a current user input that includes an alias identifier. The data and/or other sensor data are analyzed to identify the user. Based at least on identifying the user and recognizing the alias identifier, usage pattern data comprising at least one previous user input that includes the alias identifier and corresponding context information is accessed. The usage pattern data is used to resolve the alias identifier to mean the alias identifier in an alias record of a known entity. Based at least on resolving the alias identifier, an output device is controlled to one or more of generate a message and perform an action with respect to the known entity.

Claims

1. An intelligent digital assistant system, comprising:

at least one sensor in an environment;

a logic processor; and

a storage device holding instructions executable by the logic processor to;

via the at least one sensor, receive data comprising a current input provided by a user that includes an alias identifier;

parse the data to recognize the alias identifier;

computer-analyze one or more of the data and other sensor data received from the environment to identify the user as providing the current input;

based at least on identifying the user and recognizing the alias identifier in the data, access usage pattern data of the user comprising at least one previous input provided by the user that includes the alias identifier and corresponding context information;

using at least the previous input and the context information, resolve the alias identifier in the current input to mean the alias identifier in an alias record of a known entity;

based at least on resolving the alias identifier in the current input, control an output device to one or more of generate a message and perform an action with respect to the known entity;

from a new user, receive a new user input that includes the alias identifier;

based at least on usage pattern data of the new user, resolve the alias identifier in the new user input to mean the alias identifier in the alias record of the known entity; and

based at least on resolving the alias identifier in the new user input, classify the new user into a category that relates to one or more aspects of the alias identifier.

2. The intelligent digital assistant system of claim 1, wherein the at least one sensor comprises a microphone, the data comprises audio data and the current input comprises a spoken utterance of the user.

3. The intelligent digital assistant system of claim 2, wherein the at least one sensor further comprises a camera, and the data further comprises image data captured with the spoken utterance of the user.

4. The intelligent digital assistant system of claim 1, wherein the at least one sensor comprises a camera, the data comprises image data and the current input comprises a gesture of the user.

5. The intelligent digital assistant system of claim 1, wherein the context information is captured contemporaneously with the previous user input provided by the user.

6. The intelligent digital assistant system of claim 1, wherein the context information comprises one or more of audio data and image data.

7. The intelligent digital assistant system of claim 1, wherein the instructions are executable to:

from another user receive alias record input comprising the alias identifier; and

populate the alias record with the alias record input from the other user.

8. The intelligent digital assistant system of claim 7, wherein the alias record input comprises an utterance spoken by the other user.

9. The intelligent digital assistant system of claim 1, wherein the instructions are executable to:

identify a link between the alias identifier and other information of the user that relates to the one or more aspects of the alias identifier; and

use the link to classify the new user into the category that relates to the one or more aspects of the alias identifier.

10. At a computing device, a method for resolving an alias identifier, the method comprising:

via at least one sensor in an environment, receiving data comprising a current input provided by a user that includes the alias identifier;

parsing the data to recognize the alias identifier;

computer-analyzing one or more of the data and other sensor data received from the environment to identify the user as providing the current input;

based at least on identifying the user and recognizing the alias identifier in the data, accessing usage pattern data of the user comprising at least one previous input provided by the user that includes the alias identifier and corresponding context information;

using at least the previous input and the context information, resolving the alias identifier in the current input to mean the alias identifier in an alias record of a known entity;

based at least on resolving the alias identifier in the current input, controlling an output device to one or more of generate a message and perform an action with respect to the known entity;

from a new user, receiving a new user input that includes the alias identifier;

based at least on usage pattern data of the new user, resolving the alias identifier in the new user input to mean the alias identifier in the alias record of the known entity; and

based at least on resolving the alias identifier in the new user input, classifying the new user into a category that relates to one or more aspects of the alias identifier.

11. The method of claim 10, wherein the at least one sensor comprises a microphone, the data comprises audio data and the current input comprises a spoken utterance of the user.

12. The method of claim 11, wherein the at least one sensor further comprises a camera, and the data further comprises image data captured with the spoken utterance of the user.

13. The method of claim 10, wherein the at least one sensor comprises a camera, the data comprises image data and the current input comprises a gesture of the user.

14. The method of claim 10, further comprising capturing the context information contemporaneously with the previous user input provided by the user.

15. The method of claim 10, wherein the context information comprises one or more of audio data and image data.

16. The method of claim 10, further comprising:

receiving from another user alias record input comprising the alias identifier; and

populating the alias record with the alias record input from the other user.

17. The method of claim 16, wherein the alias record input comprises an utterance spoken by the other user.

20. The method of claim 10, further comprising:

identifying a link between the alias identifier and other information of the user that relates to the one or more aspects of the alias identifier; and

using the link to classify the new user into the category that relates to the one or more aspects of the alias identifier.

18. A computing device, comprising:

a plurality of sensors;

a logic processor; and

a storage device holding instructions executable by the logic processor to;

via the at least one of the sensors, receive data comprising a current input provided by a user that includes an alias identifier;

convert the data into parsable data;

parse the data to recognize the alias identifier;

computer-analyze one or more of the data and other sensor data received from an environment of the user to identify the user as providing the current input;

based at least on identifying the user and recognizing the alias identifier in the data, access usage pattern data of the user comprising at least one previous input provided by the user that includes the alias identifier and corresponding context information;

access an alias record of a known entity;

using at least the previous input and the context information, resolve the alias identifier in the current input to mean the alias identifier in the alias record of the known entity;

based at least on resolving the alias identifier in the current input, control an output device to one or more of generate a message and perform an action with respect to the known entity;

from a new user, receive a new user input that includes the alias identifier;

based at least on usage pattern data of the new user, resolve the alias identifier in the new user input to mean the alias identifier in the alias record of the known entity; and

based at least on resolving the alias identifier in the new user input, classify the new user into a category that relates to one or more aspects of the alias identifier.

19. The computing device of claim 18, wherein the context information is captured contemporaneously with the previous user input provided by the user.