Cross Monitoring Method and System Based on Voiceprint Recognition and Location Tracking

• US 20120249328A1
• Filed: 09/25/2010
• Published: 10/04/2012
• Est. Priority Date: 10/10/2009
• Status: Active Grant

• Current Assignee: Dianyuan Xiong
• Original Assignee: Dianyuan Xiong
• Inventors: Xiong, Dianyuan
• Granted Patent: US 9,218,814 B2
• Time in Patent Office: Days
• US Class Current: 340/541
• CPC Class Codes: G08B 21/22 responsive to presence or absence of persons
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

The invention discloses a cross monitoring method and system based on voiceprint recognition and location tracking, belonging to the field of monitoring systems. The invention collects personal information and voice samples of a monitored individual and constructs a voiceprint model thereof, and collects location information thereof and voiceprint information through a positioning communication device and/or wireless communication network service, which are compared with the stored information in a repository; output a complete report reflecting the human body movement trace and actual non-compliance with the restrictions for a supervisory staff or other person of authority to inspect and evaluate the performance of the monitored individual. The supervisory staff can request and evaluate the statistical analysis report concerning the activity performance of the monitored individual, observe the movement trace and the state thereof, and/or adjust the monitoring scheme thereof according to the performance. The invention realizes the remote location tracking of the monitored individual, also realizes non-contact biological identity recognition of the monitored individual and satisfies the need of remote and non-contact monitoring of the monitored individual. The invention can be widely applied to the non-contact electronic monitoring and identity recognition field.

Claims

1. A cross-monitoring method based on voiceprint recognition and location tracking, the method comprising:

collecting personal information and voice samples of an individual to register a user ID for the individual and build a voiceprint model for the individual;

designating the individual a positioning communication device for tracking the individual;

configuring a monitoring scheme and an alarming scheme;

selecting a unique network ID fully representing the positioning communication device;

establishing a unique one-to-one corresponding between the network ID and the user ID to bind the device and the individual together;

for the positioning communication device, executing repeatedly a location-tracking step in accordance with the monitoring scheme and the alarming scheme, wherein the location-tracking result is processed and stored in a repository;

for the individual, executing repeatedly a voice-tracking step based on voiceprint recognition in accordance with the monitoring scheme and the alarming scheme, wherein the voice-tracking result is processed and stored in the repository; and

for the individual and the positioning communication device designated for tracking the individual, conducting a cross statistical analysis on the location-tracking and voice-tracking result records generated within the same time period to compile an overall tracking report on the individual'"'"'s performance for the specified time period in response to a report request or periodically, wherein the overall tracking report eventually reflects the individual'"'"'s movement trace and actual status of non-compliance with the movement restrictions imposed on the individual so that a supervisory staff can review the performance and movement trace of the individual and/or adjust the monitoring and alarming schemes for the individual.

2. The cross-monitoring method of claim 1, wherein the monitoring scheme comprises some or all of the following items of information:

a time schedule and frequency of recurrence of the location-tracking step;

one or more choices on whether the monitoring system requests a location-defining information actively or receives a location-defining information passively;

a set of time and location related movement restrictions imposed on the individual;

a time schedule and frequency of recurrence of the voice-tracking step;

one or more choices on whether the individual initiates a voice call actively or answers a voice call passively for voiceprint recognition;

one or more choices on mode and content of system prompt or request versus the individual'"'"'s oral answer;

one or more demands on parallel or interactively-linked execution of the location-tracking and voice-tracking steps; and

a time schedule to conduct cross statistical analysis on the monitoring results to compile a tracking report;

wherein the alarming scheme comprises alarm-triggering conditions, alarm-delivering means, and alarm-receiving destinations;

wherein if personalized monitoring and alarming schemes are not configured for the individual, then default monitoring and alarming schemes are used for the individual.

3. The cross-monitoring method of claim 1, wherein in accordance with the monitoring scheme and the alarming scheme, the location-tracking step and the voice-tracking step can be executed in parallel mode or in interactively-linked mode.

4. The cross-monitoring method of claim 1, wherein the location-tracking step further comprises:

acquiring the network ID and geolocation of the positioning communication device as well as the positioning timestamp through location-defining technological methods;

querying the individual'"'"'s user ID that is uniquely corresponding to the network ID according to the binding relationship between the user ID and network ID;

querying the time and location related movement restrictions imposed on the individual of the user ID;

determining whether the geolocation is compliant with the movement restrictions imposed on the individual, wherein an alarm message is optionally sent to its destinations in the specified alarm-delivering means if the non-compliance with the restrictions triggers the alarm condition; and

storing the location-tracking result in the repository where the location-tracking result record is associated with the user ID;

wherein the location-tracking result comprises the network ID, geolocation, positioning timestamp and/or the status of alarming and non-compliance with the movement restrictions;

wherein the location-defining technological methods comprises satellite, positioning, base station positioning over wireless mobile communication network, hybrid positioning of satellites and mobile base stations, and/or wireless Wi-Fi positioning;

wherein the location-defining technological methods are enabled via the positioning communication device and/or wireless communication network service.

5. The cross-monitoring method of claim 1, wherein the voice-tracking step further comprises:

establishing a man-machine interactive voice dialogue;

acquiring the network ID of the positioning communication device held by the individual;

recording the individual'"'"'s voice input in response to the system prompt or request;

executing voiceprint recognition process on the recorded voice input to verify the individual'"'"'s biometric identity;

optionally adding an extra step to execute speech recognition process on the recorded voice input to determine whether the individual'"'"'s voice input matches the system prompt or request in the man-machine interactive voice dialogue in order to verify that the voice input originated from the live speech of a person who is present at the location of the positioning communication device;

utilizing the network ID and voiceprint recognition result and/or speech recognition result to determine whether the individual on-site is holding the positioning communication device designated for tracking the individual, and determine the status of the individual'"'"'s non-compliance with the restriction that the individual shall stay together with his or her tracking device;

optionally sending an alarm message to its destinations in the specified alarm-delivering means if the non-compliance with the restriction triggers the alarm condition; and

storing the voice-tracking result in the repository where the voice-tracking result record is associated with the user ID;

wherein the voice-tracking result further comprises the network ID, monitoring timestamp, the status of alarming and non-compliance with the restriction, the voiceprint recognition result and/or speech content recognition result.

6. The cross-monitoring method of claim 4, wherein the execution order between sending the alarm-message and storing the tracking result doesn'"'"'t matter in the location-tracking and voice-tracking steps.

7. The cross-monitoring method of claim 1, wherein the voiceprint recognition is classified as two phases:

voiceprint enrollment and voiceprint verification;

wherein the voiceprint enrollment phase comprises;

extracting voiceprint feature, wherein digital speech signal processing is performed on the individual'"'"'s voice samples to extract a voiceprint feature sequence that characterizes the physiological and behavioral characteristics of the individual'"'"'s speaking; and

building voiceprint model, wherein the voiceprint feature sequence extracted from the voice samples is utilized to build the voiceprint model that characterizes the characteristics of the individual'"'"'s speaking;

wherein the voiceprint verification phase comprises;

extracting voiceprint feature, wherein digital speech signal processing is performed on the individual'"'"'s voice input to extract a voiceprint feature sequence that characterizes the physiological and behavioral characteristics of the individual'"'"'s speaking; and

matching voiceprint pattern, wherein the voiceprint feature sequence extracted from the voice input is utilized to compute its similarity score against the voiceprint model of the individual so as to authenticate the individual.

8. A cross-monitoring system based on the voiceprint recognition and location tracking, involving a monitored individual and a supervisory staff, wherein the individual is monitored automatically by the cross-monitoring system and supervised by the supervisory staff, and the supervisory staff manages the individual and the operation of the cross-monitoring system, the cross-monitoring system comprising:

setting up a positioning communication device having a unique network ID, wherein the positioning communication device is attached onto a body surface of the individual or carried closely by the individual as a personal portable device for location-defining, data communication and voice communication processes, wherein the positioning communication device of different network ID is designated for different individual;

setting up an application server comprising a processing unit, a storage unit and a communication unit to provide voice communication, data communication, information processing and storage, system management, voiceprint recognition and/or speech content recognition; and

setting up an administration operation terminal as a client, wherein the administration operation terminal combines together with the application server to form the service architecture of Browser-Server or Client-Server;

wherein the supervisory staff may operate the monitoring system and manage the individual via the administration operation terminal which is connected to the application server;

wherein, through a first link network, the positioning communication device and application server are joined to execute the location-tracking and voice-tracking steps in the cross-monitoring method of claims 1, 3, 4 and 5 for the individual and the positioning communication device designated for tracking the individual, wherein in response to a cross-monitoring request, the application server further conducts a cross statistical analysis on the location-tracking and voice-tracking results generated within the same time period to compile an overall tracking report on the movement trace and performance of the individual for the supervisory staffs review;

wherein, through a second link network, the administration operation terminal and application server are joined to execute any of the following work tasks said in the cross-monitoring method of claim 1;

managing an individual'"'"'s personal information, designating a positioning communication device specifically for the individual, configuring the monitoring scheme, configuring the alarming scheme comprising the alarm-triggering conditions, alarm-delivering means, and alarm-message destinations, sending notification and command messages, receiving event notification, and observing the geolocations and movement trace of the individual and/or outputting the monitoring report by cross statistical analysis;

wherein the first link network is used for voice communication and data communication involving the individual;

wherein the second link network is used for data communication and/or voice communication involving the supervisory staff.

9. The cross-monitoring system of claim 8,

wherein the positioning communication device, being involved in the location-tracking process, acquires and transmits the positioning information, and/or receives a notification or command message from the application server;

wherein the positioning communication device, being involved in the voice-tracking process, plays back the system prompt or request, collects and transmits the voice input of the individual'"'"'s oral response to the system prompt or request during a man-machine interactive voice dialogue process;

wherein the application server is either a server comprised of a single physical computer, or a virtual server comprised of a plurality of physical computers working together harmonically through network connections;

wherein the application server comprises one or more service subsystems, and may further optionally provide such auxiliary services as text-to-speech synthesis, and geographic information services;

if the application server is a virtual server comprising a plurality of physical computers, the plurality of physical computers can be located centrally in the same local area network, or be also deployed in geographically distributed networks over network connections, they are controlled and coordinated by the system management service;

wherein the first link network is involved in the location-tracking and/or voice-tracking process;

wherein the second link network is involved in the system control and management, transmits the geolocations, movement traces, the status of non-compliance with the restrictions, and/or the statistical report on the individual.

10. The cross-monitoring system of claim 9,

wherein the man-machine interactive voice dialogue involves the individual, the positioning communication device and the application server, and allows a voice call to be initiated in two modes from the point of view of the individual:

active call-making mode and passive call-taking mode;

in the active call-making mode, the individual actively initiates a voice call to the application server via the positioning communication device bound for the individual, the application server then accepts the inbound voice call, detects and identifies the network ID of the positioning communication device;

in the passive call-taking mode, the application server queries the network ID uniquely corresponding to the individual'"'"'s user ID, makes an outbound voice call to the positioning communication device of the network ID;

then, the individual holding the positioning communication device accepts passively the voice call from the application server.

11. The cross-monitoring system of claim 8, wherein the application server, based on different positioning technological methods, obtains the geolocation information of the positioning communication device in one or more of the following ways:

the positioning communication device acquires its latest geolocation information, further transmits the geolocation information to the application server over the first link network;

or the positioning communication device acquires its latest signal data of wireless communication which can be used to derive its geolocation information indirectly, further transmits the signal data to the application server over the first link network;

then, the application server receives the signal data and obtains indirectly the geolocation information of the positioning communication device from the signal data;

or according to the network ID of the positioning communication device, the application server retrieves indirectly the geolocation information of the positioning communication device from the third-party location-defining server which can determine and store the geolocation information of the positioning communication device by a single or hybrid of the positioning technological methods.

12. The cross-monitoring method of claim 2, wherein the interactively-linked execution of the location-tracking and voice-tracking steps is characterized as follows:

while executing one of the location-tracking and voice-tracking steps, an alarm event demanding the interactively-linked execution of the two tracking steps is triggered to cause the other step to be executed once subsequently.

13. The cross-monitoring method of claim 1, wherein the movement restriction in the monitoring scheme is specified geographically in term of location range and may possibly be associated with time limit, wherein the movement restriction comprises one or more geolocation points, or even one or more electronic fences with a closed boundary delineated in the electronic map.

14. The cross-monitoring method of claim 1, wherein the alarm-triggering condition in the alarming scheme comprises any one of the following events occurring in the location-tracking step for the positioning communication device:

arrival or departure from a specified location, entering or leaving the region of an electronic fence, and staying inside or outside the region of an electronic fence for longer than a specified time limit.

15. The cross-monitoring method of claim 1, wherein the alarm-triggering condition in the alarming scheme comprises any one of the following events occurring in the voice-tracking step for a positioning communication device:

reaching a specified number limit of consecutive failures that the positioning communication device is not available for the outbound voice call initiated by the system;

reaching a specified number limit of consecutive failures that no one at the location of the positioning communication device answers the outbound voice call initiated by the system;

an individual'"'"'s failure to actively make a voice call to the application server within the specified time range in response to the monitoring request;

audio-playback fraud (that is, speech content recognition helps determine that voice response is not from the live speech of a person at the location of the positioning communication device);

decoupling physically between the individual and the device designated for tracking him or her (that is, voiceprint recognition and/or speech content recognition helps determine that the individual doesn'"'"'t stay together with the specified positioning communication device for tracking the individual).

16. The cross-monitoring method of claim 1, wherein the network ID of the positioning communication device comprises a single device attribute or combination of device attributes uniquely representing the positioning communication device;

or the network ID comprises a unique ID which one-to-one corresponds to a single device attribute or combination of device attributes uniquely representing the positioning communication device.

17. The cross-monitoring method of claim 1, wherein the network ID of the positioning communication device is acquired through different and unique device attributes of the positioning communication device while executing the location-tracking step and voice-tracking step.

18. The cross-monitoring method of claim 5, wherein the execution order between sending the alarm-message and storing the tracking result doesn'"'"'t matter in the location-tracking and voice-tracking steps.

19. The cross-monitoring method of claim 3, wherein the interactively-linked execution of the location-tracking and voice-tracking steps is characterized as follows:

while executing one of the location-tracking and voice-tracking steps, an alarm event demanding the interactively-linked execution of the two tracking steps is triggered to cause the other step to be executed once subsequently.