Ad hoc one-time pairing of remote devices using online audio fingerprinting

• US 10,348,724 B2
• Filed: 04/07/2015
• Issued: 07/09/2019
• Est. Priority Date: 04/07/2014
• Status: Active Grant

• Current Assignee: Barco NV
• Original Assignee: Barco NV
• Inventors: Brands, Johannes, Herreman, Christophe
• Primary Examiner(s): Jamshidi, Ghodrat
• Application Number:
  US15/302,668
• Publication Number:
  US 20170034160A1
• Time in Patent Office: 1,554 Days
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04K 1/00 Secret communication
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 2209/76 Proxy, i.e. using intermediary entity to perform cryptographic operations network architectures or network communication protocols using hop-by-hop encryption H04L63/0464
• CPC Class Codes: H04L 63/06 for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 63/18 using different networks or channels, e.g. using out of band channels cryptographic mechanisms or cryptographic arrangements for key distribution involving distinctive intermediate devices or communication paths H04L9/0827; cryptographic mechanisms or cryptographic arrangements for authentication using a plurality of channels H04L9/3215
• CPC Class Codes: H04L 9/0866 involving user or device identifiers, e.g. serial number, physical or biometrical information, DNA, hand-signature or measurable physical characteristics
• CPC Class Codes: H04L 9/3228 One-time or temporary data, i.e. information which is sent for every authentication or authorization, e.g. one-time-password, one-time-token or one-time-key
• CPC Class Codes: H04L 9/3231 Biological data, e.g. fingerprint, voice or retina network architectures or network communication protocols for supporting authentication of entities using biometrical features in a packet data network H04L63/0861
• CPC Class Codes: H04L 9/3234 involving additional secure or trusted devices, e.g. TPM, smartcard, USB or software token network architectures or network communication protocols for supporting authentication of entities using an additional device in a packet data network H04L63/0853
• CPC Class Codes: H04M 2203/6054 Biometric subscriber identification
• CPC Class Codes: H04M 2203/6081 Service authorization mechanisms
• CPC Class Codes: H04M 3/56 Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04W 12/04 Key management, e.g. using generic bootstrapping architecture [GBA]
• CPC Class Codes: H04W 12/041 Key generation or derivation
• CPC Class Codes: H04W 12/043 using a trusted network node as an anchor
• CPC Class Codes: H04W 12/06 Authentication
• CPC Class Codes: H04W 12/068 using credential vaults, e.g. password manager applications or one time password [OTP] applications
• CPC Class Codes: H04W 12/069 : using certificates or pre-shared keys
• CPC Class Codes: H04W 12/50 : Secure pairing of devices
• CPC Class Codes: H04W 12/63 : Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 12/64 : using geofenced areas
• CPC Class Codes: H04W 12/69 : Identity-dependent

Abstract

A system and method for sharing electronic data between participants of a phone conference, such as an online presentation, without a need to exchange passwords to link two devices together for data transfer nor go through a login procedure to access the data resource. To achieve this, the participants use a device or software application that samples the audio of the phone conversation and creates a stream of audio fingerprints. The streams of fingerprints are sent to a matching service on the internet. This matching service finds the fingerprints that correspond to the same conversation among the streams of simultaneous users. Once a match is found with a high enough confidence level, the matching service exchanges identifiers such as public IP addresses and sends those back to the fingerprinting units. When instructed by the user, the units can then proceed to setup a secure data connection.

Claims

1. A method for setting up of data channels in parallel to an existing voice communication, the method comprising:

automatically one-time ad-hoc setting up of data channels in parallel to the existing voice communication between first and second user equipment of first and second users,

respectively, the data channel being adapted for exchanging electronic data, the method comprising;

creating first and second fingerprints of ad hoc real time audio signals exchanged between the two users and available at the first and second user equipment, respectively;

comparing the first and second fingerprints in accordance with a matching algorithm, and

when the comparing meets a requirement of matching similarity, receiving a user action authenticating the data channel across a second network.

2. The method of claim 1, wherein the voice communication is selected from a bi-directional audio communication a phone call, a phone conference or a video conference.

3. The method of claim 1, wherein when the comparing step meets the requirement of matching similarity at least one user is informed thereof by a visual message, or by a tactile signal, an audible signal or any combination of these, or an identification message related to the identities of the users is passed to each user.

4. The method of claim 3, wherein the identification message can be selected from a user id token of the participant of the phone call, and a user id token structure of the participant of the phone call with authorization schemas of access granting protocol(s) and the authorization keys that grant access to networked resources.

5. The method of claim 4, wherein the user id token may include any one of or combination of the following attributes:

the name of the participant,

a picture or image of the person, and/or an avatar,

one or more email addresses,

an identity provider,

the URL or URI of online resources, such as for instance a web page, including a domain name,

an Open ID connect ID token, or

the token may be one or more device id tokens, identifying devices that the participant uses,

a network address, such as a public IP address and optionally a port number, an HTTP tracking cookie,

the URL or URI of online resources, such as for instance a web page, including a domain name,

an SSID of a wireless local area network (WLAN), where the device can be reached, a phone number, and/or an IMSI number,

a GPS location of the device.

6. The method of claim 1, further comprising generating a session key, a secret session key, a random session key, by using a fingerprint of one or more participants as a seed for an encryption routine.

7. The method of claim 1, wherein time shifts that appear in matching pairs of streams of fingerprints due to time lapse to traverse the first and/or second network are detected.

8. The method of claim 7, further comprising tracking the time shifts, to distinguish between silent participants and potential eavesdroppers and the users.

9. The method of claim 1, wherein before the comparing the first and second fingerprints in accordance with a matching algorithm an anonymizing function is applied.

10. The method of claim 1, wherein the first and second fingerprints are determined using a one-way function.

11. The method of claim 1, wherein all connections are torn down after termination of the voice communication or electronic data exchange leaving no surviving rights or permissions.

12. The method of claim 1, wherein the first and second user equipment of first and second users are discovered automatically on the first or second network.

13. The method of claim 1, wherein the comparing step includes matching similarity using metadata.

14. A system for setting up of data channels in parallel to an existing voice communication, the system comprising:

means for automatic one-time ad-hoc setting up of a data channel in parallel to the existing voice communication between first and second user equipment of first and second users, respectively, the data channel being adapted for exchanging electronic data,

means for creating first and second fingerprints of ad hoc real time audio signals exchanged between the two users and available at the first and second user equipment, respectively;

means for comparing the first and second fingerprints in accordance with a matching algorithm, and for determining when the comparing meets a requirement of matching similarity and means for receiving a user action authenticating the data channel across a second network.

15. The system of claim 14, wherein the voice communication is selected from a bi-directional audio communication, a phone call, a phone conference or a video conference.

16. The system of claim 14, further comprising means for informing at least one user by a visual message, or by a tactile signal, an audible signal or any combination of these when the comparing meets the requirement of matching similarity.

17. The system of claim 14, further comprising means for transmitting an identification message related to the identities of the users when the comparing meets the requirement of matching similarity.

18. The system of claim 14, further comprising means for detecting time shifts that appear in matching pairs of streams of fingerprints due to time lapse to traverse the first and/or second network.

19. The system of claim 18, further comprising means for tracking the time shifts, to distinguish between silent participants and potential eavesdroppers and the users.

20. The system of claim 14, further comprising means for anonymizing located before the means for comparing the first and second fingerprints in accordance with a matching algorithm.

21. The system of claim 14, wherein the means for creating the first and second fingerprints uses a one-way function.

22. The system of claim 14, wherein the means for creating first and second fingerprints is activated on receipt of an ingoing or on setting up of an outgoing voice communication.

23. The system of claim 14, wherein the means for comparing is a central matching service.

24. A non-transitory signal storage medium storing a computer program product, selected from software code segments when executed on a processing engine of an audio telecommunication device allows activation of a fingerprinting function when an outgoing or incoming call is made, a software being adapted to set up or allows other network elements to set up a secure connection to an identity provider, said software comprising:

software code segments when executed on a processing engine of an audio telecommunications device having a first finger print unit providing a sharing process in which sharing functionality is embedded in a fingerprint unit, following a positive match of finger prints from two audio communications having been made, wherein the software is adapted to present a participant with a list of possible sharing functionalities, the audio telecommunications device being adapted provide a choice to share content, whereby the identity of correspondents audible on the audio telecommunications device are authenticated, and

software code segments when executed on a processing engine of a telecommunications device execute a workflow of a sharing process, using a sharing service, wherein a sharing software is embodied in separate sharing applications, installed on separate devices, with no direct communication with fingerprint units to thereby extend a pairing bond between the fingerprinting unit and the participant to other sharing applications, even when these are installed on other devices or computers, and software code segments executed on a processing engine of a telecommunications device sets up secure and anonymizing tunnels to an anonymizing proxy having a secure connection with an audio fingerprint matching service, and

software code segments when executed on a processing engine of a telecommunications device execute an audio fingerprinting algorithm;

the software in a fingerprinting unit being adapted so that a human voice audio signal from the audio telecommunication device is sampled by a sampler, at a sampling rate, and

software code segments when executed on a processing engine of a telecommunication device executes a matching algorithm to match two audio fingerprints from different fingerprint units, the fingerprints of different fingerprinting units are compared to determine time shifts and for each time shift, a matching similarity or dissimilarity is determined by the software.