System and method for auto content recognition

• US 8,793,274 B2
• Filed: 08/08/2011
• Issued: 07/29/2014
• Est. Priority Date: 08/08/2011
• Status: Active Grant

• Current Assignee: Vobile Incorporated (Vobile Group Ltd.)
• Original Assignee: Lei Yu, Qier Lv, Yangbin Wang
• Inventors: Yu, Lei, Wang, Yangbin, Lv, Qier
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US13/204,845
• Publication Number:
  US 20110289114A1
• Time in Patent Office: 1,086 Days
• Field of Search: 707/2, 707/3, 707/769, 382/100, 704/200.1
• US Class Current: 707/769
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]

Abstract

System and method for automatically recognizing media contents comprise steps of capturing media content from the Internet and/or devices, extracting fingerprints from captured contents and transferring to the backend servers for identification, and backend servers processing the fingerprints and replying with identified result.

Claims

1. A method of auto content recognition by VDNA (Video DNA, here DNA simply stands for identifier) fingerprints of media content for both video and audio contents, said method comprising:

a) Capturing contents with audio and video sensors,

b) Processing said captured contents by extracting VDNA fingerprints so that they are feasible and secure to transfer over Internet,

c) Sending said extracted VDNA fingerprints to content identification server, and

d) Said content identification server replying with information of identified contents after identifying said VDNA fingerprint with its registered contents,

wherein

parameters of VDNA fingerprint extraction are privately predefined and agreed by both front-end devices and said identification server,

said VDNA fingerprint is very compact for transmission from said front-end to said identification server and said VDNA fingerprint can be a single file or fingerprints data stream wherein said fingerprints data stream is divided into pieces of fingerprint data with any time interval as separate VDNA fingerprint data files, and said identification server combines the received pieces of said fingerprint data from said front-end, and

said VDNA fingerprints are non-recoverable and non-reversible to protect privacy.

2. The method as recited in claim 1, wherein said captured contents can be eye sensible contents such as video and image that can be captured by camera, ear sensible contents that can be captured by recorder, or other information such as text that can be captured by sensors.

3. The method as recited in claim 1, wherein said processing comprises generating said VDNA fingerprints which are feasible and secure to transfer over communication facilities, and said VDNA fingerprints can be split into small pieces for transmission purpose and can also join together to restore said original VDNA fingerprints.

4. The method as recited in claim 3, wherein said processing of original content generates said transmissible VDNA fingerprints.

5. The method as recited in claim 3, wherein said VDNA fingerprints are used to identify contents, and said VDNA fingerprints are non-recoverable after generation to protect privacy.

6. The method as recited in claim 5, wherein said VDNA fingerprints can also be an URI (Uniform Resource Identifier) to globally and uniquely identify said content on an identification server.

7. The method as recited in claim 1, wherein said sending to said server can be through Internet by TCP/IP (Transmission Control Protocol and Internet Protocol), through mobile communications such as GSM (Global Service of Mobile communication) or CDMA (Code Division Multiplex Access) networks, and other networks.

8. The method as recited in claim 1, wherein said VDNA fingerprints can be sent as soon as said content is captured, which is defined as online mode, or saved in a terminal and sent later when network is available, which is defined as offline mode.

9. The method as recited in claim 1, wherein said information replying from said server can comprise business rules such as pre-formatted text and script used to help people recognize said content easily, or contents related to said captured content used to help people record said content.

10. The method as recited in claim 1, wherein result of said identification can be used to learn more about source of recognized media.

11. The method as recited in claim 1, wherein time between said VDNA fingerprints in said identifying process sent to said server is one of factors affecting result.

12. A system of auto content recognition by VDNA (Video DNA, here DNA simply stands for identifier) fingerprints of media content for both video and audio contents, said system comprising:

a) components of front-end sub-system with capturing function and user interface,

b) transmission sub-system with VDNA fingerprint process function on said front-end,

c) communication sub-system transferring data between said front-end and identification server together with identifying function on said server,

d) identification sub-system with said identifying function, and

e) a back-end database of registered contents,

wherein

parameters of VDNA fingerprint extraction are privately predefined and agreed by both front-end devices and said identification server,

said VDNA fingerprint is very compact for transmission from said front-end to said identification server and said VDNA fingerprint can be a single file or fingerprints data stream wherein said fingerprints data stream is divided into pieces of fingerprint data with any time interval as separate VDNA fingerprint data files, and said identification server combines the received pieces of said fingerprint data from said front-end, and

said VDNA fingerprints are non-recoverable and non-reversible to protect privacy.

13. The system as recited in claim 12, wherein said front-end can be an application program or API (application program interface) on devices playing contents.

14. The system as recited in claim 12, wherein said front-end can be application or API (application program interface) on devices that have sensors which can capture content from outside said device.

15. The system as recited in claim 12, wherein said VDNA fingerprint processor on said front-end is used to make content transmitted through said communication sub-system, and said VDNA fingerprint produced by said VDNA fingerprint processor will be said content itself or data used to identify said content.

16. The system as recited in claim 12, wherein said identifying function can work on real-time returning results during said identification progress as well as at the end of said identification progress.

17. The system as recited in claim 12, wherein said identifying function utilizes context sent to said server earlier to improve said identification results.

18. A method of database generation for auto content recognition by VDNA (Video DNA, here DNA simply stands for identifier) fingerprints of media content for both video and audio contents, said method comprising:

a) Registering media provided by content owners as master contents,

b) Generating VDNA fingerprint data of said master contents on front-end used for generating VDNA fingerprint for captured media, and

c) Collecting metadata of registered media contents in back-end database,

wherein

parameters of VDNA fingerprint extraction are privately predefined and agreed by both front-end devices and said identification server,

said VDNA fingerprint is very compact for transmission from said front-end to said identification server and said VDNA fingerprint can be a single file or fingerprints data stream wherein said fingerprints data stream is divided into pieces of fingerprint data with any time interval as separate VDNA fingerprint data files, and said identification server combines the received pieces of said fingerprint data from said front-end, and

said VDNA fingerprints are non-recoverable and non-reversible to protect privacy.

19. The method as recited in claim 18, wherein said master contents are media contents ready to be identified.

20. The method as recited in claim 18, wherein said metadata of said master contents can also be used to identify said media contents.