VIEWER BEHAVIOR TRACKING USING PATTERN MATCHING AND CHARACTER RECOGNITION

• US 20140282668A1
• Filed: 03/14/2013
• Published: 09/18/2014
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Gava, Fabio, Zhu, Yingnan
• Granted Patent: US 8,955,005 B2
• Time in Patent Office: Days
• US Class Current: 725/19
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104

Abstract

A method for tracking user behavior includes providing multiple frames from a source to an electronic device, automatically capturing one of the multiple frames viewed on a display device connected to the electronic device based on an event, matching a pattern of the captured frame with a frame template by searching a plurality of frame templates, using the matched frame template to extract information from the captured frame, and storing the extracted information for tracking user behavior.

Claims

1. A method for tracking user, comprising:

providing multiple frames from a source to an electronic device;

capturing one of the multiple frames viewed on a display device connected to the electronic device automatically based on an event;

matching a pattern of the captured frame with a frame template by searching a plurality of frame templates;

using the matched frame template to extract information from the captured frame; and

storing the extracted information for tracking user behavior.

2. The method of claim 1, wherein the source comprises one or more of a set-top box (STB), a disc player, a game console, a digital streaming device, a browser, a broadcast provider, a programming provider, and a computing device.

3. The method of claim 2, wherein the event is one or more of a periodic interval time period, a detected channel change, and a detected remote control signal.

4. The method of claim 3, wherein the plurality of frame templates are one of stored on the electronic device and stored in a cloud computing environment.

5. The method of claim 4, wherein each frame template of the plurality of frame templates comprises information for a particular screen layout.

6. The method of claim 5, wherein the information for the particular screen layout comprises one or more of an associated device, an associated application or an associated event, a first crop area used for pattern matching, an optical character recognition (OCR) mask for extracting information based on a masked area in a frame, a second crop area used for pattern matching and extracting information, training data used for OCR, and pre-processing information for pre-processing a frame.

7. The method of claim 6, wherein the extracted information comprises available text information included in the captured frame.

8. The method of claim 1, further comprising:

reducing a search domain for searching the plurality of frame templates for matching the pattern.

9. The method of claim 8, wherein reducing the search domain comprises:

determining an input source for the captured frame based on one or more of location for multi-system operators, type of input signal, type of content, type of device and input source resolution.

10. The method of claim 9, wherein matching the pattern of the captured frame with the frame template by searching the plurality of frame templates comprises determining a result of performing pixel level matching, and comparing the result to a threshold.

11. The method of claim 9, wherein matching the pattern of the captured frame with the frame template by searching the plurality of frame templates comprises determining a result of frequency domain matching based on forward and reverse transformation for a cropped image.

12. The method of claim 4, wherein the plurality of templates are collected from a plurality of users and stored in a database in the cloud computing environment, wherein the plurality of templates are collected from each user based on extracting information from captured frames after a remote control command is activated, wherein programming banners on the captured frames are used for extracting the information.

13. A system for tracking user behavior, comprising:

a display device that receives multiple image frames from a source; and

an electronic device coupled to the display device, the electronic device comprising a frame recognition module that automatically captures one of the multiple frames based on an event, matches a pattern of the captured frame with a frame template by searching a plurality of frame templates stored in a first memory, uses the matched frame template to extract information from the captured frame, and stores the extracted information in a second memory for tracking user behavior.

14. The system of claim 13, wherein the source comprises one or more of a set-top box (STB), a disc player, a game console, a digital streaming device, a browser, a broadcast provider, a programming provider, and a computing device.

15. The system of claim 14, wherein the event is one or more of a periodic interval time period, a detected channel change, and a detected remote control signal.

16. The system of claim 15, wherein the first memory and the second memory are one of coupled to the electronic device and stored in a cloud computing environment.

17. The system of claim 16, wherein each frame template of the plurality of frame templates comprises information for a particular screen layout that comprises one or more of an associated device, an associated application or an associated event, a first crop area used for pattern matching, an optical character recognition (OCR) mask for extracting information based on a masked area in a frame, a second crop area used for pattern matching and extracting information, training data used for OCR, and pre-processing information for pre-processing a frame.

18. The system of claim 17, wherein the extracted information comprises available text information included in the captured frame.

19. The system of claim 18, wherein the frame recognition module filters the plurality of frame templates by detecting an input source for the captured frame based on one or more of location for multi-system operators, type of input signal, type of content, type of device used for providing the captured frame, and input source resolution.

20. The system of claim 19, further comprising a cloud computing environment that stores the plurality of templates in a database, wherein the plurality of templates are collected from each user by a respective electronic device based on extracting information from captured frames after a remote control command is activated, wherein programming banners on the captured frames provide the extracted information.

21. The system of claim 13, wherein the electronic device comprises a television device.

22. A non-transitory computer-readable medium having instructions which when executed on a computer perform a method comprising:

providing multiple display frames from a source to an electronic device;

capturing one of the multiple display frames viewed on a display device connected to the electronic device automatically based on an event;

matching a pattern of the captured frame with a frame template by searching a plurality of frame templates;

using the matched frame template to extract information from the captured frame; and

storing the extracted information for tracking user behavior.

23. The medium of claim 22, wherein the source comprises one or more of a set-top box (STB), a disc player, a game console, a digital streaming device, a browser, a broadcast provider, a programming provider, and a computing device.

24. The medium of claim 23, wherein the event is one or more of a periodic interval time period, a detected channel change, and a detected remote control signal, and the plurality of frame templates are one of stored on the electronic device and stored in a cloud computing environment.

25. The medium of claim 24, wherein each frame template of the plurality of frame templates comprises information for a particular screen layout that comprises one or more of an associated device, an associated application or an associated event, a first crop area used for pattern matching, an optical character recognition (OCR) mask for extracting information based on a masked area in a frame, a second crop area used for pattern matching and extracting information, training data used for OCR, and pre-processing information for pre-processing a frame.

26. The medium of claim 25, wherein the extracted information comprises available text information included in the captured frame.

27. The medium of claim 26, further comprising:

reducing a search domain for searching the plurality of frame templates for matching the pattern based on determining an input source for the captured frame based on one or more of location for multi-system operators, type of input signal, type of content, type of device and input source resolution.

28. The medium of claim 27, wherein matching the pattern of the captured frame with the frame template by searching the plurality of frame templates comprises determining a result of performing pixel level matching, and comparing the result to a threshold.

29. The medium of claim 27, wherein matching the pattern of the captured frame with the frame template by searching the plurality of frame templates comprises determining a result of frequency domain matching based on forward and reverse transformation for a cropped image.

30. The medium of claim 24, wherein the plurality of templates are collected from a plurality of users and stored in a database in the cloud computing environment, wherein the plurality of templates are collected from each user based on extracting information from captured frames after a remote control command is activated, wherein programming banners on the captured frames are used for extracting the information.