Generating alerts based upon detector outputs

• US 10,148,928 B2
• Filed: 05/22/2017
• Issued: 12/04/2018
• Est. Priority Date: 09/09/2013
• Status: Active Grant

• Current Assignee: Bison Patent Licensing, LLC
• Original Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Inventors: Emeott, Stephen P., Baum, Kevin L., Gandhi, Bhavan, Ishtiaq, Faisal, Ould Dellahy, Isselmou
• Primary Examiner(s): Alcon, Fernando
• Application Number:
  US15/601,559
• Publication Number:
  US 20170257612A1
• Time in Patent Office: 561 Days
• Field of Search: None
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4882 for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

Systems and methods for generating alerts and enhanced viewing experience features using on-screen data are disclosed. Textual data corresponding to on-screen text is determined from the visual content of video data. The textual data is associated with corresponding regions and frames of the video data in which the corresponding on-screen text was detected. Users can select regions in the frames of the visual content to monitor for a particular triggering item (e.g., a triggering word, name, or phrase). During play back of the video data, the textual data associated with the selected regions in the frames can be monitored for the triggering item. When the triggering item is detected in the textual data, an alert can be generated. Alternatively, the textual data for the selected region can be extracted to compile supplemental information that can be rendered over the playback of the video data or over other video data.

Claims

1. A system comprising:

a client device comprising one or more processors;

the client device coupled, via electronic communications, to a server computer comprising one or more processors, the server computer configured to receive, from a video source, video data comprising a plurality of frames, wherein each frame comprises a plurality of pixels;

the client device configured to;

determine, by the one or more processors of the client device, a monitoring region in the plurality of frames, wherein the monitoring region comprises a subset of the plurality of pixels, by associating the subset of the plurality of pixels with a key item comprising a particular pixel pattern;

extract, by the one or more processors of the client device, pixel data from the subset of the plurality of pixels in the monitoring region in a subset of the plurality of frames;

perform an analysis, by the one or more processors of the client device, of the extracted pixel data from the monitoring region; and

perform, by the one or more processors of the client device, an action based on the analyzing.

2. The system of claim 1, wherein the client device is further configured, as a part of the analysis of the extracted pixel data, to:

define, by the one or more processors of the client device, the key item comprising a particular pixel pattern;

compare, by the one or more processors of the client device, the pixel data to the particular pixel pattern; and

determine, by the one or more processors of the client device, a match based on the comparing of the pixel data to the particular pixel pattern.

3. The system of claim 2, wherein the key item comprises a text string, and wherein the particular pixel pattern comprises rendered text corresponding to the text string.

4. The system of claim 3, wherein the client device is further configured, as a part of comparison of the pixel data to the particular pattern, to:

perform, by the one or more processors of the client device, a character recognition operation on the pixel data to generate textual data; and

compare, by the one or more processors of the client device, the textual data to the text string.

5. The system of claim 1, wherein the client device is further configured, as a part of performance of the action, to:

generate, by the one or more processors of the client device, an alert that the key item has been detected.

6. The system of claim 5, wherein the client device is further configured, as a part of generation of the alert, to:

generate, by the one or more processors of the client device, an alert message; and

embed, by the one or more processors of the client device, the alert message in other video data.

7. The system of claim 6, wherein the client device is further configured, as a part of extraction of the pixel data, to:

perform, by the one or more processors of the client device, a character recognition operation on the pixel data to generate textual data;

wherein the alert message comprises the textual data.

8. The system of claim 5, wherein the video data comprises an audio component, and wherein the client device is further configured, as a part of generation of the alert, to:

change, by the one or more processors of the client device, a volume setting associated with the audio component during playback of the video data.

9. The system of claim 2, wherein the particular pixel pattern comprises an image of an object or a person.

10. The system of claim 2, wherein the client device is further configured, as a part of performance of the action, to associate the subset of the plurality of frames with the key item.

11. The system of claim 1, wherein the client device is further configured, as a part of performance of the action, to render, by the one or more processors of the client device, the subset of the plurality of frames on a display device coupled to the computer system according to a predetermined order.

12. The system of claim 1, wherein the client device is further configured, as a part of analysis of the extracted pixel data, to perform, by the one or more processors of the client device, a character recognition operation on the pixel data to generate textual data.

13. The system of claim 12, wherein the client device is further configured, as a part of performance of the action, to embed, by the one or more processors of the client device, the textual data into other video data, wherein the textual data can be rendered in superimposition over images rendered from the other video data.

14. The system of claim 12, wherein the plurality of frames are arranged according to ordered positions, and wherein the client device is further configured to render, by the one or more processors of the client device, text, based on the textual data, in superimposition over pixels of the subset of the plurality of frames according to the ordered positions.

15. The system of claim 14, wherein the client device is further configured to:

extract, by the one or more processors of the client device, additional pixel data from the subset of the plurality of pixels in the monitoring region in another subset of the plurality of frames;

perform, by the one or more processors of the client device, a character recognition operation on the additional pixel data to generate additional textual data; and

render, by the one or more processors of the client device, additional text, based on the additional textual data, in superimposition over pixels of the subset of the plurality of frames according to the ordered positions.

16. The system of claim 15 wherein the other subset of the plurality of frames is associated with a first subset of the ordered positions different from a second subset of the ordered positions associated with the subset of the plurality of frames.

17. The system of claim 14, wherein the text comprises controls for navigating to one or more of the frames in the plurality of frames.

18. A system comprising:

a client device comprising one or more processors;

the client device coupled, via electronic communications, to a server computer comprising one or more processors, the server computer configured to receive, from a video source, video data comprising a plurality of frames;

the client device configured to;

receive, by the one or more processors of the client device, previously determined detector output data corresponding to the video data, wherein the previously determined detector output data comprises textual data corresponding to images of text detected in corresponding regions of the plurality of frames;

determine, by the one or more processors of the client device, a monitoring region in the plurality of frames, wherein the monitoring region comprises at least one of the regions in the plurality of frames, by associating the at least one of the regions in the plurality of frames with a key item comprising a particular pixel pattern;

define, by the one or more processors of the client device, a text string; and

for each frame in the plurality of frames;

determine, by the one or more processors of the client device, the textual data from the previously determined detector output data for the monitoring region;

compare, by the one or more processors of the client device, the textual data with the text string;

determine, by the one or more processors of the client device, a match based on the comparing; and

perform, by the one or more processors of the client device, one or more actions in response to the match.

19. The system of claim 18, wherein at least one of the one or more actions comprises generating an alert message.