Characterizing dynamic regions of digital media data

• US 9,077,463 B2
• Filed: 10/21/2013
• Issued: 07/07/2015
• Est. Priority Date: 11/04/2005
• Status: Active Grant

• Current Assignee: EyeTracking, Inc.
• Original Assignee: EyeTracking, Inc.
• Inventors: Weatherhead, James J.
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US14/059,291
• Publication Number:
  US 20140118537A1
• Time in Patent Office: 624 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04H 60/66 for using the result on distributors' side

Abstract

A media analysis tool is provided for defining dynamic regions of a digital media segment. The dynamic regions may contain at least part of a visible feature of the segment. Correlation of the defined regions with external data quantifying attention of a subject viewing the segment to locations on the screen provides measures of interest level and attention to visible features in the segment. The dynamic regions may be defined in only some of the frames of a segment. The dynamic region may be interpolated or extrapolated for frames in which it is not explicitly defined.

Claims

1. A computer-implemented method comprising:

defining a dynamic region in a plurality of frames of a digital video stream, the dynamic region containing at least part of a visible feature in the plurality of frames and being moveable between a first frame and a second frame of the plurality of frames to coincide with movement of the at least part of the visible feature;

analyzing a relationship between the dynamic region and an external data set that is collected from a subject viewing the visible feature, the analyzing comprising determining a correlation between the dynamic region and the external data set; and

outputting a result of the analyzing, the result being indicative of at least one of interest level of the subject in the at least part of the visible feature and an attention level of the subject to the at least part of the visible feature.

2. A computer-implemented method as in claim 1, wherein the defining the dynamic region comprises producing a mathematical definition of a defined region in at least one of the plurality of frames, the defined region containing the at least part of the visible feature, the producing comprising use of at least one of an image recognition algorithm, a pattern matching algorithm, a neural network, an edge detecting algorithm, a template matching algorithm, and a frame-by-frame comparison algorithm.

3. A computer-implemented method as in claim 1, further comprising:

defining a second dynamic region in the digital video stream, the second dynamic region containing at least part of a second visible feature;

analyzing a second relationship between the external data and the second dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second relationship, the second result being indicative of at least one of a second interest level of the subject in at least part of the second visible feature and a second attention level of the subject to the at least part of the second visible feature.

4. A computer-implemented method as in claim 1, wherein the external data set comprises eye-tracking data for the subject.

5. A computer-implemented method as in claim 1, wherein the external data set comprises one or more of cognitive activity, cognitive engagement, and emotional response measurements.

6. A computer-implemented method as in claim 4, further comprising:

analyzing a second relationship between a second external data set comprising one or more of cognitive activity, cognitive engagement, and emotional response measurements and the dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second metric relationship, the second result being indicative of one or more of a level of subject cognitive activity, a level of subject cognitive engagement, and a level of subject emotional response stimulated by the visible feature.

7. A computer-implemented method as in claim 1, further comprising generating a predicted definition of the dynamic region for an additional frame other than the first frame and the second frame, the dynamic region not being defined in the additional frame.

8. A computer-implemented method as in claim 7, wherein the generating comprises interpolating the predicted definition based on positions of the dynamic region in at least the first frame and the second frame.

9. A computer-implemented method as in claim 1, wherein the defining of the dynamic region comprises specifying a first location of a vertex point of a polygon in the first frame, and the defining of the second screen region comprises specifying a second location of the vertex point in the second frame.

27. A computer-implemented method as in claim 1, wherein the defining the dynamic region comprises receiving a definition of the dynamic region in at least one of the plurality of frames via user input.

28. A computer-implemented method as in claim 1, wherein the defining the dynamic region comprises use of an algorithm that algorithm may looks for one or more areas on one or more of the plurality of frames of the digital video stream that match a predefined pattern comprising one or more of a shape, a color, a size, and an image.

10. A computer program product comprising a non-transitory computer-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

defining a dynamic region in a plurality of frames of a digital video stream, the dynamic region containing at least part of a visible feature in the plurality of frames and being moveable between a first frame and a second frame of the plurality of frames to coincide with movement of the at least part of the visible feature;

analyzing a relationship between the dynamic region and an external data set that is collected from a subject viewing the visible feature, the analyzing comprising determining a correlation between the dynamic region and the external data set; and

outputting a result of the analyzing, the result being indicative of at least one of interest level of the subject in the at least part of the visible feature and an attention level of the subject to the at least part of the visible feature.

11. A computer program product as in claim 10, wherein the defining the dynamic region comprises producing a mathematical definition of a defined region in at least one of the plurality of frames, the defined region containing the at least part of the visible feature, the producing comprising use of at least one of an image recognition algorithm, a pattern matching algorithm, a neural network, an edge detecting algorithm, a template matching algorithm, and a frame-by-frame comparison algorithm.

12. A computer program product as in claim 10, wherein the digital video stream comprises digital video from a live or streamed video source;

and wherein the defining the dynamic region occurs at least approximately in real time as the visible feature is viewed by the subject.

13. A computer program product as in claim 10, wherein the operations further comprise:

defining a second dynamic region in the digital video stream, the second dynamic region containing at least part of a second visible feature;

analyzing a second relationship between the external data and the second dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second relationship, the second result being indicative of at least one of a second interest level of the subject in at least part of the second visible feature and a second attention level of the subject to the at least part of the second visible feature.

14. A computer program product as in claim 10, wherein the external data set comprises one or more of cognitive activity, cognitive engagement, and emotional response measurements.

15. A computer program product as in claim 10, wherein the external data set comprises eye-tracking data for the subject.

16. A computer program product as in claim 15, wherein the operations further comprise:

analyzing a second relationship between a second external data set comprising one or more of cognitive activity, cognitive engagement, and emotional response measurements and the dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second metric relationship, the second result being indicative of one or more of a level of subject cognitive activity, a level of subject cognitive engagement, and a level of subject emotional response stimulated by the visible feature.

17. A computer program product as in claim 10, further comprising generating a predicted definition of the dynamic region for an additional frame other than the first frame and the second frame, the dynamic region not being defined in the additional frame.

18. A computer program product as in claim 17, wherein the generating comprises interpolating the predicted definition based on positions of the dynamic region in at least the first frame and the second frame.

22. A system as in claim 18, wherein the external data set comprises one or more of cognitive activity, cognitive engagement, and emotional response measurements.

19. A system comprising:

computer hardware configured to perform operations comprising;

defining a dynamic region in a plurality of frames of a digital video stream, the dynamic region containing at least part of a visible feature in the plurality of frames and being moveable between a first frame and a second frame of the plurality of frames to coincide with movement of the at least part of the visible feature;

analyzing a relationship between the dynamic region and an external data set that is collected from a subject viewing the visible feature, the analyzing comprising determining a correlation between the dynamic region and the external data set; and

outputting a result of the analyzing, the result being indicative of at least one of interest level of the subject in the at least part of the visible feature and an attention level of the subject to the at least part of the visible feature.

20. A system as in claim 19, wherein the defining the dynamic region comprises producing a mathematical definition of a defined region in at least one of the plurality of frames, the defined region containing the at least part of the visible feature, the producing comprising use of at least one of an image recognition algorithm, a pattern matching algorithm, a neural network, an edge detecting algorithm, a template matching algorithm, and a frame-by-frame comparison algorithm.

21. A system as in claim 19, further comprising:

defining a second dynamic region in the digital video stream, the second dynamic region containing at least part of a second visible feature;

analyzing a second relationship between the external data and the second dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second relationship, the second result being indicative of at least one of a second interest level of the subject in at least part of the second visible feature and a second attention level of the subject to the at least part of the second visible feature.

23. A system as in claim 19, wherein the external data set comprises eye-tracking data for the subject.

24. A system as in claim 23, wherein the operations further comprise:

analyzing a second relationship between a second external data set comprising one or more of cognitive activity, cognitive engagement, and emotional response measurements and the dynamic region in the digital video stream; and

outputting a second result of the analyzing of the second metric relationship, the second result being indicative of one or more of a level of subject cognitive activity, a level of subject cognitive engagement, and a level of subject emotional response stimulated by the visible feature.

25. A system as in claim 19, further comprising generating a predicted definition of the dynamic region for an additional frame other than the first frame and the second frame, the dynamic region not being defined in the additional frame.

26. A system as in claim 25, wherein the generating comprises interpolating the predicted definition based on positions of the dynamic region in at least the first frame and the second frame.