Subselection of portions of an image review sequence using spatial or other selectors

• US 9,098,503 B1
• Filed: 10/06/2011
• Issued: 08/04/2015
• Est. Priority Date: 10/06/2011
• Status: Active Grant

• Current Assignee: Pixar (The Walt Disney Company)
• Original Assignee: Pixar (The Walt Disney Company)
• Inventors: Shah, Apurva, Kass, Michael, Johnson, Michael B., Chen, Yu-Chung
• Primary Examiner(s): Liu, Zhengxi
• Application Number:
  US13/267,765
• Time in Patent Office: 1,398 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

An image review system presents an image to a reviewer and records reviewer feedback, including reviewer position indicators relating to portions of the image and reviewer commentary data, which can be audio, visual, textual, etc. The reviewer position indicators and reviewer commentary data each have a time component, corresponding to when the reviewer indicated a position on the image and when the reviewer provided the commentary, respectively. During playback, the viewer can select a subset of the image of interest and get a playback of the reviewer commentary data that corresponds to the times represented by the reviewer position indicators that fall within the selected subset of the image. The reviewer position indicators can be collectively indicated by a heat map that, when laid over the image, indicates which portions of the image are commented on and how much time is spent there.

Claims

1. An image review system for processing reviewer inputs related to an image, comprising:

an input for receiving reviewer position indicators, wherein a first reviewer position indicator of the reviewer position indicators corresponds to a first subset region of the image to which a reviewer indicates;

storage for the reviewer position indicators;

wherein the first reviewer position indicator of the reviewer position indicators is stored with a first timestamp representing a review time at which the reviewer indicated the first subset region of the image;

an input for receiving reviewer commentary data;

storage for the reviewer commentary data, wherein first reviewer commentary data corresponds to the first subset region of the image associated with the first reviewer position indicator, and wherein the first reviewer commentary data is stored with the first reviewer position indicator and the first timestamp;

logic for generating a map of aggregated reviewer position indicators, wherein the map, when overlaid on the image, indicates portions of the image commented on by the reviewer at a plurality of review times; and

a display for displaying the map of aggregated reviewer position indicators, wherein the map of aggregated reviewer position indicators is two dimensional, and wherein the aggregated reviewer position indicators correspond to summations of review activity over a plurality of actions on a plurality of reviewed images, and

wherein the summations of review activity over the plurality of actions on the plurality of reviewed images comprise summations based on one or more objects'"'"' varying positions from image to image, wherein the varying positions correspond to motions of the one or more objects from image to image in the plurality of reviewed images.

2. The image review system of claim 1, wherein displaying the map of aggregated reviewer position indicators comprises generating an overlay image wherein each of a plurality of regions of the overlay image are colored with a color that encodes a number of reviewer position indicators that occur in each of the plurality of regions.

3. A method of processing image review data, comprising:

displaying, by a computer, an image to be reviewed;

recording, by the computer, reviewer commentary data, wherein the reviewer commentary data has associated timestamps;

recording, by the computer, a plurality of reviewer position indicators, wherein a first reviewer position indicator of the reviewer position indicators corresponds to a first subset region of the image to which a reviewer indicates while commenting, and wherein recording the first reviewer position indicator includes storing a first timestamp representing a review time at which the reviewer indicated the first subset region of the image; and

generating, by the computer, a map of aggregated reviewer position indicators, wherein the map, when overlaid on the image, indicates portions of the image commented on by the reviewer at a plurality of review times,

wherein the map of aggregated reviewer position indicators is two dimensional,

wherein the aggregated reviewer position indicators correspond to summations of review activity over a plurality of actions on a plurality of reviewed images, and

wherein the summations of review activity over the plurality of actions on the plurality of reviewed images comprise summations based on one or more objects'"'"' varying positions from image to image, wherein the varying positions correspond to motions of the one or more objects from image to image in the plurality of reviewed images.

4. The method of claim 3, further comprising:

receiving user input as to a selected subset region of the image;

identifying a subset of the plurality of reviewer position indicators that have corresponding locations that correspond to the selected subset region of the image;

identifying a subset of the reviewer commentary data that corresponds to the identified subset of the plurality of reviewer position indicators; and

replaying reviewer commentary data that is part of the subset of the reviewer commentary data while omitting reviewer commentary data that is not part of subset of the reviewer commentary data.

5. The method of claim 4, wherein the corresponding locations of the subset of the plurality of reviewer position indicators are each within the selected subset region or within an indicator adjustment range of the selected subset region, and wherein the corresponding locations of the subset of the reviewer commentary data correspond to the identified subset of the plurality of reviewer position indicators by having timestamps that are equal or within a predetermined time difference.

6. The method of claim 5, wherein the predetermined time difference is a fixed time period added to starting and ending times of the subset of the reviewer commentary data that is played.

7. The method of claim 6, wherein the reviewer commentary data includes audio data and replaying the audio data includes playing audible indicators of transitions between reviewer commentary data that is part of the subset of the reviewer commentary data and reviewer commentary data that is not part of subset of the reviewer commentary data that is omitted.

8. The method of claim 4, wherein receiving user input as to the selected subset region of the image comprises presenting a user with a heat map of the plurality of reviewer position indicators and accepting a user selection of portions of the heat map.

9. The method of claim 4, wherein the selected subset region of the image is determined based on detecting a closed curve of nonzero area produced by a user on the image.

10. The method of claim 4, wherein identifying the subset of the reviewer commentary data that corresponds to the identified subset of the plurality of reviewer position indicators includes:

identifying reviewer position indicators that overlap with the selected subset region, wherein the subset of the plurality of reviewer position indicators is not equal to the selected subset region.

11. The method of claim 4, wherein the reviewer commentary data includes a streaming data file, and wherein identifying the subset of the reviewer commentary data that corresponds to the identified subset of the plurality of reviewer position indicators comprises:

filtering, by the computer, disjointed portions from the streaming data file associated with the identified subset of the plurality of reviewer position indicators; and

combining, by the computer, the disjointed portions into a playback file.

12. The method of claim 3, further comprising weighting, by the computer, aggregated reviewer position indicators unequally such that some of the aggregated reviewer position indicators are weighted more heavily than other aggregated reviewer position indicators.

13. The method of claim 12, wherein weights used for aggregated reviewer position indicators correspond to one or more of input device pressure, speed of position movement, reviewer voice volume, and/or reviewer gestures.

14. The method of claim 3, wherein the plurality of reviewer position indicators comprise one or more of points, grid locations, text insert locations, drawing marking locations, and/or an object selection.

15. The method of claim 3, wherein the plurality of reviewer position indicators comprise grid locations from a grid defined by lines on the image that divide the image into rectangular regions each more than one pixel wide and more than one pixel high, such that at least some reviewer position indicators corresponding to different locations on the image are mapped to a same grid location.

16. The method of claim 3, wherein the reviewer commentary data comprises audio having identified playtimes that correspond to at least one timestamp of one of the plurality of reviewer position indicators.

17. The method of claim 3, wherein the reviewer commentary data comprises drawing inputs.

18. The method of claim 3, further comprising:

generating, by the computer, representations of portions of the plurality of images based on portions associated with the aggregated reviewer position indicators; and

indicating, by the computer, associations between the representations and portions of the map of aggregated reviewer position indicators, such that the representations provide an indication of which of the plurality of images and which portions of the plurality of images are associated with various aggregated reviewer position indicators.

19. The method of claim 18, wherein the representations comprise one or more of a mosaic of the portions, a film strip of at least the portions, and a collage of the portions.