Relating items to objects detected in media

• US 10,091,556 B1
• Filed: 12/12/2012
• Issued: 10/02/2018
• Est. Priority Date: 12/12/2012
• Status: Active Grant

• Current Assignee: IMDb.com, Inc. (Amazon.com, Inc.)
• Original Assignee: IMDb.com, Inc. (Amazon.com, Inc.)
• Inventors: Liden, Lars Hasso, MacTaggart, Rachel Jane, Kulkarni, Ashutosh Vishwas
• Primary Examiner(s): Pierorazio, Michael B
• Application Number:
  US13/711,659
• Time in Patent Office: 2,120 Days
• Field of Search: 725 14, 725 34, 725 60
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region

Abstract

Disclosed are various embodiments for relating items to objects detected in media. An item recognition application obtains media information associated with a broadcast being consumed form the client device. The item recognition application identifies objects in the media based on user input of the location of the object on the client device display and/or heuristic detection of the object. The user may indicate the object'"'"'s location through an input device. The object may be heuristically detected based on an algorithm that differentiates the object from the background. The item recognition application identifies items related to the object. The item recognition application then performs an action with the item.

Claims

1. A non-transitory computer-readable medium embodying a program that, when executed in at least one computing device, causes the at least one computing device to at least:

obtain at least one coordinate associated with a media frame based on at least one of;

a user input through a touch screen client device, or

a user input through a pointing device;

perform an image recognition on the media frame to identify an object depicted at the at least one coordinate;

generate an indicator of the object identified in the media frame;

send the indicator to a client device;

obtain a user selection of the object; and

perform an action with respect to an item related to the object in response to the user selection, the action being indicated in at least one of a plurality of media viewing preferences defined with respect to a user;

wherein the action comprises at least one of;

initiating a purchase of an item related to the object,

obtaining a user identified item related to the object,

automatically identifying an item related to the object,

generating an item detail page related to the object, or

performing a search for an item related to the object.

2. The non-transitory computer-readable medium of claim 1, wherein the indicator is based at least in part on an overlay user interface on top of the media frame.

3. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to at least obtain the at least one of a plurality of media viewing preferences.

4. A system, comprising:

at least one computing device; and

a memory device comprising an item detection executable, the item detection executable, when executed by the at least one computing device, causing the at least one computing device to at least;

obtain a coordinate in a media frame corresponding to a location of an object depicted in the media frame, the coordinate being based at least in part on a user input;

perform an image recognition analysis of the media frame to detect an object depicted proximate to the coordinate;

generate an indicator of the object in the location as depicted in the media frame;

obtain a user selection of the object at the location identified by the indicator; and

perform, in response to the user selection, an action with respect to an item related to the object, the action being indicated in at least one of a plurality of media viewing preferences defined with respect to a user.

5. The system of claim 4, wherein the item detection executable further causes the at least one computing device to at least define the plurality of media viewing preferences based on at least one of a plurality of user inputs.

6. The system of claim 4, wherein the item detection executable further causes the at least one computing device to at least define the plurality of media viewing preferences based on an analysis of a media viewing history associated with the user.

7. The system of claim 4, wherein the location of the object is identified based on at least one of a timestamp, a user identified coordinate or a masking algorithm.

8. The system of claim 4, wherein the indicator comprises an overlay user interface.

9. The system of claim 4, wherein the indicator comprises an addition to the media frame.

10. The system of claim 4, wherein the indicator comprises the location of the object.

11. The system of claim 4, wherein the action comprises receiving a user selection of an item related to the object.

12. The system of claim 4, wherein the action comprises adding the item related to the object corresponding to the user selection to a list.

13. The system of claim 12, wherein the list comprises a shopping cart list.

14. The system of claim 12, wherein the list comprises a wish list.

15. The system of claim 7, wherein the masking algorithm identifies the coordinates based at least in part on a background mask.

16. The system of claim 4, wherein the action further comprises changing a viewing speed of a media broadcast.

17. The system of claim 4, wherein the action comprises generating an item detail page associated with the item related to the object corresponding to the user selection.

18. The system of claim 4, wherein the action further comprises initiating an incentive to a user providing input.

19. The system of claim 4, wherein at least one of the media viewing preferences indicates whether to pause a media broadcast in response to the user selection.

20. A method, comprising:

obtaining, in a computing device, a plurality of coordinates associated with an object depicted in a media frame;

perform an image recognition on the media frame to detect the object based at least in part on the plurality of coordinates;

generating, in the computing device, an indicator of the object at the coordinates;

obtaining, in the computing device, a user selection of the object; and

performing an action with respect to an item related to the object, the action being indicated in a plurality of media viewing preferences defined with respect to a user.

21. The method of claim 20, wherein the coordinates are based at least in part on at least one user input through a touchscreen on the computing device.

22. The method of claim 20, wherein the coordinates are based at least in part on at least one user input through a pointing device associated with the computing device.

23. The method of claim 20, wherein the coordinates are based at least in part on a masking algorithm.

24. The method of claim 23, wherein the masking algorithm identifies the coordinates based at least in part on a background mask.

25. The method of claim 24, wherein the background mask is based at least in part on a Canny edge detection algorithm to the media frame.

26. The method of claim 23, wherein the masking algorithm identifies the coordinates based at least in part on a human feature mask.

27. The method of claim 26, wherein the human feature mask is based at least in part on a Hue, Saturation, Value model to the media frame.

28. The method of claim 20, wherein the action comprises at least one of an adding the item related to the object to a list, a generation of a user interface of an item detail page associated with the item related to the object, changing viewing speed of a media broadcast, initiating an incentive for a user providing input, or a generation of a user interface of a plurality of potential actions from which the user may dynamically select.