Systems and methods for processing media content that depict objects

• US 10,733,478 B2
• Filed: 08/31/2016
• Issued: 08/04/2020
• Est. Priority Date: 08/31/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Gomez, Zachary W., He, Lin, Ruben, William M.
• Primary Examiner(s): Winder, Patrice L
• Assistant Examiner(s): Aung, Aye M
• Application Number:
  US15/253,658
• Publication Number:
  US 20180063227A1
• Time in Patent Office: 1,434 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

Access to a set of media content items is acquirable. Identified processors can perform, in parallel, object detection for the set. In some cases, information about a current system state, a user, and/or object popularity metrics is acquirable for selecting a subset of object models. Object recognition is performable, based on the subset, for the set of media content items. In some instances, a camera view can be provided. Object recognition is performable for representations of the view. An object depicted in the representations is identifiable. An interface portion is presentable to provide a label for the object. In some cases, object recognition is performable for the set of media content items to identify an object depicted in a content item. A label is associable with the content item. A search through the set of media content items can identify, based on the label, a subset that depicts the object.

Claims

1. A computer-implemented method comprising:

acquiring, by a computing system, access to a set of media content items;

identifying, by the computing system, a plurality of processors;

prioritizing, by the computing system, the set of media content items for object detection, wherein a particular media content item with which a user is currently interacting is prioritized over other media content items in the set of media content items and wherein the object detection is performed with a lower central processing unit (CPU) priority than at least one other operation performed by the computing system; and

utilizing, by the computing system, each processor in the plurality of processors, in parallel, to perform the object detection with respect to a respective media content item based at least in part on the prioritizing, wherein the object detection includes determining an image class associated with the respective media content item based at least in part on a visual pattern template associated with the image class.

2. The computer-implemented method of claim 1, wherein a first quantity associated with the set of media content items is greater than a second quantity associated with the plurality of processors.

3. The computer-implemented method of claim 2, further comprising:

determining that object detection has yet to be performed with respect to at least some media content items in the set of media content items;

determining that a particular processor in the set of processor has completed performing object detection with respect to a particular media content item in the set of media content items; and

utilizing the particular processor to perform object detection with respect to a first media content item out of the at least some media content items.

4. The computer-implemented method of claim 1, further comprising:

acquiring recency data for the set of media content items; and

sorting the set of media content items based on the recency data to produce an order in which object detection is to be performed with respect to the set of media content items, wherein the order is from most recent to least recent.

5. The computer-implemented method of claim 1, further comprising:

detecting that a user of the computing system is accessing a particular media content item in the set of media content item; and

sorting the set of media content items such that the particular media content item is prioritized over at least one other media content item when object detection is performed with respect to the set of media content items.

6. The computer-implemented method of claim 1, wherein the object detection is performed with the lower CPU priority and the at least one other operation performed by the computing system includes a scrolling operation.

7. The computer-implemented method of claim 1, wherein object detection performed with respect to the respective media content item identifies a portion of the respective media content item in which a detected object is depicted.

8. The computer-implemented method of claim 7, further comprising:

transmitting the portion of the respective media content item to a server configured to perform object recognition with respect to the portion of the respective media content item.

9. The computer-implemented method of claim 1, wherein object detection includes face detection.

10. A system comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, cause the system to perform;

acquiring access to a set of media content items;

identifying a plurality of processors;

prioritizing set of media content items for object detection, wherein a particular media content item with which a user is currently interacting is prioritized over other media content items in the set of media content items and wherein the object detection is performed with a lower central processing unit (CPU) priority than at least one other operation performed by the computing system; and

utilizing each processor in the plurality of processors, in parallel, to perform the object detection with respect to a respective media content item based at least in part on the prioritizing, wherein the object detection includes determining an image class associated with the respective media content item based at least in part on a visual pattern template associated with the image class.

11. The system of claim 10, wherein a first quantity associated with the set of media content items is greater than a second quantity associated with the plurality of processors.

12. The system of claim 11, wherein the instructions cause the system to further perform:

determining that object detection has yet to be performed with respect to at least some media content items in the set of media content items;

determining that a particular processor in the set of processor has completed performing object detection with respect to a particular media content item in the set of media content items; and

utilizing the particular processor to perform object detection with respect to a first media content item out of the at least some media content items.

13. The system of claim 10, wherein the instructions cause the system to further perform:

acquiring recency data for the set of media content items; and

sorting the set of media content items based on the recency data to produce an order in which object detection is to be performed with respect to the set of media content items, wherein the order is from most recent to least recent.

14. The system of claim 10, wherein the instructions cause the system to further perform:

detecting that a user of the system is accessing a particular media content item in the set of media content item; and

sorting the set of media content items such that the particular media content item is prioritized over at least one other media content item when object detection is performed with respect to the set of media content items.

15. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:

acquiring access to a set of media content items;

identifying a plurality of processors;

prioritizing the set of media content items for object detection, wherein a particular media content item with which a user is currently interacting is prioritized over other media content items in the set of media content items and wherein the object detection is performed with a lower central processing unit (CPU) priority than at least one other operation performed by the computing system; and

utilizing each processor in the plurality of processors, in parallel, to perform the object detection with respect to a respective media content item based at least in part on the prioritizing, wherein the object detection includes determining an image class associated with the respective media content item based at least in part on a visual pattern template associated with the image class.

16. The non-transitory computer-readable storage medium of claim 15, wherein a first quantity associated with the set of media content items is greater than a second quantity associated with the plurality of processors.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions cause the computing system to further perform:

determining that object detection has yet to be performed with respect to at least some media content items in the set of media content items;

determining that a particular processor in the set of processor has completed performing object detection with respect to a particular media content item in the set of media content items; and

utilizing the particular processor to perform object detection with respect to a first media content item out of the at least some media content items.

18. The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the computing system to further perform:

acquiring recency data for the set of media content items; and

sorting the set of media content items based on the recency data to produce an order in which object detection is to be performed with respect to the set of media content items, wherein the order is from most recent to least recent.

19. The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the computing system to further perform:

detecting that a user of the computing system is accessing a particular media content item in the set of media content item; and

sorting the set of media content items such that the particular media content item is prioritized over at least one other media content item when object detection is performed with respect to the set of media content items.