Updating social graph data using physical objects identified from images captured by smartphone

• US 9,223,893 B2
• Filed: 08/09/2012
• Issued: 12/29/2015
• Est. Priority Date: 10/14/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F.
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US13/571,049
• Publication Number:
  US 20130094702A1
• Time in Patent Office: 1,237 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]

Abstract

This technology concerns, in one aspect, using a person'"'"'s social network graph data as a virtual visual cortex—taking image input from a smartphone or the like, and processing it with the graph data to yield a personalized form of processing based on the imagery. The user'"'"'s network graph data is typically updated by such processing—providing a form of virtual image memory that can influence future social network behavior. In another aspect, the technology concerns identifying content (e.g., audio) by both fingerprint-based and watermark-based techniques, and arrangements employing such identification data. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

analyzing first data corresponding to imagery captured by a user'"'"'s smartphone camera, to yield identification data corresponding to a first non-human physical object depicted in the imagery, said analyzing being performed using electronic circuitry and comprising an image fingerprinting operation that includes (a) identifying candidate keypoints by convolving the first data with a blur filter to detect space-scale extrema of a scale-localized Laplacian transform of the image, (b) discerning and discarding keypoints having a contrast below a threshold value, said discerning including performing a second order Taylor expansion, and (c) checking remaining keypoints for matches with reference keypoints for reference physical objects; and

issuing instructions that a social network graph data structure be updated to add node and tie elements, the node element corresponding to the first physical object, and the tie element indicating interaction by the user with said first physical object, information in said social network graph data structure being published by a service for viewing by one or more third parties;

wherein the first physical object is identified by reference to the first data, and the user'"'"'s interaction with the first physical object is automatically publicized to said one or more third parties.

2. The method of claim 1 that further includes providing a response to the user'"'"'s smartphone that is based on the first physical object, and is also based on one or more other node or tie elements of a social network graph data structure.

3. The method of claim 2 in which said response includes advertising.

4. The method of claim 1 that further includes providing a response to the user'"'"'s smartphone that depends on the first physical object, and also depends on a second user'"'"'s interaction with a second physical object.

5. The method of claim 4 in which said second user comprises one of said third parties.

6. The method of claim 4 in which both the first and second physical objects are both members of a vehicle object class.

7. The method of claim 1 that further includes issuing instructions that the data structure be updated with metadata selected by the user.

8. The method of claim 1 that includes:

providing the first data to processor circuitry remote from the smartphone, said processor circuitry being controlled by said service; and

said service updating the social network graph data structure to add said node and tie elements;

wherein said service both performs said analyzing and said updating.

9. The method of claim 1 in which said one or more third parties comprise social network friends of the user.

10. The method of claim 1 in which said first physical object comprises a printed medium selected from the list consisting of a poster, a magazine, a newspaper, an electronic display, and a book.

11. The method of claim 1 that includes notifying a proprietor of the first physical object about said interaction.

12. The method of claim 1 that includes storing at least a portion of the captured imagery in association with said node.

13. The method of claim 1 that includes storing at least a portion of the captured imagery in said data structure in association with said node.

14. The method of claim 1 that includes storing at least a portion of said identification data in association with said node.

15. The method of claim 1 that includes updating the social network graph data structure to add said node, and to indicate the user'"'"'s interaction with the first physical object.

16. The method of claim 15 that includes storing information relating to said interaction in the data structure in the form of a semantic tuple.

17. The method of claim 15 that includes storing information relating to said interaction in the data structure in the form of resource description framework triple.

18. A method comprising the acts:

analyzing first data corresponding to imagery captured by a user'"'"'s smartphone camera, to yield identification data corresponding to a first non-human physical object depicted in the imagery, said analyzing comprising an image fingerprinting operation that includes (a) identifying candidate keypoints by convolving the first data with a blur filter to detect space-scale extrema of a scale-localized Laplacian transform of the image, (b) discerning and discarding keypoints having a contrast below a threshold value, said discerning including performing a second order Taylor expansion, and (c) checking remaining keypoints for matches with reference keypoints for reference physical objects;

issuing instructions that a social network graph data structure be updated to add node and tie elements, the node element corresponding to the first physical object, and the tie element indicating interaction by the user with said first physical object;

analyzing second data corresponding to imagery captured by the user'"'"'s smartphone camera to yield identification data corresponding to a second physical object depicted in said imagery, said analyzing comprising an image fingerprinting operation that includes (a) identifying candidate keypoints by convolving the second data with a blur filter to detect space-scale extrema of a scale-localized Laplacian transform of the image, (b) discerning and discarding keypoints having a contrast below a threshold value, said discerning including performing a second order Taylor expansion, and (c) checking remaining keypoints for matches with reference keypoints for reference physical objects;

issuing instructions that a social network graph data structure be updated to add node and tie elements, the node element corresponding to the second physical object, and the tie element indicating interaction by the user with said second physical object; and

storing rule data, the rule data establishing that the user'"'"'s interaction with the first physical object may be published to a first group of third parties, and that the user'"'"'s interaction with the second physical object may not be published to at least certain parties of said first group;

wherein said analyzing acts are performed using electronic circuitry.

19. A method comprising the acts:

using first software and electronic circuitry, analyzing first data corresponding to first content captured by a sensor in a user'"'"'s smartphone from a first non-human object, to yield first identification data for the first object, said analyzing comprising an image fingerprinting operation that includes (a) identifying candidate keypoints by convolving the first data with a blur filter to detect space-scale extrema of a scale-localized Laplacian transform of the image, (b) discerning and discarding keypoints having a contrast below a threshold value, said discerning including performing a second order Taylor expansion, and (c) checking remaining keypoints for matches with reference keypoints for reference physical objects;

issuing instructions that a social network graph data structure be updated to add node and tie elements, the node element corresponding to the first object, and the tie element indicating interaction of the user with said first object;

storing at least some first content data in association with said node;

using second software, analyzing said first content data to obtain second identification data, the second identification data being different than the first identification data; and

issuing instructions that said social network graph data structure be updated to reflect said second identification data.

20. The method of claim 19 in which the first object comprises a physical object.

21. The method of claim 19 in which the second identification data comprises digital watermark data.