Object information derived from object images

• US 9,310,892 B2
• Filed: 12/14/2014
• Issued: 04/12/2016
• Est. Priority Date: 11/06/2000
• Status: Expired due to Term

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/569,763
• Publication Number:
  US 20150205868A1
• Time in Patent Office: 485 Days
• Field of Search: 382/305, 382/100, 705 2601- 272
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A content providing server system comprising:

a memory storing a digital representation of at least one article of interest to a portable device; and

an identification server coupled with the memory and configured to;

derive salient parameters from the digital representation;

recognize at least one thing of interest as being related to the at least one article based on the salient parameters;

derive at least one query from the salient parameters, the query being related to at least one of the at least one article and the at least one thing of interest, and targeting at least one content server;

receive a results set comprising content information from the at least one content server in response to the at least one query;

process the results set to form a tailored results set including a portion of the content information and accommodating the portable device, the content information accessible via at least one information address; and

enable the portable device to access the portion of the content information in the tailored results set over a network and via the at least one information address, and to present the portion of the content information via a display.

2. The system of claim 1, wherein the at least one article comprises a magazine article.

3. The system of claim 1, wherein the at least one article comprises a newspaper article.

4. The system of claim 1, wherein the at least one article is associated with a web site.

5. The system of claim 1, wherein the at least one thing of interest is associated with on-screen content.

6. The system of claim 5, wherein the on-screen content comprises television programming.

7. The system of claim 1, wherein the at least one thing of interest comprises a promotion.

8. The system of claim 7, wherein the promotion comprises an interactive promotion.

9. The system of claim 1, wherein the at least one thing of interest comprises a product.

10. The system of claim 1, wherein the digital representation comprises a representation of alphanumeric characters.

11. The system of claim 10, wherein the salient characteristics include text information decoded from the alphanumeric characters.

12. The system of claim 1, wherein the digital representation comprises a representation of a face.

13. The system of claim 12, wherein the at least one thing of interest relates to a person associated with the face and the portion of the content information relates to the person.

14. The system of claim 1, wherein the tailored results set comprises interactive content.

15. The system of claim 1, wherein the at least one query is based at least in part on a time.

16. The system of claim 1, wherein the at least one query is based at least in part on a location.

17. The system of claim 1, wherein the at least one article comprises at least two dimensions of relevance.

18. The system of claim 17, wherein the at least one article comprises at least three dimensions of relevance.

19. The system of claim 18, wherein the at least one article comprises at least four dimensions of relevance.

20. The system of claim 1, wherein the portion of the content information in the tailored results set includes at least one of the following:

text, audio, an image, a video, and a graphic.

21. The system of claim 1, wherein the portion of the content information in the tailored results set is related to at least one of the following object attributes:

a name, a type, a color, a position, a location, and text.

22. The system of claim 1, wherein the content information comprises an image in JPEG format.

23. The system of claim 1, wherein the at least one information address comprises a URL.

24. The system of claim 1, wherein the at least one information address comprises a network address.

25. The system of claim 1, wherein the portion of the content information in the tailored result set is configured to enable the portable device to perform an action.

26. The system of claim 25, wherein the action comprises a transaction.

27. The system of claim 26, wherein the action comprises an HTTP transaction.

28. The system of claim 26, wherein the transaction comprises a commercial transaction.

29. The system of claim 26, wherein the transaction is associated with at least one of the following accounts:

an account linked to a user, an account liked to the portable device, a bank account, and a credit card.

30. The system of claim 25, wherein the action comprises accessing a web site via the at least one information address.

31. The system of claim 25, wherein the action comprises playing at least one of the following:

streaming audio, and streaming video.

32. The system of claim 25, wherein the action comprises the portable device speaking aloud text information associated with the thing of interest.

33. The system of claim 1, wherein the digital representation of the at least one article comprises an image of the at least one article.

34. The system of claim 1, wherein the portable device comprises at least one of the following:

a cell phone and a digital assistant.

35. The system of claim 1, wherein the network comprises at least one of the following:

the Internet, a cellular network, and a wireless network.

36. The system of claim 1, wherein the portion of the content information in the tailored results set is a whittled down version of the content information in the results set.

37. The system of claim 1, wherein the tailored result set comprises a file that includes the at least one information address.

38. The system of claim 37, wherein the identification server is further configured to send the file to the portable device over the network.

39. The system of claim 1, wherein the identification server is further configured to translate some of the tailor results set.

40. The system of claim 1, wherein the identification server is further configured to convert the portion of the content information in the tailored result set to another form.