Object information derived from object images

• US 8,792,750 B2
• Filed: 04/08/2013
• Issued: 07/29/2014
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/858,897
• Publication Number:
  US 20130230210A1
• Time in Patent Office: 477 Days
• Field of Search: 382/100, 382/103, 382/165, 382/181, 382/305, 705 261- 272
• US Class Current: 382/305
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A method of providing information comprising:

recognizing, via a mobile communication device, an object associated with a business, wherein the object is local to the mobile communication device;

retrieving, via the mobile communication device, object information including an address related to the object associated with the business; and

displaying within a display of the mobile communication device an information locator that provides access to content information via the address, the information locator linked to the object associated with the business.

2. The method of claim 1, further comprising capturing, by the mobile communication device, images of the object associated with the business.

3. The method of claim 2, further comprising conducting, via a computing device, object recognition processing to the images to identify the images as corresponding to the object associated with the business.

4. The method of claim 2, further comprising detecting, via a computing device, a code on the object associated with the business;

and decoding the code to help identify the object associated with the business.

5. The method of claim 1, further comprising wirelessly receiving, at the mobile communication device, the content information, wherein the content information includes an identity related to the object associated with the business.

6. The method of claim 1, further comprising determining, via a computing device, a location of the mobile communication device and identifying an identity associated with the object associated with the business based at least in part on the location.

7. The method of claim 1, wherein the step of retrieving the object information includes retrieving the content information in real-time.

8. The method of claim 1, wherein the step of retrieving the object information includes retrieving the object information associated with news.

9. The method of claim 1, wherein the content information comprises user selectable links to on-line purchasable products.

10. The method of claim 1, wherein the content information comprises an identity of a person.

11. The method of claim 1, further comprising communicating, via the mobile communication device, an indication that the content information is available.

12. The method of claim 11, wherein the step of communication, via the mobile communication device, that the content information is available includes tailoring the indication that that content information is available.

13. The method of claim 1, further comprising allowing a user to interact, at the mobile communication device, with the locator to access the content information.

14. The method of claim 13, further comprising, at the mobile communication device, initiating an on-line transaction in response to the interaction with the locator.

15. The method of claim 13, further comprising, at the mobile communication device, initiating an interaction with at least one of the following types of data:

audio data, video data, and text data.

16. The method of claim 1, further comprising displaying a video stream associated within the information locator.

17. An apparatus for providing content information comprising:

a mobile device computing platform having a memory and at least one processor in communication with the memory;

an object recognition application stored in the memory, executable by the processor and configured to recognize an object associated with a business, wherein the object is local to a mobile communication device;

a content information application stored in the memory, executable by the processor and configured to retrieve object information including an address related to the object associated with the business; and

a display application stored in the memory, executable by the processor and configured to display within a display of the mobile communication device an information locator that provides access to content information via the address, the information locator linked to the object associated with the business.

18. The apparatus of claim 17, wherein the object recognition application is further configured to recognize an object associated with the business based on captured images of the object associated with the business.

19. The apparatus of claim 18, wherein the object recognition application is further configured to conduct object recognition processing on the images to identify the images as corresponding to the object associated with the business.

20. The apparatus of claim 18, wherein the object recognition application is further configured to detect a code on the object associated with the business;

and decoding the code to help identify the object associated with the business.

21. The apparatus of claim 17, wherein the object recognition application is further configured to wirelessly receive the content information, wherein the content information includes an identity related to the object associated with the business.

22. The apparatus of claim 17, wherein the object recognition application is further configured to determine a location of the mobile communication device and identify an identity associated with the object associated with the business based on the location.

23. The apparatus of claim 17, wherein the content information application is further configured to retrieve the object information in real-time.

24. The apparatus of claim 17, wherein the content information application is further configured to retrieve the object information associated with news.

25. The apparatus of claim 17, wherein the content information comprises user selectable links to on-line purchasable products.

26. The apparatus of claim 17, wherein the content information comprises an identity of a person.

27. The apparatus of claim 17, wherein the content information application is further configured to provide an indication that the content information is available.

28. The apparatus of claim 27, wherein the content information application is further configured to tailor the indication that the content information is available.

29. The apparatus of claim 17, wherein the content information application is further configured to allow a user to have an interaction with the locator to access the content information.

30. The apparatus of claim 29, wherein the interaction includes initiating an on-line transaction.

31. The apparatus of claim 29, wherein the interaction includes interaction with at least one of the following types of data:

audio data, video data and text data.

32. The apparatus of claim 17, wherein the display application is configured to display a video stream associated within the information locator.

33. A computer program product, the computer program product comprising a non-transitory computer-readable medium having computer executable instructions to cause a mobile device to implement the steps of:

recognizing, via a mobile communication device, an object associated with a business, wherein the object is local to the mobile communication device;

retrieving, via the mobile communication device, object information including an address related to the object associated with the business; and

displaying within a display of the mobile communication device an information locator that provides access to the content information via an address, the information locator linked to the object associated with the business.

34. The computer program product of claim 33, further comprising the step of capturing, by the mobile communication device, images of the object associated with the business.

35. The computer program product of claim 34, further comprising the step of conducting, via a computing device, object recognition processing to the images to identify the images as corresponding to the object associated with the business.

36. The computer program product of claim 34, further comprising the step of detecting, via a computing device, a code on the object associated with the business;

and decoding the code to help identify the object associated with the business.

37. The computer program product of claim 33, further comprising the step of wirelessly receiving, at the mobile communication device, the content information, wherein the content information includes an identity related to the object associated with the business.

38. The computer program product of claim 33, further comprising the step of determining, via a computing device, a location of the mobile communication device and identifying an identity associated with the object associated with the business based on the location.

39. The computer program product of claim 33, wherein the step of retrieving the object information includes the step of retrieving the content information in real-time.

40. The computer program product of claim 33, wherein the step of retrieving the object information includes the step of retrieving the object information associated with news.

41. The computer program product of claim 33, wherein the content information comprises user selectable links to on-line purchasable products.

42. The method of claim 33, wherein the content information comprises an identity of a person.

43. The computer program product of claim 33, further comprising the step of communicating, via the mobile communication device, an indication that the content information is available.

44. The computer program product of claim 43, wherein the step of communication, via the mobile communication device, that the content information is available includes the step of tailoring the indication that the content information is available.

45. The computer program product of claim 33, further comprising the step of allowing a user to interact, at the mobile communication device, with the locator to access the content information.

46. The computer program product of claim 45, further comprising, at the mobile communication device, the step of initiating an on-line transaction in response to the interaction with the locator.

47. The computer program product of claim 45, further comprising, at the mobile communication device, the step of initiating an interaction with at least one of the following types of data:

audio data, video data and text data.

48. The computer program product of claim 33, further comprising the step of displaying a video stream associated within the information locator.