Object information derived from object images

• US 9,152,864 B2
• Filed: 02/24/2014
• Issued: 10/06/2015
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/187,717
• Publication Number:
  US 20140172575A1
• Time in Patent Office: 589 Days
• Field of Search: 382/100, 382/103, 382/165, 382/181, 382/305, 705 261- 272
• US Class Current: 1/1
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A method of providing information related to a provider of goods or services, the method comprising:

obtaining, via a mobile device, a digital representation of a scene including at least one object related to a first goods or services provider;

recognizing, via at least one of the mobile device and a processor, the at least one object as a target object from the digital representation;

identifying, via the processor, a result set comprising a plurality of goods or services providers proximate in location to the mobile device based on the target object;

determining, via the processor, which of the plurality of goods or services providers proximate in location to the mobile device are consistent with a first user preference to generate a customized result set associated with at least one goods or services provider from the plurality of goods or services providers; and

presenting, via a display and an address, at least one indication associated with the customized result set and the target object.

2. The method of claim 1, wherein the digital representation comprises video imagery.

3. The method of claim 2, wherein the video imagery comprises real-time video imagery.

4. The method of claim 1, wherein the digital representation includes a symbol that identifies the at least one object.

5. The method of claim 1, further comprising receiving, via the mobile device, at least one of an identification tag, a barcode, a transceiver and a transmitter associated with the at least one goods or services provider.

6. The method of claim 1, wherein identifying the result set comprises determining, via the processor, a location of the mobile device, and wherein determining which of the plurality of goods or services providers are consistent with the first user preference is based at least in part on the location of the mobile device.

7. The method of claim 1, wherein the mobile device comprises a mobile phone.

8. The method of claim 3, wherein the real-time video imagery is captured by the mobile device, and wherein the display composes the mobile device.

9. The method of claim 1, wherein determining which of the plurality of goods or services provider are consistent with the first user preference further comprises obtaining information related to a first user.

10. The method of claim 1, wherein the digital representation includes representation of at least one of a symbol, a telephone number, a website address, a business sign, and an advertisement.

11. The method of claim 9, wherein obtaining information related to the first user comprises obtaining account information associated with the first user.

12. The method of claim 1, wherein determining which of the plurality of goods or services providers proximate in location to the mobile device are consistent with the first user preference comprises obtaining, via the mobile device, a first user preference, and querying a database storing information related to the at least one object based on the first user preference.

13. The method of claim 9, wherein obtaining information related to the first user comprises obtaining user location information.

14. The method of claim 1, further comprising, presenting, via the display, an option to initiate a transaction with the at least one goods or services provider.

15. The method of claim 14, wherein the transaction comprises a purchase related to the at least one object.

16. The method of claim 1, wherein the at least one object comprises at least one of a billboard and a sign associated with the first goods or services provider.

17. The method of claim 11, wherein the account information is related to at least one of a bank account and a credit card.

18. The method of claim 15, further comprising billing an account of a first user for the purchase.

19. The method of claim 1, wherein presenting the at least one indication comprises presenting a menu of options.

20. The method of claim 19, further comprising enabling, via the processor, a user selection from the menu of options.

21. The method of claim 20, further comprising, upon selection by a user, presenting the user with information related to the first goods or services provider.

22. The method of claim 20, further comprising, upon selection by a user, presenting the user with an option to purchase the target object.

23. An apparatus comprising:

a mobile device configured to capture a digital representation that includes at least one image of an object related to at least one good or service provider; and

recognize the at least one object as a target object from the digital representation;

a processor communicatively coupled to the mobile device and configured to identify a first goods or services provider of the at least one goods or services provider as consistent with a first user preference; and

a display communicatively coupled to the processor and configured to present at least one indication related to the target object and associated with an address and the first goods or services provider.

24. The apparatus of claim 23, wherein the digital representation comprises video imagery.

25. The apparatus of claim 23, wherein the mobile device comprises a mobile telephone.

26. The apparatus of claim 23, wherein the processor is further configured to obtain information related to a first user.

27. The apparatus of claim 24, wherein the video imagery comprises real-time video imagery.

28. The apparatus of claim 23, wherein the digital representation includes a representation of a symbol that identifies the at least one object.

29. The apparatus of claim 23, wherein the digital representation includes a representation of at least one of an identification tag, a barcode, a transceiver and a transmitter associated with the at least one goods or services provider.

30. The apparatus of claim 23, wherein the processor is further configured to obtain a location of the mobile device, and identify the first goods or services provider as consistent with the first user preference based at least in part on the location.

31. The apparatus of claim 23, wherein the mobile device comprises the display.

32. The apparatus of claim 23, wherein at least one of the mobile device and the processor is configured to obtain information related to the first user.

33. The apparatus of claim 32, wherein the information related to the first user comprises account information.

34. The apparatus of claim 33, wherein the account information is related to at least one of a bank account and a credit card.

35. The apparatus of claim 23, wherein the at least one indication comprises a menu of options.

36. The apparatus of claim 35, wherein the menu of options comprises options that are user selectable.

37. The apparatus of claim 36, wherein the processor is further configured to present information related to the first goods or services provider upon a user selection from the menu of options.

38. The apparatus of claim 36, wherein the processor is further configured to enable a purchase of the target object upon a user selection from the menu of options.

39. A computer program product, comprising:

a non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform the steps of;

obtaining, via a mobile device, a digital representation including imagery of an object related to at least one goods or services provider;

recognizing the object as a target object based on the digital representation;

determining that a first goods or services provider from the at least one goods or services provider is consistent with a first user preference; and

presenting, via a display, an indication associated with the first goods or services provider, wherein the indication includes at least one information address related to the target object.

40. The computer program product of claim 39, wherein the instructions, when executed by the processor, further cause the processor to perform the step of obtaining, via the mobile device, information related to first user.

41. The computer program product of claim 39, wherein the instructions, when executed by the processor, further cause the processor to perform the steps of obtaining, via the mobile device, a location of the mobile device, and determining that the first goods or services provider is consistent with the first user preference based at least in part on the location.

42. The computer program product of claim 40, wherein the information related to the first user comprises account information.

43. The computer program product of claim 39, wherein the digital representation includes a representation of a symbol that identifies the at least one object.

44. The computer program product of claim 43, wherein the symbol comprises at least one of an identification tag, a barcode, a matrix code, and alphanumeric characters associated with the at least one goods or services provider.

45. The computer program product of claim 39, wherein the mobile device comprises the display.

46. The computer program product of claim 39, wherein the digital representation comprises video imagery.

47. The computer program product of claim 39, wherein the wherein digital representation comprises real-time video imagery.

48. The computer program product of claim 42 wherein account information is related to at least one of a bank account and a credit card.

49. The computer program product of claim 39, wherein the at least one indication comprises a menu of options.

50. The computer program product of claim 49, wherein the menu of options comprises options that are user selectable.

51. The computer program product of claim 50, wherein the instructions, when executed by the processor, further cause the processor to perform the step of presenting, via the display, information related to the first goods or services provider upon a user selection from the menu of options.

52. The computer program product of claim 50, wherein the instructions, when executed by the processor, further cause the processor to perform the step of enabling a purchase of the target object upon a user selection from the menu of options.

53. The computer program product of claim 39, wherein the mobile device comprises a mobile telephone.