Object information derived from object images

• US 8,849,069 B2
• Filed: 04/26/2013
• Issued: 09/30/2014
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/872,032
• Publication Number:
  US 20130246256A1
• Time in Patent Office: 522 Days
• Field of Search: 382/181, 382/100, 382/103, 382/224, 382/165, 382/305, 705 261- 271, 705/7.33, 705/1.1, 705/43, 3482112-2116, 348/207.1, 348/150, 348/239, 348/460, 348/552, 4554143-411, 455/414.2, 455/412.1, 4554561-4562, 713/186, 713/168, 235/380
• US Class Current: 382/305
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. An information processing system comprising:

a device connection configured to acquire a digital representation of a container from a device; and

a processing server coupled with the device connection and configured to;

receive the digital representation from the device over the device connection;

discriminate whether the digital representation of the container includes a representation of a recognizable identifying marking;

obtain container identifying information from the identifying markings, the container identifying information identifying the container as a specific target object;

obtain first target object information based on the container identifying information; and

cause a second device to initiate a transaction as a function of the first target object information.

2. The system of claim 1, wherein the processing server is further configured to obtain the first target object information over at least one of the following networks:

a cellular telephone network, a wireless network, the Internet, and wire line network.

3. The system of claim 1, wherein the container comprises a beverage container.

4. The system of claim 1, wherein the identifying marking comprises at least one of the following:

a label, a printing, a logo, a text, and a recognizable symbol.

5. The system of claim 4, wherein the recognizable symbol comprises at least one of the following:

a barcode, a matrix code, and alphanumeric characters.

6. The system of claim 1, wherein the digital representation comprises at least one of image data and sound data.

7. The system of claim 1, wherein the processing server is further configured to:

determine that the digital representation provides insufficient information for identifying the container;

send a message to the mobile device; and

direct a user of the mobile device to acquire another digital representation of the container.

8. The system of claim 1, wherein the second device is the mobile device.

9. The system of claim 1, wherein the transaction comprises a purchase related to the container.

10. The system of claim 1, wherein the transaction comprises an on-line transaction.

11. The system of claim 1, wherein the transaction relates to an account linked to a user of the mobile device.

12. The system of claim 11, wherein the account comprises at least one of a bank account and a credit card.

13. The system of claim 1, wherein the processing server is further configured to:

obtain container identifying information based on the digital representation of the container, wherein container identifying information comprises a search term;

send a query to a search engine using the search term; and

obtain second target object information based on search results from the search engine.

14. The system of claim 13, wherein the processing server is further configured to:

link to a second server based on at least one of the first and second target object information;

transmit the second target object information to the second server; and

cause the second server to initiate a transaction as a function of the second target object information.

15. A mobile device transaction system comprising:

a mobile device comprising an image sensor programmed to acquire a digital representation of a container; and

an image processing platform within the mobile device and programmed to;

receive the digital representation;

discriminate whether the digital representation of the container includes a recognizable identifying marking;

enable decoding of container identifying information from the recognizable identifying markings, the container identifying information identifying the container as a specific target object;

identify first target object information based on the container identifying information; and

cause the mobile device to initiate a transaction as a function of the first target object information.

16. The system of claim 15, wherein the image processing platform is further configured to obtain the first target object information over at least one of the following:

a cellular telephone network, a wireless network, the Internet, and wire line network.

17. The system of claim 15, wherein the container comprises a beverage container.

18. The system of claim 15, wherein the recognizable identifying marking comprises at least one of the following:

a label, a printing, a logo, a text, and a recognizable symbol.

19. The system of claim 18, wherein the recognizable identifying marking comprises a recongnizable symbol, and wherein the recognizable symbol comprises at least one of the following:

a barcode, a matrix code, and alphanumeric characters.

20. The system of claim 15, wherein the digital representation comprises at least one of image data and sound data.

21. The system of claim 15, wherein the image processing platform is further configured to:

determine that the digital representation provides insufficient information for identifying the container;

send a message to the mobile device; and

direct a user of the mobile device to acquire another digital representation of the container.

22. The system of claim 15, wherein the transaction comprises a purchase related to the container.

23. The system of claim 15, wherein the transaction comprises an on-line transaction.

24. The system of claim 15, wherein the transaction relates to an account linked to a user of the mobile device.

25. The system of claim 24, wherein the account comprises at least one of a bank account and a credit card.

26. The system of claim 15, wherein the image processing platform is further configured to:

obtain container identifying information based on the digital representation of the container, wherein the container identifying information comprises a search term;

send a query to a search engine using the search term; and

obtain second target object information based on search results from the search engine.

27. The system of claim 26, wherein the image processing platform is further configured to:

link to a second platform based on second target object information;

transmit the second target object information to the second platform; and

cause the second platform to initiate a transaction as a function of the second target object information.

28. A computer program product comprising a non-transitory computer readable storage medium storing instructions that cause a processor to implement the steps of:

configuring a device to operating as an image processing platform;

receiving, by the device, a digital representation of a container;

discriminating, by the device, whether the digital representation of the container includes a recognizable identifying marking;

obtaining container identifying information from the recognizable identifying marking, the container identifying information identifying the container as a specific target object;

obtaining first target object information based on the container identifying information; and

causing a second device to initiate a transaction as a function of the first target object information.

29. The computer program product of claim 28, wherein the step of obtaining the first target object information comprises receiving the target object information over at least one of the following networks:

a cellular telephone network, a wireless network, the Internet, and wire line network.

30. The computer program product of claim 28, wherein the instructions further cause the processor to implement the step of identifying the container as a beverage container.

31. The computer program product of claim 28, wherein the identifying marking comprises at least one of the following:

a label, a printing, a logo, a text, and a recognizable symbol.

32. The computer program product of claim 28, wherein the step of discriminating includes identifying the identifying marking as at least one of the following:

a barcode, a matrix code, and alphanumeric characters.

33. The computer program product of claim 28, wherein the digital representation comprises at least one of image data and sound data.

34. The computer program product of claim 28, wherein the instructions further cause the processor to implement the steps of:

determining that the digital representation provides insufficient information for identifying the container;

sending a message to the mobile device; and

directing a user of the mobile device to acquire another digital representation of the container.

35. The computer program product of claim 28, wherein the second device is the mobile device.

36. The computer program product of claim 28, wherein the transaction comprises a purchase related to the container.

37. The computer program product of claim 28, wherein the transaction comprises an on-line transaction.

38. The computer program product of claim 28, wherein the transaction relates to an account linked to a user of the mobile device.

39. The computer program product of claim 38, wherein the account comprises at least one of a bank account and a credit card.

40. The computer program product of claim 28, wherein the instructions further cause the processor to implement the steps of:

obtaining container identifying information based on the digital representation of the container, wherein the container identifying information comprises a search term;

sending a query to a search engine using the search term; and

obtaining second target object information based on search results from the search engine.

41. The computer program product of claim 40, wherein the instructions further cause the processor to implement the steps of:

linking to a second platform based on second target object information;

transmitting the second target object information to the second platform; and

causing the second platform to initiate a transaction as a function of the second target object information.

42. A method comprising:

configuring a mobile device to operate as an image processing recognition platform;

acquiring, by the mobile device, a digital representation of a container;

recognizing, by the mobile device, the container as a target object based on the digital representation;

obtaining first target object information via an information address associated with the target object;

discriminating, by mobile the device, whether the digital representation of the container includes a recognizable identifying marking;

obtaining container identifying information based on the recognizable identifying markings and based on the first target object information, wherein the container identifying information identifies the container as discriminated from other similar target objects; and

causing a second device to initiate a transaction as a function of the obtained container identifying information.

43. The method of claim 42, wherein obtaining the first target object information comprises receiving the first target object information over at least one of the following networks:

a cellular telephone network, a wireless network, the Internet, and wire line network.

44. The method of claim 42, further comprising identifying the container as a beverage container.

45. The method of claim 42, wherein the recognizable identifying marking comprises at least one of the following:

a label, a printing, a logo, a text, and a recognizable symbol.

46. The method of claim 45, wherein discriminating includes identifying the recognizable identifying marking as at least one of the following:

a barcode, a matrix code, and alphanumeric characters.

47. The method of claim 42, wherein the digital representation comprises at least one of image data and a sound data.

48. The method of claim 42, further comprising:

determining that the digital representation provides insufficient information for identifying the container;

sending a message to the mobile device; and

directing a user of the mobile device to acquire another digital representation of the container.

49. The method of claim 42, wherein the second device is the mobile device.

50. The method of claim 42, wherein the transaction comprises a purchase related to the container.

51. The method of claim 42, wherein the transaction comprises an on-line transaction.

52. The method of claim 42, wherein the transaction relates to an account linked to a user of the mobile device.

53. The method of claim 52, wherein the account comprises at least one of a bank account and a credit card.

54. The method of claim 42, further comprising:

obtaining at least a portion of the container identifying information based on the digital representation of the container, wherein the container identifying information comprises a search term;

sending a query to a search engine using the search term; and

obtaining second target object information based on search results from the search engine.

55. The method of claim 54, further comprising:

linking to a second platform based on the second target object information;

transmitting the second target object information to the second platform; and

causing the second platform to initiate a transaction as a function of the second target object information.