Object information derived from object images

• US 8,520,897 B2
• Filed: 12/04/2012
• Issued: 08/27/2013
• Est. Priority Date: 11/06/2000
• Status: Expired due to Term

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/705,071
• Publication Number:
  US 20130094708A1
• Time in Patent Office: 266 Days
• Field of Search: 382/181, 382/100, 382/103, 382/224, 382/165, 705 261- 271, 705/7.33, 705/1.1, 705/43, 3482112-2116, 348/207.1, 348/150, 348/239, 348/460, 348/552, 455/414.3, 455/411, 455/414.2, 455/412.1, 4554561-4562, 235/380, 713/186, 713/168
• US Class Current: 382/103
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. An interactive system comprising:

an optical sensor configured to capture image information representative of a displayed image on a display screen; and

an image processing platform coupled with the optical sensor and configured to;

identify features from the image information relating to the displayed image;

recognize a target based on the features;

associate the target with target information pertinent to the target; and

perform a transaction with an account based on the target information.

2. The system of claim 1, wherein the image information is representative of a static image as the displayed image.

3. The system of claim 1, wherein the image information is representative of a time varying image as the displayed image.

4. The system of claim 1, wherein the image information is representative of a pattern within the displayed image.

5. The system of claim 1, wherein the image information is representative of an identification code the displayed image comprises an identification code.

6. The system of claim 5, wherein the identification code comprises a symbol.

7. The system of claim 6, wherein the symbol comprises a bar code.

8. The system of claim 6, wherein the symbol comprises an alpha-numeric character.

9. The system of claim 1, wherein image processing platform is disposed within a portable device.

10. The system of claim 1, wherein the image processing platform is disposed within a server.

11. The system of claim 1, wherein the image processing platform is disposed within automated sales machine.

12. The system of claim 1, wherein the image processing platform is disposed within a vending machine.

13. The system of claim 1, wherein the optical sensor comprises a camera.

14. The system of claim 1, wherein the target comprises an object.

15. The system of claim 1, wherein the target comprises an identification code.

16. The system of claim 15, wherein the identification code comprises a symbol.

17. The system of claim 16, wherein the symbol comprises a bar code.

18. The system of claim 16, wherein the symbol comprises an alpha-numeric character.

19. The system of claim 1, wherein the transaction comprises a payment to the account.

20. The system of claim 1, wherein the account comprises at least one of the following types of accounts:

an on-line account, an account linked with the portable device, an account linked to a user of the portable device, a bank account, and a credit card account.

21. The system of claim 1, wherein the transaction comprises a commercial transaction.

22. The system of claim 1, wherein the transaction is associated with a target item.

23. The system of claim 21, wherein the target item comprises at least one of the following:

a video stream, an audio stream, an audio recording, a video recording, a ticket, an entry ticket, and a vending machine product.

24. The system of claim 21, wherein the transaction comprises a purchase of the target item.

25. A method of conducting a transaction with an interactive system, the method comprising

providing access to a device having a display;

displaying a displayed image on the display of the device;

presenting the displayed image proximate to an optical sensor; and

enabling, via the interactive system, identification of features from the displayed image, recognition of a target based on the features, association of the target with target information pertinent to the target, and performance of a transaction based on the target information.

26. The method of claim 25, further comprising receiving, by the device, information from the interactive system in response to presenting the display image.

27. The method of claim 26, where in the information comprises interactive information.

28. The method of claim 25, wherein the step of displaying the displayed image includes displaying a pattern.

29. The method of claim 25, wherein the step of displaying the displayed image includes displaying a static image.

30. The method of claim 25, wherein the step of displaying the displayed image includes displaying a time varying image.

31. The method of claim 25, wherein the step of displaying the displayed image includes displaying an identifier code.

32. The method of claim 31, wherein the identifier code comprises at least one of the following:

a bar code, a matrix code, a 2D code, a 3D code, a recognizable symbol, an alphanumeric characters, and a logo.

33. The method of claim 25, wherein the step of enabling performance of the transaction includes initiating a verification.

34. The method of claim 25 wherein the step of enabling performance of the transaction includes initiating an authorization.

35. The method of claim 25, wherein the step of enabling performance of the transaction includes initiating a billing.

36. The method of claim 25, wherein the step of enabling performance of the transaction includes initiating a tracking of inventory.

37. The method of claim 25, wherein the step of enabling performance of the transaction includes initiating a payment.

38. The method of claim 25, wherein the step of enabling performance of the interaction includes initiating a transaction with an account.

39. The method of claim 38, wherein the account includes at least one of the following:

an on-line account, an account linked with the portable device, an account linked to a user of the portable device, a bank account, and a credit card account.

40. The method of claim 25, wherein the step of displaying the image includes displaying the image on a luminescent display screen.

41. The method of claim 25, wherein the device comprises a portable device.

42. The method of claim 41, wherein the portable device comprises a cellular phone.

43. The method of claim 25, wherein the step of presenting the displayed image proximate to the optical sensor includes presenting the display to a gas pump.

44. The method of claim 25, wherein the step of presenting the displayed image proximate to the optical sensor includes presenting the display to a vending machine.

45. The method of claim 25, wherein the step of presenting the displayed image proximate to the optical sensor includes presenting the display to an automated teller machine.

46. The method of claim 25, wherein the step of presenting the displayed image proximate to the optical sensor includes presenting the display to a portable device.

47. The method of claim 25, further comprising configuring a second device to operate as the interactive system.

48. The method of claim 47, wherein the second device comprises at least one of the following:

a cell phone, a vending machine, a portable device, a gasoline pump, and an automated teller machine.