Object information derived from object images

• US 9,036,862 B2
• Filed: 03/03/2014
• Issued: 05/19/2015
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/195,768
• Publication Number:
  US 20140177922A1
• Time in Patent Office: 442 Days
• Field of Search: 382/100, 382/103, 382/305, 382/181, 382117-118, 705 261- 272
• US Class Current: 382/100
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. An image-based transaction system comprising:

a mobile device image sensor configured to capture image data that represents a face; and

an image processing device coupled with the image sensor and configured to;

recognize the face as a known target object from a known object database based on features derived from the image data;

retrieve target object information from an information indexing system, wherein the target object information is associated with the known target object and an address, and includes an identity of a specific known target object associated with the face; and

enable a device to initiate a transaction based on the target object information, using at least one of the address and the identity, wherein the transaction relates to the specific known target object.

2. The system of claim 1, wherein the transaction comprises an on-line transaction.

3. The system of claim 1, wherein the transaction comprises a protocol transaction.

4. The system of claim 3, wherein the protocol transaction comprises an HTTP transaction.

5. The system of claim 3, wherein the protocol transaction comprises a wireless network protocol transaction.

6. The system of claim 1, wherein the transaction comprises a transaction with an account.

7. The system of claim 6, wherein the account comprises at least one of the following:

an account linked to the device, an account linked to a user, a bank account, and a credit card account.

8. The system of claim 1, wherein the transaction comprises an interaction with a promotion.

9. The system of claim 8, wherein the promotion comprises at least one of the following:

a product promotion, a reduced price promotion, a television programming promotion, a limited time promotion, and a promotion linked to a web site.

10. The system of claim 1, wherein the transaction comprises a purchase.

11. The system of claim 1, wherein the image sensor comprises a camera.

12. The system of claim 11, wherein the camera comprises a digital camera.

13. The system of claim 11, wherein the camera comprises a video camera.

14. The system of claim 1, wherein the image data includes at least one of the following:

a single image, a series of images, a video, and a video stream.

15. The system of claim 1, wherein the image processing device comprises a mobile device.

16. The system of claim 15, wherein the mobile device comprises at least one of the following:

a vehicle, a cellular telephone, a digital camera, a video camera, a personal data assistant, and a security hand-held device.

17. The system of claim 1, wherein the image processing device comprises a server.

18. The system of claim 17, wherein the server comprises a distal server relative to the image sensor.

19. The system of claim 1, wherein the device is the image processing device.

20. The system of claim 1, wherein the device includes at least one of the following:

a point of sales terminal, a vending machine, a content server, a search engine, a distal server, a cellular telephone, a personal data assistant, and a television.

21. The system of claim 1, wherein the image data represents the face as an animated object.

22. The system of claim 21, wherein the animated object is representative of at least one of multiple faces of people.

23. The system of claim 1, wherein the face comprises a person'"'"'s face.

24. The system of claim 1, wherein the image processing device is further configured to enable the device to initiate the transaction using a location.

25. The system of claim 24, wherein the location comprises at least one of the following:

a location of a person, a vehicle location, a physical location, a device location, a global positioning system location, a user location, and a vendor location.

26. The system of claim 1, wherein the image processing device is further configured to enable the device to initiate the transaction using a time.

27. The system of claim 26, wherein the time includes a time the image data was captured.

28. The system of claim 1, wherein the features is associated with a tag.

29. The system of claim 1, wherein the features is associated with a lack of a tag.

30. The system of claim 1, wherein the target object information includes a code and wherein the device is enabled to initiate the transaction using the code.

31. The system of claim 1, wherein the target object information includes at least one of a position, an orientation, an identity and a telephone number.

32. The system of claim 1, wherein the target object information comprises a search query results set.

33. The system of claim 32, wherein the results set includes at least one of the following:

a URL, a URI, an IP address, a telephone number, a channel, a frequency, and a location.

34. The system of claim 1, wherein the features comprise at least one of the following:

identifying marks, and image characteristics.

35. The system of claim 1, wherein the address includes at least one of the following:

a URL, a URI, a network address, an IP address, a web-site address, a telephone number, an on-line address, a computing device address, and an information address.

36. The system of claim 1, wherein the transaction comprises a vote registration.