Object information derived from object images

• US 9,014,516 B2
• Filed: 02/26/2014
• Issued: 04/21/2015
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/191,355
• Publication Number:
  US 20140177918A1
• Time in Patent Office: 419 Days
• Field of Search: 382/100, 382/103, 382/181, 382/305, 705 261- 272
• US Class Current: 382/305
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A method for providing target object information, the method comprising:

obtaining, via a mobile communication device, a video stream including a representation of at least a portion of a plurality of objects;

identifying, via a computing device processor, at least one object of the plurality of objects as a first target object based at least in part on the video stream and a query of a database indexing data associated with a plurality of target objects including the first target object;

presenting, via an information address and on a display of the mobile communication device, an indication related to the first target object based at least in part on the query of the database.

2. The method of claim 1, wherein identifying the at least one object as the first target object comprises identifying one or more images in the video stream that correspond to one or more of the plurality of target objects.

3. The method of claim 1, wherein identifying the at least one object as the first target object comprises identifying one or more objects represented in the video stream that corresponds to one or more of the plurality of target objects.

4. The method of claim 1, wherein identifying the at least one object as the first target object is further based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

5. The method of claim 1, wherein the indication is related to biometric information.

6. The method of claim 1, wherein identifying the at least one object as the first target object comprises comparing information related to the video stream to the data indexed by the database.

7. The method of claim 6, wherein the indication is associated with a retail store related to the first target object.

8. The method of claim 1, wherein the indication is customized based on a user preference.

9. The method of claim 1, further comprising obtaining information related to a user preference, and populating the database with the information related to the user preference.

10. The method of claim 1, further comprising populating the database with information related to a user preference, and presenting a customized result set including the indication based at least in part on the user preference.

11. The method of claim 1, wherein presenting the indication comprises sending an associated file to the mobile communication device.

12. An apparatus for providing information related to a target object, the apparatus comprising:

a mobile communication device having a processor, a memory in communication with the processor;

image capture software stored in the memory, executable by the processor, and configured to capture at least one video stream including a representation of at least a portion of an object;

object identification software stored in the memory, executable by the processor and configured to identify the object as a first target object based at least in part on the video stream and a query of a database indexing data associated with a plurality of target objects including the first target object; and

target object information presentation software stored in the memory, executable by the processor and configured to present, via an information address and on a display of the mobile communication device, an indication related to the first target object based at least in part on the query of the database.

13. The apparatus of claim 12, wherein the object identification software is further configured to identify one or more images in the video stream that corresponds to one or more of the plurality of target objects.

14. The apparatus of claim 12, wherein the object identification software is further configured to identify one or more objects represented in the video that corresponds to one or more of the plurality of target objects.

15. The apparatus of claim 12, wherein the object identification software is further configured to identify the object as the first target object based at least in part on global positioning system information associated with them mobile communication device or other information indicating a location of the mobile communication device.

16. The apparatus of claim 12, wherein the indication is related to biometric information.

17. The apparatus of claim 12, wherein the target object information presentation software is configured to compare information related to the video stream to the data indexed by the database.

18. The apparatus of claim 17, wherein the indication is associated with a retail store related to the target object.

19. The apparatus of claim 12, wherein the indication is customized based on a user preference.

20. The apparatus of claim 12, wherein the object identification software is further configured to obtain information related to a user preference, and populate the database with the information related to the user preference.

21. The apparatus of claim 12, wherein the object identification software is further configured to populate the database with information related to the user preference, and wherein the target object information presentation software is further configured to present a customized result set including the indication based at least in part on the user preference.

22. The apparatus of claim 12, wherein the target object information presentation software is further configured to present the indication by sending an associated file to the mobile communication device.

23. A computer program product comprising a memory comprising computer-executable software for providing target object information, the software comprising:

software for obtaining, via a mobile communication device, a video stream including a representation of at least a portion of a plurality of objects;

software for identifying, via a computing device processor, at least one object of the plurality of objects as a first target object based at least in part on the video stream and a query of a database indexing data associated with a plurality of target objects including the first target object;

software for presenting, via an information address and on a display of the mobile communication device, an indication related to the first target object based at least in part on the query of the database.

24. The computer program product of claim 23, wherein identifying the at least one object as the first target object comprises identifying one or more images in the video stream that correspond to one or more of the plurality of target objects.

25. The computer program product of claim 23, wherein identifying the at least one object as the first target object comprises identifying one or more objects represented in the video stream that corresponds to one or more of the plurality of target objects.

26. The computer program product of claim 23, wherein identifying the at least one object as the first target object is further based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

27. The computer program product of claim 23, wherein the indication is related to biometric information.

28. The computer program product of claim 23, wherein identifying the at least one object as the first target object comprises comparing information related to the video stream to the data indexed by the database.

29. The computer program product of claim 28, wherein the indication is associated with a retail store related to the target object.

30. The computer program product of claim 23, wherein the indication is customized based on a user preference.

31. The computer program product of claim 23, further comprising software for obtaining information related to a user preference, and populating the database with the information related to the user preference.

32. The computer program product of claim 23, further comprising software for populating the database with information related to a user preference, and presenting a customized result set including the indication based at least in part on the user preference.

33. The computer program product of claim 23, wherein presenting the indication comprises sending an associated file to the mobile communication device.

34. A method for providing target object information, the method comprising:

obtaining, via a mobile communication device, an imagery including a representation of at least a portion of a plurality of objects,

identifying, via a server in communication with the mobile communication device, at least one object of the plurality of objects as a first target object based at least in part on the imagery and a query of a database indexing data associated with a plurality of target objects including the first target object;

enabling a presentation, via an information address on a display of the mobile communication device, an indication related to the first target object based at least in part on the query of the database.

35. The method of claim 34, wherein the imagery comprises a video stream, and wherein identifying the at least one object as the first target object comprises identifying one or more images in the video stream that correspond to one or more of the plurality of target objects.

36. The method of claim 34, wherein the imagery comprises a video stream, and wherein identifying the at least one object as the first target object comprises identifying one or more objects represented in the video stream that corresponds to one or more of the plurality of target objects.

37. The method of claim 34, wherein identifying the at least one object as the first target object is further based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

38. The method of claim 34, wherein the indication is related to biometric information.

39. The method of claim 34, wherein identifying the at least one object as the first target object comprises comparing information related to the imagery to the data indexed by the database.

40. The method of claim 39, wherein the indication is associated with a retail store related to the target object.

41. The method of claim 34, wherein the indication is customized based on a user preference.

42. The method of claim 34, further comprising populating the database with information related to a user preference, and presenting a customized result set including the information address based at least in part on the user preference.

43. The method of claim 34, further comprising obtaining information related to a user preference, and populating the database with the information related to the user preference.

44. The method of claim 34, wherein enabling the presentation of the indication comprises sending an associated file to the mobile communication device.

45. An apparatus for providing target object information, the apparatus comprising:

a server having a processor, a memory in communication with the processor communication software stored in the memory, executable by the processor, and configured to obtain, via a mobile communication device, a video stream including a representation or at least a portion of a plurality of objects;

object identification software stored in the memory, executable by the processor and configured to identify at least one object of the plurality of objects as a first target object based at least in part on the video stream and a query of a database indexing data associated with a plurality of target objects including the first target object; and

target object information presentation software stored in the memory, executable by the processor, and configured to present, via an information address and on a display of the mobile communication device, an indication related to the first target object based at least in part on the query of the database.

46. The apparatus of claim 45, wherein the object identification software is further configured to identify one or more images in the video stream that corresponds to one or more of the plurality of target objects.

47. The apparatus of claim 45, wherein the object identification software is further configured to identify one or more objects represented in the video that corresponds to one or more of the plurality of target objects.

48. The apparatus of claim 45, wherein the object identification software is further configured to identify the object as the first target object based at least in part on global positioning system information associated with them mobile communication device or other information indicating a location of the mobile communication device.

49. The apparatus of claim 45, wherein the indication is related to biometric information.

50. The apparatus of claim 45, wherein the target object information software is configured to compare information related to the video stream to the data indexed by the database.

51. The apparatus of claim 50, wherein the indication is associated with a retail store related to the target object.

52. The apparatus of claim 45, wherein the indication is customized based on a user preference.

53. The apparatus of claim 45, wherein the object identification software is further configured to populate the database with information related to the user preference, and wherein the target object information presentation software is further configured to present a customized result set including the indication based at least in part on the user preference.

54. The apparatus of claim 45, wherein the object identification software is further configured to obtain information related to a user preference, and populate the database with the information related to the user preference.

55. The apparatus of claim 45, wherein the target object information presentation software is further configured to present the indication by sending an associated file to the mobile communication device.

56. A computer program product comprising a memory comprising computer-executable software for execution on a server in communication with a mobile communication device, the instructions for providing target object information, the instructions comprising:

software for identifying an object represented in an image captured on the mobile communication device as a first target object based on the image;

software for querying a database storing data associated with a plurality of target objects including the first target object; and

software for presenting, via an information address, an indication related to the first target object based on querying the database.