Object information derived from object images

• US 9,046,930 B2
• Filed: 07/15/2014
• Issued: 06/02/2015
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/332,216
• Publication Number:
  US 20140328513A1
• Time in Patent Office: 322 Days
• Field of Search: 382/103, 382/181, 382/305, 382/100
• US Class Current: 1/1
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A method comprising:

obtaining image information associated with an image, wherein the image was captured by a mobile device and includes a representation of at least one object;

determining, based at least partially on the image information, that the image depicts a motion of the at least one object;

recognizing, based at least partially on the image information, that the motion is associated with a sport and a target object;

obtaining an address related to the target object; and

presenting, via the mobile device, content information associated with the motion based on a query submitted via the address to a database storing data corresponding to a plurality of known target objects including the target object.

2. The method of claim 1, wherein the motion is related to a change in a speed over time.

3. The method of claim 1, wherein the motion is related to a flight.

4. The method of claim 1, wherein the motion comprises a gesture.

5. The method of claim 1, wherein the motion comprises a change in a shape of the at least one object over time.

6. The method of claim 1, wherein the at least one object comprises a person.

7. The method of claim 1, wherein the image includes a plurality of partial views of the at least one object based on the movement.

8. The method of claim 1, wherein the image comprises at least one of a video and motion imagery.

9. The method of claim 1, wherein the image comprises a real time video.

10. The method of claim 1, wherein the image information includes at least one of a location, a position, and a time.

11. The method of claim 10, wherein presenting the content information is based at least in part on the at least one of the location, the position, and the time.

12. The method of claim 1, wherein the image includes a representation of an object comprising a tag.

13. The method of claim 1, wherein the image lacks a representation of a tag.

14. The method of claim 1, further comprising enabling, via the mobile device, an interaction associated with the content information based on the address.

15. The method of claim 14, wherein the interaction comprises at least one of a transaction, a purchase, accessing a web site, and obtaining a location of a store based at least in part on a location of the mobile device.

16. The method of claim 1, wherein the content information associated with the motion comprises guidance related to a maneuver.

17. The method of claim 1, wherein the content information associated with the motion comprises an identity of a machine.

18. The method of claim 1, further comprising:

receiving, via the mobile device, user information associated with at least some of the plurality of known target objects and a user of the mobile device;

comparing the image information and the user information associated with the at least some of the plurality of known target objects with the data corresponding to the plurality of known target objects; and

presenting, via the mobile device, information associated with at least one object of the plurality of known target objects.

19. The method of claim 18, wherein the information associated with at least one object of the plurality of known target objects is customized based at least in part on the user information.

20. The method of claim 19, further comprising:

receiving, via the mobile device, location information associated with the mobile device; and

presenting, via the mobile device, the content information associated with the motion based at least in part on the location information.

21. An apparatus for providing target object information, the apparatus comprising:

a computing platform comprising a memory and at least one processor; and

an object identification application stored in the memory, executable by the processor, and configured to;

obtain image information associated with an image, wherein the image was captured by a mobile device and includes a representation of at least one object;

determine, based at least partially on the image information, that the image depicts a motion of the at least one object;

recognize, based at least partially on the image information, that the motion is associated with a sport and a target object;

obtain an address related to the target object; and

present, via the mobile device, content information associated with the motion based on a query submitted via the address to a database storing data corresponding to a plurality of known target objects including the target object.

22. The apparatus of claim 21, wherein the motion is related to a change in a speed over time.

23. The apparatus of claim 21, wherein the motion is related to a flight.

24. The apparatus of claim 21, wherein the motion comprises a gesture.

25. The apparatus of claim 21, wherein the motion comprises a change in a shape of the at least one object over time.

26. The apparatus of claim 21, wherein the at least one object comprises a person.

27. The apparatus of claim 21, wherein the image includes a plurality of partial views of the at least one object based on the movement.

28. The apparatus of claim 21, wherein the image comprises at least one of a video and motion imagery.

29. The apparatus of claim 21, wherein the image comprises a real time video.

30. The apparatus of claim 21, wherein the image information includes at least one of a location, a position, and a time.

31. The apparatus of claim 30, wherein the object identification application is further configured to present the content information based at least in part on the at least one of the location, the position, and the time.

32. The apparatus of claim 21, wherein the image includes a representation of an object comprising a tag.

33. The apparatus of claim 21, wherein the image lacks a representation of a tag.

34. The apparatus of claim 21, wherein the object identification application is further configured to enable, via the mobile device, an interaction associated with the content information based on the address.

35. The apparatus of claim 34, wherein the interaction comprises at least one of a transaction, a purchase, accessing a web site, and obtaining a location of a store based at least in part on a location of the mobile device.

36. The apparatus of claim 21, wherein the content information associated with the motion comprises guidance related to a maneuver.

37. The apparatus of claim 21, wherein the content information associated with the motion comprises an identity of a machine.

38. The apparatus of claim 21, wherein the object identification application is further configured to:

receive, via the mobile device, user information associated with at least some of the plurality of known target objects and a user of the mobile device;

compare the image information and the user information associated with the at least some of the plurality of known target objects with the data corresponding to the plurality of known target objects; and

present, via the mobile device information associated with at least one object of the plurality of known target objects.

39. The apparatus of claim 38, wherein the information associated with at least one object of the plurality of known target objects is customized based at least in part on the user information.

40. The apparatus of claim 39, wherein the object identification application is further configured to:

receive, via the mobile device, location information associated with the mobile device; and

present, via the mobile device, the content information associated with the motion based at least in part on the location information.

41. A computer program product comprising a non-transitory computer-readable medium having computer executable instructions to cause a computer to implement the steps of:

obtaining image information associated with an image, wherein the image was captured by a mobile device and includes a representation of at least one object;

determining, based at least partially on the image information, that the image depicts a motion of the at least one object;

recognizing, based at least partially on the image information, that the motion is associated with a sport and a target object;

obtaining an address related to the target object; and

presenting, via the mobile device, content information associated with the motion based on a query submitted via the address to a database storing data corresponding to a plurality of known target objects including the target object.

42. The computer program product of claim 41, wherein the motion is related to a change in a speed over time.

43. The computer program product of claim 41, wherein the motion is related to a flight.

44. The computer program product of claim 41, wherein the motion comprises a gesture.

45. The computer program product of claim 41, wherein the motion comprises a change in a shape of the at least one object over time.

46. The computer program product of claim 41, wherein the at least one object comprises a person.

47. The computer program product of claim 41, wherein the image includes a plurality of partial views of the at least one object based on the movement.

48. The computer program product of claim 41, wherein the image comprises at least one of a video and motion imagery.

49. The computer program product of claim 41, wherein the image comprises a real time video.

50. The computer program product of claim 41, wherein the image information includes at least one of a location, a position, and a time.

51. The computer program product of claim 50, wherein presenting the content information is based at least in part on the at least one of the location, the position, and the time.

52. The computer program product of claim 41, wherein the image includes a representation of an object comprising a tag.

53. The computer program product of claim 41, wherein the image lacks a representation of a tag.

54. The computer program product of claim 41, wherein the instructions further cause the computer to implement the step of enabling, via the mobile device, an interaction associated with the content information based on the address.

55. The computer program product of claim 54, wherein the interaction comprises at least one of a transaction, a purchase, accessing a web site, and obtaining a location of a store based at least in part on a location of the mobile device.

56. The computer program product of claim 41, wherein the content information associated with the motion comprises guidance related to a maneuver.

57. The computer program product of claim 41, wherein the content information associated with the motion comprises an identity of a machine.

58. The computer program product of claim 41, wherein the instructions further cause the computer to implement the steps of:

receiving, via the mobile device, user information associated with at least some of the plurality of known target objects and a user of the mobile device;

comparing the image information and the user information associated with the at least some of the plurality of known target objects with the data corresponding to the plurality of known target objects; and

presenting, via the mobile device, information associated with at least one object of the plurality of known target objects.

59. The computer program product of claim 58, wherein the information associated with at least one object of the plurality of known target objects is customized based at least in part on the user information.

60. The computer program product of claim 59, wherein the instructions further cause the computer to implement the steps of:

receiving, via the mobile device, location information associated with the mobile device; and

presenting, via the mobile device, the content information associated with the motion based at least in part on the location information.