Use of image-derived information as search criteria for internet and other search engines

• US 20060002607A1
• Filed: 08/15/2005
• Published: 01/05/2006
• Est. Priority Date: 11/06/2000
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Evryx Technologies Inc (Ipplex Holdings Corporation)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Granted Patent: US 7,680,324 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/768 : using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/80 : Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 2201/07 : Target detection
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Claims

1. A system comprising:

a portable device that optically captures an image; and

transmits information regarding the image to a distal server;

at least one of the device and the server deriving a search term from at least one of the image and the transmitted information;

at least one of the device and the server causing the search term to be submitted to a search engine that indexes publicly accessible information; and

the search engine using the search term to produce a results set, and causing at least a portion of the results set to be transmitted back to the portable device.

2. The system of claim 1, wherein the portable device includes cell phone telephony capability.

3. The system of claim 1, wherein the search term is contained as a literal within the image.

4. The system of claim 1, wherein the search term is not contained as a literal within the image.

5. The system of claim 1, wherein the device derives the search term.

6. The system of claim 1, wherein the device submits the search term to the search engine through the server.

7. The system of claim 1, wherein the device submits the search term directly to the search engine.

8. The system of claim 1, wherein the server derives the search term.

9. The system of claim 1, wherein the server submits the search term to the search engine.

10. The system of claim 1, wherein the search term comprises a key word.

11. The system of claim 1, wherein the search term comprises a portion of the image.

12. The system of claim 1, wherein the search term comprises a symbol.

13. The system of claim 11, wherein the symbol comprises a logo.

14. The system of claim 11, wherein the symbol comprises a bar code.

15. The system of claim 11, wherein the results set includes at least one hyperlinked address.

16. The system of claim 11, wherein the search engine is selected from the list consisting of Google™

, Yahoo!™

, MSN™

, and Alta Vista™

.

17. A cell phone that runs software that allows it to operate as the device in a system according to claim 1.

19. The method of claim 16, further comprising:

using the device to take a second image of the object;

running the computer software to derive a second search term from at least a portion of the second object; and

submitting the second search term along with the first search term to the search engine.

20. The method of claim 16 wherein the step of submitting the first search term comprises:

sending at least the portion of the image to a distal server;

running the software on the server; and

the server sending the search term to the search engine.

18. A method of using a search engine to obtain information, comprising:

using a cell phone enabled portable device to take an image of an object;

running computer software that automatically derives a first search term from at least a portion the image;

submitting the first search term to the search engine; and

transmitting the information to the device.

21. A portable device, comprising:

a portable power source;

a camera portion that optically captures an image; and

a processor that processes instructions to derive a search term from the image and transmit the term to a search engine.

22. A results set derived using the following steps:

capturing an image using a portable device;

the portable device transmitting information regarding the image to a distal server;

at least one of the device and the server automatically deriving a search term from at least one of the image and the transmitted information;

at least one of the device and the server causing the search term to be submitted to a search engine that indexes publicly accessible information; and

the search engine using the search term to produce the results set.