SYSTEM AND METHOD FOR RELIABLE CONTENT ACCESS USING A CELLULAR/WIRELESS DEVICE WITH IMAGING CAPABILITIES

• US 20070175998A1
• Filed: 08/30/2006
• Published: 08/02/2007
• Est. Priority Date: 09/01/2005
• Status: Active Grant

• Current Assignee: Zvi Lev
• Original Assignee: Zvi Lev
• Inventors: Lev, Zvi
• Granted Patent: US 7,575,171 B2
• Time in Patent Office: Days
• US Class Current: 235/454
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06K 7/10 by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00236 using an image reading or reproducing device, e.g. a facsimile reader or printer, as a local input to or local output from a computer image input to or image output from a computer via a network H04N1/00209
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00355 Mark-sheet input sensing record carriers G06K7/00; character recognition G06V30/10; document-oriented image-based pattern recognition G06V30/40
• CPC Class Codes: H04N 1/00363 Bar codes or the like
• CPC Class Codes: H04N 1/00374 on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00968 by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/00973 from a remote device, e.g. receiving via the internet instructions input to a computer terminal
• CPC Class Codes: H04N 2201/3249 data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3269 of machine readable codes or marks, e.g. bar codes or glyphs
• CPC Class Codes: H04N 2201/3273 Display
• CPC Class Codes: H04N 2201/3278 Transmission
• CPC Class Codes: H04N 2201/328 Processing of the additional information

Abstract

A system and method for reliable content access using a cellular/wireless device with imaging capabilities, to use part of a printed or displayed medium for identifying and using a reference to access information, services, or content related to the reference, including capturing an image of the reference with an imaging device, sending the image via a communications network to a processing center, pre-processing to identify relevant frames within the image and to perform general purpose enhancement operations, detecting the most relevant frame within the image, and frame properties, applying geometric, illumination, and focus correction on the relevant frame, using color, aspect ration, and frame color, to perform a coarse recognition and thereby limit the number of possible identifications of a reference within the relevant frame, and using specific techniques of resealing, histogram equalization, block labeling, edge operation, and normalized cross correlation, to identify the reference within the image.

Claims

1. A method to use part of a printed or displayed medium for identifying a reference and using the reference to access at least one of information, services, and content related to the reference, the method comprising:

capturing an image of the reference with an imaging device;

sending the image via a communications network to a processing center;

pre-processing by the processing center to identify at least one relevant frame within the image and to perform general purpose enhancement operations;

detecting most relevant frame within the image and frame properties comprising at least one of pixels, axis ratio, and illumination characteristics;

applying geometric, illumination, and focus correction on the most relevant frame;

using color, aspect ration, and frame color, to perform a coarse recognition and thereby limit a number of possible identifications of the reference within the most relevant frame; and

using resealing, histogram equalization, block labeling, edge operation, and normalized cross correlation, to identify the reference within the image.

2. The method of claim 1, wherein:

the identified reference is correlated with stored information related to the reference; and

said stored information is sent via the communication network to a user of the imaging device.

3. The method of claim 1, wherein:

the identified reference is a banner; and

the stored information is sent to the user via the communication network.

4. The method of claim 3, wherein:

the identified reference is an electronic banner.

5. The method of claim 3, wherein:

the identified reference is a printed banner.

6. The method of claim 1, wherein:

the reference within the image is at least one of;

a logo, a portrait or drawing of a person, a portrait or drawing of scenery, an advertising slogan, a graphical symbol, alphanumeric code, a customer designed frame, a representation of at least one texture.

7. The method of claim 6, wherein:

said at least one template is used to identify the reference within the image.

8. The method of claim 2, wherein:

the reference is one of a plurality of banners and logos associated with the stored information;

the stored information is accessed via any of the plurality of banners and logos; and

the stored information is sent to the user via the communication network.

9. The method of claim 2, wherein:

relative sizes of various data within the image are used to identify the reference.

10. The method of claim 1 wherein:

if the at least one relevant frame containing the reference is not identified using automated techniques, then a message is sent to a user of the imaging device, asking the user to identify the at least one relevant frame within the image; and

the user selects the at least one relevant frame within the image, and communicates the selection, using the communication network, to the processing center.

11. The method of claim 1 wherein:

if the reference within the most relevant frame is not identified using automated techniques, then a message is sent to a user of the imaging device, asking the user to identify the reference within the most relevant frame within the image; and

the user selects the reference within the most relevant frame of the image, and communicates the selection, using the communication network, to the processing center.

12. The method of claim 1, wherein:

knowledge of a type of specific imaging device is used to detect the most relevant frame.

13. The method of claim 12, wherein:

the type of specific imaging device is identified by type of communication protocol used by the imaging device.

14. The method of claim 12, wherein:

the type of specific imaging device is identified by a specific identification number associated with the imaging device that is communicated to the processing center.

15. The method of claim 1, wherein:

knowledge of a type of specific imaging device is used to detect the reference.

16. The method of claim 15, wherein:

the type of specific imaging device is identified by type of communication protocol used by the imaging device.

17. The method of claim 15, wherein:

the type of specific imaging device is identified by a specific identification number associated with the imaging device that is communicated to the processing center.

18. The method of claim 1, wherein:

the reference is identified with an aid of at least one of normalized cross-correlation, multiple normalized cross-correlation, multiple color band analysis, aspect ration analysis, image histograms, analysis of text density distribution, and analysis of control points.

19. The method of claim 1, which is further used to determine effectiveness of an advertising medium, the method further comprising:

a service provider captures and records information about images taken by a user of the imaging device;

the service provider analyzes said information to determine whether and when the user has reacted to images in at least one of billboards, newspapers, magazines, Web sites, television ads, and other communication media; and

the service provider sends said information to at least one provider of the images.

20. A computer program product, comprising instructions for causing a computer to perform a method comprising:

capturing an image of a reference with an imaging device;

sending the image via a communications network to a processing center;

pre-processing by the processing center to identify at least one relevant frame within the image and to perform general purpose enhancement operations;

detecting most relevant frame within the image, and frame properties comprising at least one of pixels, axis ratio, and illumination characteristics;

applying geometric, illumination, and focus correction on the most relevant frame;

using color, aspect ration, and frame color, to perform a coarse recognition and thereby limit a number of possible identifications of the reference within the most relevant frame; and

using specific techniques of resealing, histogram equalization, block labeling, edge operation, and normalized cross correlation, to identify the reference within the image.

21. A system for using part of a printed or displayed medium to identify a reference and use the reference to access at least one of information, services, and content related to the reference, the system comprising:

a publisher that provides a page design, allocates space for code links, facilitates the activation of the code links to a user, provides content for the user from each activation code, verifies that experience of the user is adequate;

a page with natural links to sources of information;

wherein the user detects the natural links of interest and scans said natural links to receive information;

a user interface unit used by the user to scan the natural links;

a communication network that provides a communication link between the user interface unit and computational facilities, wherein the computational facilities provide code detection, code recognition, content routing, and interaction with the user via the user interface unit; and

content related to the natural link.

22. The system of claim 21, wherein said user interface unit comprises:

an image and video input channel which allows imaging of the natural link for further link recognition;

an image and video display channel which allows feedback from the input channel and the computational facilities;

a keypad input channel that allows the user to communicate with remote computational facilities when activating said code detection and said code recognition; and

an audio communication channel which allows the user to communicate with said remote computational facilities via integrated voice response, and to receive instruction in an on-line mode.

23. The system of claim 21, wherein said computational facilities comprise:

a data storage server which stores user'"'"'s requests, logs request information, stores content later presented to the user, and stores all relevant links in a database for recognition purposes;

an application server which allows access to content based on the recognized link, provides billing services, and manages communication with the user;

an image detection and recognition server which detects the frame of the natural link performs recognition on a symbol of the natural link;

a video and image preprocessing server that combines video frames and frames from various requests for the best resulting image, supports zoom-on activation mechanisms, and provides basic image improvement services prior to additional dedicated processing;

and a data routing server which allows interface with various communication protocols supported by the communication network, user feedback, and data distribution to various content providers.

24. The system of claim 23, wherein the computational facilities further comprise a speech recognition server which enables the application server to perform audio interface with the user.

25. The system of claim 21 in which the system measures advertising effectiveness, computes bills, and sends said bills to the appropriate parties, the system further comprising:

an end user who performs imaging and receives the content;

a network provider that provides content to the end user, generates a first bill to the end user and collects money from the end user, and generates a second bill to the service provider and collects money from the service provider;

a content and service provider that provides content to the user via the network provider, generates a third bill to the end user and collects money from the end user, measures advertising effectiveness, generates a fourth bill to advertisers and collects money from the advertisers, generates a fifth bill to content owners and collects money from the content owners; and

a plurality of suppliers of media to the user.

26. A device for using part of a printed or displayed medium to access at least one of information, services, and content related to a natural link, the device comprising:

an image and video input channel which allows imaging of a natural link used to access said at least one of information, services, and content related to the natural link;

an image and video display channel which allows feedback from a keypad input channel and remote computational facilities, wherein the computational facilities detect and recognize the natural link, obtain and route said at least one of information, services, and content related to the natural link; and

a keypad input channel that allows the user to communicate with remote computational facilities when activating said code detection and said code recognition.

27. A device for identifying a natural link in part of a printed or displayed medium and using the natural link to access at least one of information, services, and content related to a natural link, the device comprising:

a data storage server which stores user'"'"'s requests, logs request information, stores content later presented to the user, and stores natural links corresponding to the content in a database;

an application server which allows access to content based on a recognized natural link, provides billing services, and manages communication with a user;

an image detection and recognition server which detects frame of the natural link and performs recognition on a symbol of the natural link;

a video and image preprocessing server that combines video frames and frames from various requests for best resulting image, supports zoom-on activation mechanisms, and provides basic image improvement services prior to additional dedicated processing; and

a data routing server which allows interface with various communication protocols supported by a communication network, user feedback, and data distribution to various content providers,

wherein the natural link is on a printed or displayed medium and is a link to corresponding content.