System and method for improved searching on the internet or similar networks and especially improved MetaNews and/or improved automatically generated newspapers

• US 8,589,373 B2
• Filed: 09/14/2004
• Issued: 11/19/2013
• Est. Priority Date: 09/14/2003
• Status: Active Grant

• Current Assignee: Batya Barhon Mayer
• Original Assignee: Batya Barhon Mayer
• Inventors: Mayer, Yaron
• Primary Examiner(s): Truong, Thanhnga B
• Application Number:
  US10/939,454
• Publication Number:
  US 20050114324A1
• Time in Patent Office: 3,353 Days
• Field of Search: 707 3- 5, 707/737, 707/749, 707/704, 707/708, 707/999.01, 707/999.003, 707/201, 707705-707, 709/217, 709/238, 709/224, 715/243, 715/251, 713/2, 439/152, 726/1
• US Class Current: 707/706
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

System and method is provided to improve searching on the Internet or similar networks and especially improved MetaNews and/or improved automatically generated newspaper. The present invention solves the problem by creating recursive clustering, so that preferably at any level in the tree the user can preferably either choose a specific news item from the cluster or from the shown sub-clusters or continue in the tree. Another improvement is that searching the Meta News by keywords can generate an automatic newspaper in a way similar to the original automatically generated newspaper. Many additional improvements to the concept of automated newspapers and/or news MetaSearch are also shown. Other improvements are suggested for improved shareware MetaSearch, improved Web pages search, and other types of searches.

Claims

1. A method for an improved News Meta-Search over a large number of Online news sources on the Internet or similar networks, comprising providing a meta-search system which includes at least one server, and displaying news items to a user through a browser on a computer,

wherein the server performs, under software instruction from the meta-search system, at least one of the steps of:

i. Switching between news items from the same cluster or sub-cluster which are displayed in a given position in an automatically generated newspaper page, wherein said switching is done automatically or with user intervention; and

ii. Switching between news images from the same cluster or sub-cluster which are displayed in a given position in an automatically generated newspaper page, wherein said switching is done automatically or with user intervention, and wherein said images are at least one of still images and streaming data;

wherein at least one of the following features exists;

a. Recursive sub-clustering is performed and the recursive sub-clustering continues until there are sufficiently few items in the final sub-category or until the items are too different to group further;

b. If the user searches for keywords in the News Meta Search, the results are displayed recursively in clusters and sub-cluster in a way similar to the automatically generated newspaper page;

c. If the user searches for keywords in the News Meta Search, the results can have all the features that exist in the automatically generated newspaper page;

d. The system enables the user to switch between a mode that displays also images and a mode without images;

e. The same news item or same sub-cluster can belong to more than one cluster or sub-cluster, and thus it is shown and/or can be reached from all the sufficiently relevant clusters or sub-clusters to which it is related;

f. The system enables the user to request to sort a list of related items by relevance and/or by time and date to create order between and/or within the sub-clusters, so that the system performs the sorting without interfering with the cluster structure itself;

g. The system enables the user to request to sort the items by at least one of;

1. The country of the source, so that the system orders or clusters the news items in addition or instead also according to the country of the news source, 2. The level of reliability of the source, so that the system orders or clusters the news items in addition or instead also according to the reliability of the news source;

h. The system enables the user to view a graphical or textual hierarchical representation which shows simultaneously the multi-level structure of clusters and sub-clusters, showing more than two levels of the hierarchy at the same time, or showing the structure down to the end-nodes;

i. The Meta News system automatically chooses only images that are within a certain reasonable range of sizes;

j. As additional new related news items come in, the headlines and/or the images can be automatically updated even if the user does not click on any refresh button;

k. The user gets a different indication when the items or images themselves have changed or new items or images are brought in (compared to the normal swapping between items), and said indication is at least one of sound indication and visual indication of the item that has changed or the new item that has been inserted;

l. The html protocol and/or the html command set is expanded to allow an image to be requested with a given size limit, so that if the original image is bigger it is either truncated automatically to fit in the allowed window, or is automatically downscaled in order to fit completely into the allowed space;

m. The html protocol and/or the html command set is expanded to allow an image to be requested with a given size limit, so that if the original image is bigger it is truncated automatically to fit in the allowed window and for said truncation the improved html protocol allows the web programmer to specify for each image the x-y coordinates of its central point of interest, and/or various heuristics are used by the browser or by the server in order to find the central point of interest automatically;

n. When switching images contain also streaming data, at least one of the following is done;

1. Automatic switching of images is disabled so that the user has to click on something in order to view related streaming data from a different source or other still images, and 2. Each streaming source remains in the position for a longer time than still images until switching to the next streaming source or to the next still image;

o. The system determines which item to use as the main item of the general cluster by at least one of;

1. First picking the sub-cluster that has the largest number of items and/or the most recent cluster that is big enough relative to other sub-clusters, 2. Picking the item within the chosen first sub-cluster which has the highest average similarity to other items in that sub-cluster and/or belongs to the largest sub-cluster of that sub-cluster and/or is most relevant within the cluster or within the sub-cluster and/or is most recent within the cluster or within the sub-cluster;

p. When requesting News alerts, instead of being able to request only by specific keywords, the system enables the user to also at least one of;

1. Mark a cluster or a specific sub-cluster, so that he/she is notified automatically on any new items that belong to that cluster or after sufficient changes have accumulated in the cluster, 2. Use semantic qualifiers, 3. Mark words in a way that indicates that synonyms should also be checked for these words, so that he/she will be notified also about items that contain synonyms of these marked words;

and wherein at least one of the following features exists;

q. In order to improve the clustering ability, the time the items were published is taken into account, with the assumption that the closer the time of publication between them, the higher the chance that two items are dealing with the same event;

r. Temporal words or phrases used in the news item are used to decide when the event occurred, and this time is used to separate between news items that occurred before this time and items that occurred after this time and/or to help decide the similarity between items that might be referring to the same event;

s. Temporal words or phrases used in the news item are used to decide when the event occurred, and in order to analyze the temporal phrases used in the item, the system is able to perform also at least some minimal type of semantic analysis and/or has at least knowledge of the relevant temporal nouns and relevant verbs; and

t. When sorting automatically generated news clusters the number of items in each cluster is normalized by the time factor, since clusters that have exited for a longer time would normally have more items than a newer cluster even if the new cluster is more important.

2. The method of claim 1 wherein the system enables the user to request to automatically spread still images and/or streaming images of the same cluster or sub-cluster together next to each other so that they can be viewed simultaneously, and at least one of the following features exists:

a. By clicking on or near one of the simultaneous streaming data images the user is transferred to that source to view it normally there;

b. The system enables the user to switch the sound between simultaneous streaming data sources;

c. The group of images is automatically and dynamically generated according to the item of interest and according to availability in the various sources, so that images or streaming date can be automatically added or removed accordingly.

3. The method of claim 1 wherein as additional new related news items come in, the headlines and/or the images can be automatically updated even if the user does not click on any refresh button and wherein said automatic updating is done by partial refresh on a need basis by at least one of the following ways:

a. The refresh command is initiated automatically by the site when there is any change in the page, so that the browser can get a refresh even if it didn'"'"'t ask for it;

b. The browser can ask for refresh, but if nothing has changed then the browser gets just a code that tells it to keep the current page or window as is;

c. When the refresh is sent, it is a smart refresh, which tells the browser only what to change on the page instead of having to send the entire page again.

4. The method of claim 1 wherein at least one of the following features exists:

a. In order to enable the multi-level sub-clustering the same or similar principles are applied similarly at all levels, except that in each step they are applied now to the items of the previous cluster or sub-cluster in order to further divide them into additional sub-clusters;

b. For clustering the system analyses the similarity in the occurrence of combinations of two or more words in the headline and/or in the first 1 or 2 sentences and/or in the entire item.

5. The method of claim 1 wherein the system has at least one of:

a. A knowledge base of at least one of;

country names, city names, and other geographical areas;

b. A knowledge base of at least the most common or most important verbs that typically appear in headlines and/or in the first one or two sentences of news items and/or in entire news items;

c. A knowledge base of verbs that uses semantic trees and/or semantic graphs and/or various rules, so that each verb can be characterized by scores on a number of relevant variables or dimensions;

d. A database of synonyms for the comparisons of nouns and/or of verbs, so that the system can know if two words are different or similar even without “

understanding”

their meaning;

e. A knowledge base of major known political names and organizations;

f. The ability to take into account also similarity in words at least in the headlines, even if they are not exactly identical.

6. The method of claim 1 wherein at least one of the following features exists:

a. If the user chooses one of the top level subject categories, the number of clusters is not limited to a specific number, so the system enables the user to continue further to the next issues as long as the next cluster or clusters are sufficiently important and/or as long as there are sufficient items and/or sufficient sub-clusters in the next cluster or clusters;

b. Instead of just a few top categories in the automatically generated newspaper, there are also subcategories, so that choosing a top category can open a submenu that lets the user choose also a sub-category in one or more additional sub-levels;

c. When the user chooses a certain country in the general choices available for the automatically generated newspaper, the system allows the user to choose if he wants more emphasis on news related to this country, more emphasis on sources related to this country, or both;

d. Automatic recursive clustering and sub-clustering is used also in normal web searches and/or in newsgroup searches;

e. The switching between items and/or between images of the same cluster and/or sub-cluster is done locally on the user'"'"'s browser, so that after the images and the headlines are loaded for the first time, the browser can easily switch between them from the local cache and/or from internal memory, until real new data is sent again;

f. Sending the same data to many users or to many servers or minor sites at the same time so that identical data packets are sent only once in each condensed packet by creating automatically multicast groups and/or sub groups and assigning automatically users to them;

g. Instead of keeping the list of target addresses in the condensed packet, the list of targets is sent first to a server or router in the target area, and the following condensed packets for the same group can be sent to that server or router without the list of targets and instead the condensed packets include a code that identifies the multicast group and/or the desired list of targets that the server or router in that area already has.

7. The method of claim 1 wherein when displaying automatically generated news at least one of the following features exists:

a. The system enables the user to define the desired time span to cover;

b. The system enables the user to mark certain news items or clusters or sub-clusters as no longer interesting;

c. The user is notified automatically when a sufficient number of new clusters has become available and/or when a sufficient number of new items has become available in existing clusters which the user has not requested to remove, and/or in clusters which the user has marked as especially interesting for him/her;

d. The system enables the user to review historically the automatically generated items that were on the automatic newspaper or on a specific section of the automatic newspaper, by at least one of;

1. Specifying the time in hours and/or days to jump back, 2. Specifying the time in hours and/or days to jump back, and then jumping automatically the same interval in the next jumps, 3. Specifying a specific date and/or time, 4. The system enables the user to press some link which automatically takes him/her back at each step automatically according the amount of change.