Wake Up Web 2.0

Why Web 2.0 Spells a Reckoning for Google and Mass Media

2007-04-22T10:26:00.001-07:00

Semantic Web 2.0 Mashup

2006-11-16T20:23:00.000-08:00

The online marketing world is finally realizing that social media is here to stay.

The first step was being amazed by how quickly leaders such as Myspace could tap into the market created by sites like friendster, and promoted through existing legacy networks such as the AOL instant messenger and the AOL messenger users.

The attention that big media has paid to these large networks was another big part of all the hype surrounding social networking.

Marketers and Publishers are still trying to figure out what social networking is, and what makes it so popular. Why now?

Other trends include Web 2.0 , application / data mashups, and the ‘semantic web’ envisioned by www creator Tim Berners-Lee.

The truth is that social networking is as old as the web. Newsgroups, USENET, forums, bulletin board systems were all the traditional ways of online social networking.

Most of the components that comprise a social network, such as user profiles, picture galleries, forums, groups, rate-me, etc., had already been around.

Three novelties that allowed the acceleration of the new generation of social networking sites were:

One stop shop for all “my” things. Music, Video, Friends, Work, Info
Public information of who your friends are.
Viral marketing techniques to promote social networks on away messages.

Never before had all of these features been available in one place that could be at the same time available to everyone else.

The public buddy list allowed you to delve into social groups that you might not have been able to locate just on keyword searches alone. By being able to see who is friends with who, the user has a vastly greater perspective as to the interrelationships among the friend users.

Forums in the past focused more on the topics, and the postings of the users, not on the friends and the detailed life of the user.

The focus away from the intent of the forum, and put the person and their social connections at the center of the service.

Social networks have also flourished by viral marketing campaigns on media such as away messages. These have offered their participants to sign up for their free page. Also, networks have strategically forced users to register for a profile if they want to see the photos or extended information of other users.

So, the social networking sites are here to stay. Now what?

What impact do these have on the entertainment / publishing universe?

What impact do they have on the current Web Search, Web advertising marketplace?

Social networks are all about decentralization. They enable groups to form and disseminate ideas and media without needing specialized web knowledge.

Social media has been around forever, but now it has crossed over to the mainstream.

It is now part of the young internet user’s everyday internet experience.

Myspace comments and mail messages are just another form of email. Myspace photo albums have replaced the other free digital photo gallery services.

The idea is all of a person’s digital files in one place, and every person in one place.

What are the possibilities with social networks?

What are the limitations of social networks as they currently are structured?

One thing that is clear is that social networks like open-ness.

AOL’s member profiles were 100% hidden from non-AOL members.

The result: AOL’s member profiles became irrelevant.

Large social networks may have much more openness in terms of free se, open information, but their owners will still prefer to restrict what kind of content they accept.

Originally there was a large controversy when Myspace users were linking to videos on revver.com, a video sharing site. Myspace had a policy of deleting such links, as they were apparently a competitive service.

These knee-jerk reactions show the weaknesses and vulnerabilities of a business model such as Myspace which on one hand relies on open standards and open access, and free flow of information, and on the other hand wants to maintain a degree of control over the services its users adopt.

No publishing format in web history has had so much success.

It is very reasonable to believe that the structures, relationships, features and tagging ability that is either supported or supportable by these social networks will impel developers on the leading edge to develop new standards that expand the legacy WWW, and HTML formats.

Social Media and Search:

Social media is great first because it presents an alternative to search.

Social media really offers another option between a long term, challenging, unpredictable natural SEO campaign and an expensive, take-it-or-leave-it PPC pricing environment on the major engines.

Marketers can engage their pre-formed, pre-screened audience in a direct and non-intrusive way with their message, product or service.

Marketers can also obtain market research, demographic, psychographic, and behavioral information with the greatest of ease.

Social networks have become one of the largest and most diverse collections of focus groups.

How will social search effect standard search engine results?

Large scale search engines such as Google were designed to index and retrieve documents from HETEROGENEOUS document collections. This means that the documents have no common thread regarding topic, length, format, or components.

Some documents may contain general knowledge, others contain technical information. Some documents are lengthy, others are terse. Some documents are on one topic, others on another. There is absolutely no similarity among the documents other than that they use standard HTML markup strategies and their hardware conforms to the WWW standard.

Searching heterogeneous data collections is a unique problem, and one that large companies such as Google have made a great deal of progress on.

One of the most interesting aspects of social media, is how it has created on of the largest datawarehouses of HOMOGENEOUS documents.

Profiles and other sections on Myspace are Homogeneous because each page contains certain structured information such as Name, Location, Favorite Music, List of Friends, Blog Article, etc.

The fact that the data collection is fully structured means that many of the generalities that drove the engines such as Google do not apply, and that more specific and powerful, and relevant techniques for information retrieval can be applied to said datasets.

Currently, it is believed that engines such as Google, Yahoo and MSN treat social networking pages, (as well as social bookmarking pages) just as any other page.

Note: Myspace profile pages are indexable, but the other information is not.

Social networks also have the power to make a lot of the questions in search no longer relevant.

Who cares how to construct an algorithm to predict hard to find spam, when user behavior pins the tail right on the spam for you?

If distributed human computing and behavior is enough to predict what is relevant, spammy, and authoritative, then, who needs AI?

The vast availability of user data, on top of that user data being correlated with search behavior, advertising behavior, shopping behavior, even television viewing behavior, this data now becomes a rich source of knowledge.

New search engine AI will not be totally separated from the real human world, it will become inter-twined with it.

The most challenging aspects of knowledge retrieval and discovery will take place with a combination of the best computational processes along with the most elegant introductions of the social behavior and human interaction with data sets.

What are the new strategies for search marketers? What are things to watch for?

Social networking allows the quick and easy dissemination of information. Any negative sentiment can be spread lightning fast to almost all the important venues online.

Instead of managing “branding”, managing the REAL brand experience is now the prime importance.

Companies will find that there is no shortcut, or way around the reality they create with their users and customers.

In the new world of social media it’s publish, or die.

Engage a community, or be forgotten.

Social Media and Video content.

What social media has done for online chatting and some of its benefits of information retrieval through social tagging are really only the beginning of its transformation of the media landscape.

Large companies get this trend. They see these as the new media networks.

Myspace is the modern day MTV, which dominates the youth audience with a new and improved format.

Social networks will continue to play a part as the network that offers original and syndicated and linked content to it’s audience.

The mass entertainment, as well as the niche information and entertainment areas are currently being developed by major media companies.

The other networks had been delaying their push into broadband video entertainment until being beaten over the head by Myspace and its success.

Myspace has begun to vertically integrate itself by diving head first into the media universe. They have a joint venture with Interscope records, and have already released compilation and standard albums of their signed talent. They are also in development of a movie production studio, a television studio, and various Satellite Radio stations.

Myspace: A place for friends.

The question every Web 2.0 idealist is asking is, where is the place for friend feeds?

Forms of Thought

2006-10-29T10:51:00.000-08:00

The Topology of the Forms of Thought

Topology of Thought

Every system with a top-down component of structure needs some rules by which it is governed.

The simple coherence, and integrity of the system is based on it's adoption of universal rules and structures.

Hence, the model incorporates the of logic of both descriptive and prescriptive methods.

Both inductive creation as well as deductive determination are supported.

The bridge between the form of thought, and the content of language is crossed by symbolic representation.

This is a model of symoblic representation. Or, rather, a meta-model.

This model includes all systems of deductive and inductive logic, as well as conceptual spaces, interpreted by symbolic logical systems.

The classes of compontents to the system:

1. Logic - Grammars - This is the rules governing the logical systems. It defines the process by which sentences, questions and proofs are interpreted.

Grammars represent well-formedness of conceptualization within the model. They also contain the logical process interpretations to evaluate truth statements or procedures.

The logic or grammer class has falls into the Interpretation super-class of core classes. This is because is plays a clarifying or elucidating role.

Other synonyms that relate to the logical layer are : system, network, constraints, meta-model, graph, constraints, rules, game.

As a whole class Logical Rule systems can describe collections including itself. As well, sub-classes of rule systems include Processes and Procedures, that are NOT self-referrential.

The core essence of a logic is that it is desriptive. The core property is well-formedness.

- The next two core classes fall into the type of "Identifiers", since they are involved in the process of naming, and identifying objects as they pertain to the platonic world of forms, or the material.

2. Concept

It represents some "space" projected from some model.

Concepts have internal topologies as well as external topologies and properties within their space.

Other notions of concepts are abstractions, simplifications, generalizations, . Understandings and implications are a sub-class of concepts.

Concepts are used in a bottom-up fashion. They are flexible, unrestricted, ad-hoc, and interpretive.

Their creation process is one of identification. Their nature is determined by their properties.

Concepts refer and may contain objects.
Concept creation is a process, and processes are concepts.
Symbols represent a bridge among concepts, to map and convey higher-order concepts.
Symbols as a technique prepresent the concept of conceptualization or abstraction.
Symbols as a useage however represent the sub-class concept of a grammar.

The concept is interpreted using a Model, a concept can or can not be a Model (Grammar). Some concepts are grammars.

Concepts have properties. Properties are founded on concepts.

The core process of a concept is formulation. Creation. Thought.

The core property is Intangibility. Abstraction. Idea.

3. Object

An Object has the core properties is their name or designation.

Objects serve the process of Identification, Encapsulation, Forming boundaries, Defining, Discriminating, Differentiating.

Objects are Concepts. Objects can be colletions, or atomic. Collections are objects.

A concept is one kind of object.

An object relies on a concept for it's meaning, but also refers to a specific name or designation to connect that meaning to a subspace of a conceptual space.

The object is a label on the map of meaning.

Words are objects. Objects use words as components to identify their scope.

Node, Edge.

In our model, a concept is an object, and an object is a concept.

4. Process - Procedures, that are NOT self-referrential. Role as Verb. Sub-class of Logic.

Procedures are applications of rules.

Proof is a Process. (The proof symbol is a separate object)

Establishment of truth or a result. Returning an answer.

Answer.

Processes enable consistency, compliance,.

Evaluation. Applicaitons.

Processes fall into the super-class of "Interprative Classes".

This is because their purpose is clarification, or elucidation.

Procedures are determinative.

They represent an applications of rules, to symbols that may or may not comply with those rules.

Processes take an abstract rule-bases system, and apply it to a symbol, which is presumably mapped to concepts or objects.

Process is the Action that enables language.

Process Verb. IN grammer a process applies to a subject. The subject of a process of typically a symbol, or space.

Depending on if it is an logical, analytic, algebraic, or graphical process.

In design, Process belongs to the "Structural" super-class.

In usage, Process belongs to the "Interpretation". super-class.

5. Properties. Are Attribues.

COmponents.

Defining Characteristics.

Data Members

Members.

Fields.

Properties can be "natural kinds", or composite properties aggregating concepts.

Every other class has properties.

THe concept of a property is that of characteristics.

The usage of a property is that of identification, or differentiation. Properties can be used to unify objects and concepts, or to divide them.

Properties are objects. (everything is an object)

Properties fall into the interpretation, or clarifying aspect.

Property names are symbols that are used to evaluate the scope of conceptual spaces.

6. Collection.

Resource. Data. Store. Set.

Simple ontology. List + document types.

Document Types are collections of properties.

Experienced through a frame of reference.

May be a conceptual collection which represents a notion.

Structure of Resources. Map of Resources.

Interpretations and Intuitions and Understanding are resources.

Collections are structural.

Collections are objects, contain objects, contain properties, may represent properties.

7. Symbol

Information. Small self-contained units of data. Composed of Nothing. Empty. Irreducable representations.

Representation, Data, Number, Abstraction.

Signal. Sign. Message. Word. Letter.

Names are symbols that designate objects.

Visualization.

Properties of symbols.

Symbols are objects, and make use of the concept of abstract representation.

Symbols have no components. Properties are interpreted, not intrinsic.

8. Relationships consist of a special kind of set or collection.

They consist of objects as well as a property.

THey connect collections to one another and properties.

Colletions (of objects, concepts, sybmols, properties, ...) can be related to one another, or other collections heterogeneous.

An object can be related to an object through a property.

And an object can be related to a property. (through another property)

They are used to signify a property of a collection.

They are enabled by the process of identification.

Connetions. Links. vertex.

Set of pairs with values.

Functions are a Surjective relation.

Relationships are used for "structural intent" or "structural use"

And are comprosed and enabled through "structural process".

9. Use. Meaning, Purpose, Concext.

This refers to how an object from a class is used.

Use is functionally a representation for Meaning.

Behaviour type.

Part of Speech represents a use of a collection of symbols as it relates to the process of relating other symbols.

ACtion. Use is a type of process.

Purpose. Meaning. Concext. Semantic Context.

Essence.

This concluded the core classes that have well-defined properties.

Another core class is that of the user of the language.

10 . The User

The reader, listener, or audience.

Agent. Client. Speaker. Input. Frame. Boundary. Terminus. External Node. Uncertainty. Externality. Limitation. Environment.

Frame. Perspective. Dimension. Vector.

Other supplimentay kinds of class types.

Operatior - Conjuction.

And, or, if , if then , then if, if and only if. else. until.

Perepositions, Postpositions. Related to relationships between objects related to spacial and orientation metrics.

(applied to spacial dimensions.)

Punctuation - Document Type. Model of Well-formedness.

Word - Symbol
Pronoun - Pointer (special kind of symbol that points to a location in space.)
Article (a, the) - identifier that contains the property of context, scope, semantic meaning.

Adjective is a property.

Definition - Core property, Axiom.

Meaning, part of speech, use - Intent, Use

Relation - Set Function, Value

Sentence - Collection Notion thought Resource message

Participant - User

Adverbs - Modifier of process, or meaning, Descriptor of meaning.

that modify verbs - sub process, Meta-process- ex. Process Differently - or Process Descriptively.

that modify adjectives - sub property. meta-property. Ex. Identify Meta-describtively

that modify adverbs - Purposefully Alternatively Descriptively Process

One can see that the adverb plays a special role, and needs further investigation.

Tense

Voice

Aspect

Implicature

Google Needs a Classification System - such as Dewey Decimal

2006-10-20T11:02:00.000-07:00

Google Needs a Dewey Decimal System

Google's algorithm is already very useful for what it does - Delivers

likely possible solutions to what might have been the intent of your

search.

It will always be good at that, and continue to get better at doing

JUST THAT.

But, without a dewey-decimal-like system, it falls short as an

information retrieval ideology.

Google as ideology fails when it cannot realize that the world's

library belongs to the world; not to a private advertisers. Or information

hogs. Or media giants. Or software companies.

In, Of, and By the Public this Library will be.

Benefits of Distributed system.

In a purely technical sense, the distribted system's does not directly

make search better, in and of itself.

It allows search to become better indirectly by making the development

of search filters and classification systems to be developed and shared

by everyone in a standard and low-cost environ.

Aggregating resources is extremely easy with this system architecture.

This will encourage many current web publishers to adopt rigorous niche

classification systems that fit into larger-scope Labeling systems.

It also lowers the cost of innovation in all aspects of the search

process, by making crawling obsolete, and by making aggregation of

resources a technologically trivial process (even if it is still a complex and

multifarious process in terms of the information architecture, the

"code work" and interfaces are all taken care of. There will be various

competing providers of the classification schemes, and the markets will

decide what ones thrive. The architecture allows these to be made

available to the user.

In the exact same vein, it opens the market for enhanced providers of

filters and algorithms, and various processes that sort, map, and

predictively analyze the result sets imported from other tools.

In this way, the distributed system greatly changes the economics of

the many search engine development areas.

By isolating the various components of the "search engine', this new

system will foster greater custmization and innovation - The components

being the aggregation, the relevance processes (either symbolic,

conceptual, authoriative, socially computed, or a blend) and the sorting and

filtering algorithms.

The distributed system also changes the politics of search indexes,

social networks, and any kind of structured resource.

The current system does not aid those that seek exhaustive,

authoritative, definative document sets...not sets of possible matches.

When the search result set represents a complete, spam-free, and is

clear of off-topic content, then it can be seen as a datum in and of

itself.

It becomes an index, of a certian kind of query intent.

This datum then can be meaningfully combined, or summarized and

combined with other complimentary, or supplimentary resources to allow this

strucuted index to be used by other resources to provide greated depth

and breadth to the search experience.

The other fundamental limitation of Google is it's ignorance of the

intent of the searcher.

When searching for "jaguar", it gives some possible document lists.

But, to result of the "relevance" process, and the sorting process

cannot be considered aything but an attempt at possible relevance.

The sorting the results as a SET in this case is informationally

valueless, because the results collect from various contexts of the term -

the cat, the car, the sporsclub.

Its true that a subset of the results represent value to that searcher.

But it is the laborious job of the searcher to define that subset for

himself, unfortunately one document at a time. And, worst of all, even

after the searcher has essentially sorted the search results based on

relevancy for his intent, the information is usually uncaptured, or

completely lost, and even the implicit indirect clues lying in

user-behaviour that could be used to corrolate to the user's intent and satisfaction

with the results are horded by a company that can't benefit from it's

value, due to it's antiquated strategic central position as aggregator.

So, it is definately the classification that is the largest improvement

of all.

The distributed platform doesn't directly offer any solution itself to

classify all objects by mechanical processes.

But, it does allow systems of communication by PEOPLE, who after all

are the final judge of the merit of a search results for a query intent.

The searcher is the final judge of relevancy, why not make them the

primary judge?

After all, keyword density, pagerank, anchor links, are only

CORROLATIONS with authority and usefullness, they do not represent this

themselves.

And, the more the patterns are known by manipulators, the more they can

be gamed in that fasion.

Google suffers an adversarial classificaiton problem, there's always a

way in - spam, or investment in white hat SEO.

By lowering costs to build ontologies, (the dewey decimal systems of

tomorrow), and advanced search processes and filters - distributed

indexing has an indirect on search relevance, and cost.

Crawling the web is now a redundant process.  Just like other examples

of peer to peer technilogy making processes obsolete such as ripping

your own CD, when you can just download the MP3 from a p2p network.

Storing for the first time is done. Its just re-getting, finding from one of

many shared resources.

Looking beyond the page -

References - Google only makes it's final sorting decision based on

popularity, not the degree to which a document references other

authoritative sources.

That means a document that is highly referenced will have priority over

those that are not.

All else being equal, this is a good technique for determining a

corrolation to authority.

Some searchers may only want to search sources that themselves are well

documented, well-cited, and employ rich sets of references within their

document.

Having references within their document, to other authoritative

documents is a corrolation not only with relevance, but also authority in the

true sense of the word.

It is true that Google does give some benefit to those linking out to

sites with Good trust rank, and that outgoing links can effect the SEO

results, but this is more of a way to avoid the spam filter than

something that will cause a surge in the rankings.

Basically, there is no way for the searcher to specify the degree of

references in the documents they search. This is a limitation.

Not all queries require a document heavy in citiations, but, with spam,

and shoddy content, made-for-AdSense pages, the option to filter

documents without citations and refences is a necessary one for true

knowledge retrievers.

Document Density (of Topics or Concepts)

Just as contemporary web "Page" based search engines use "keyword

density" to estimate or predict the likely relevance of a resulting page,

the new system of searching "resources" will be largely influenced by

that resources proportion of documents pertaining to the given topic.

For instance, one resource may have fewer overall results, but have

results almost completely dedicated to your intended information.

THis is certainly a more time-efficient resource to begin with, and

resources of this type make better resources for beginners and experts

(perhaps not the same ones, but denser resources will be better in both

instances)

Sparse resources will many times actually give more results.

Anyone who's gotten over 1 million Google results for a query probably

hadn't made time to go through them all, and probably realizes that

this would be useless.

Sparse results mean that although the resource contains results, since

it is a miniscule part of its document set, it is unlikely to have

well-structured indexes and supplemental indexes pertaining to the topic

and is also likely to lack useful query-intent clarification procedures

to your specific intent.

The sparse resource may have fewer results, or may have more results.

THe more results usually contain garbage, spam, and "lesser sources",

one that probably aren't worth of informational content, but are

on-topic nonetheless.

Even if the sparse collection contains truly more "good resources", the

fact that they are essentially HIDDEN in the result set with other junk

makes it a less valueable resource.

The dense sources have none of these distractions.

The sparse sources may have more 'good resources', but they don't have

any classification system, or sub-indicies, or query clarification

tools to let a person search with tools that pertain to their domain.

In a sense, sparse sourcss are just one area of a tangled web. The

larger tangled web isn't structured conceptually, and neither will be the

'good resource' page results, however numerous they are.

The SCOPE of Progress

2006-10-20T10:17:00.000-07:00

The Public Venture : How public resources will revolutionize data

markets and information retrieval.

I show how centralized sources, obscured by secrets and

limitations, are not the future of finding information.

I see technology allowing large data sets to be interoperable, to

effectively merge, while I forsee the underlying business interests and

markets on a course to fragmentation into various components of the

info retrieval process. (crawling, indexing, aggregating, categorizing, sorting, filters and interfaces)

I give an idea of what the world might be like if social network

profiles began to use a system of publishing now used by blogs. T

his would be a more 'open' and free form of mass publication, due

to the non-exclusive role of large publishers to control their medium.

I shows how in his world, the popular social networking activity

would never be able to be acquired or controlled.

I no longer see the social network services delivered by the current

giants as remaining the  "destination" for their users. They will be

reduced to just a few of the many tools people use to reach and

communicate with the larger outside databases - The SocialSphere.

This doesn't mean that these companies won't be wildly successful as media companies, it

means that their power as exclusive social networks is inherently unstable, and they will constantly

have to re-define themselves.

Myspace will always be a successful web destination, but it's focus on

core social networking will have to get less and less, (just like

Google's focus on search)

Google says it's core is 70% search, but I think most of that is

advertising research and optimization. This clearly has it's place, but

Google has realized that to improve their search DRASTICALLY, would be to

admit there is a DRASTIC problem with their current technique.

Myspace will be the new MTV, with pretty faces, new studio promotions

and large brand atmosphere.

The users will remain, but will begin using more powerful tools to go

beyond only the myspace experience.

There is one area of innovation that large markets like Google and

Myspace cannot even go near.

That is the market of becoming part of a larger world of information

sharing.

Current, each social network is a gated community.

Even if membership is open, you can only search and see information

from ONE site, ONE database.

This means that if myspace doesn't get around to implement a feature,

it won't happen.

Even if the feature takes less than 1 hour of computer coder time.

Google will lose relevance as far as aggregating documents and lose some prominence in search, but will remain highly successful at incubating other internet properties, primarily in web media, and probably not as much in web widgets.

The reason is that anyone can make widgets, but not everyone can make

large media acquisitions which place themselves as or with large publsher networks and advertiser demand.

So, now google will remain strong from their cash position, and their

media portfolio, as well as their technological eminance as a data

processor.

Google will have an exciting future as a company, more focused on

advertising and media in the future rather than web document search.

Web search will continue to get more corporatized, with larger and more

numerous advertising placements. If a paid placement program was to

ever occur for indexing as it does with Yahoo's paid inclusion, it would

mean the end of the Google resource as we know it.

Evolving advertising trends suggest advertisers will begin to track the entire search experience, along with

email, social networking and video viewing experience to optimize campaigns

just for YOU.

Top 12 things Google will never do.

1. Google won't ever know what it's searching.

No conceptual understanding.

         No categorization scheme. Google doesn’t discern between fiction, nonfiction, reference, age-appropriate material, ...

2. Google will never understand what your intent was from your keyword. It can't understand the context or purpose of a search from just symbols alone. It has no conceputal framework.

Can't refine based on conceptual query intent, not keyword. Offers no way to refine search.

 Does "baseball" mean the "game", the "league" (MLB), or the "ball"? It

is used in all of these contexts interchangably.

3. Can't learn from users, or let users use groups to filter. The goal of 'making the world's information accessible, may be noble, but unlessy ou open the system to
LET OTHER make the world's information structured and accessable, it's going to remain a artificially "intelligent" GAME. - Feeling lucky?

4. No privacy, one company masses all data and sells or uses the data on its own.

5. Ownership and Control

         Censorship.

         Advertising corrupting experience or results.

         Market position inhibits integration, and possibilities.

         Non-monopoly status of search, and publishing industries.

By contrast, the Benefits of an Open Platform - open data, open summaries, open relevance scores,

sorts and filters.

          Customization, Upgrades, Flexibility

         More competitive markets

         Interoperability

         Non-monopoly status of search, and publishing industries.

         Everyone gets to use the cool toys at the Googleplex.

6. Never complete search of all libraries.

 If Google can't find it, forget about it. They won't point you in the direction of another resource that is known to have good results
for your kind of query. Google doesn't summarize other rich resources. They try to be the one resource. They can't admit their limitations.

Even Yahoo would direct you to another engine if your query gave no results.

You need to admit when another resource has better results. Google is too centered around 'documents' and not enough on resources as a whole.

7. Always prone to spam, manipulation through data, manipulation with

money, non-first-rate resources by design.

 There is bad spam, and then there is manipualation of results that are somehow approved, like buying web sites , buying links, doing PR stunts.

Does this lead to better results? Or just more "competitive" ones?

8. No authority, or authenticity, or certainty. Everything is just a

guesswork, probability, mass voting, mob rule, educated guesses. They

give you up to millions of results, because, you AREN'T feeling lucky.

 I can't even trust which mob I want to rule my results? I'm stuck with the largest mob.

9. Advertising Banditry. Publishers don't know how much they make.

Click fraud rampant. CPC model ensures advertisers pay the most. But,

publisher sees little of this.

Adsense inefficiencies.

10. Made only to search unstructured data; Can't understand structured

knowledge.

 Google's intent, and very assumption for it to work, is that it knows NOTHING about what it is searching in AND what it is searching for.

Google was made to sort the structureless web. It's great at that, but the structureless web sucks.

11. Only designed to crawl the outer web. The inner web is uncrawled.

 Other applications of distributed computing:

Anti-manipulation, Pro-accountability in areas of

Accounting
Elections
Trade

Google is a tool, not an answer. It's initial results, in aggregate,

are valueless without human sifting afterwards.

The environment that created Google was one of total lack of

cohesiveness to the web.

Google Needs a Dewey Decimal System

Google's algorithm is already very useful for what it does - Delivers

likely possible solutions to what might have been the intent of your

search.

It will always be good at that, and continue to get better at doing

JUST THAT.

But, without a dewey-decimal-like system, it falls short as an

information retrieval ideology.

Google as ideology fails when it cannot realize that the world's

library belongs to the world; not to private advertisers or a small band of dorks. Or information

hogs. Or media giants. Or software companies.

In, Of, and By the Public this Library will be.

Benefits of Distributed system.

In a purely technical sense, the distribted system does not directly

make search better, in and of itself.

It allows search to become better indirectly by making the development

of search filters and classification systems to be developed and shared

by everyone in a standard and low-cost environ.

Aggregating resources is extremely easy with this system architecture.

This will encourage many current web publishers to adopt rigorous niche

classification systems that fit into larger-scope Labeling systems.

It also lowers the cost of innovation in all aspects of the search

process, by making crawling obsolete, and by making aggregation of

resources a technologically trivial process (even if it is still a complex and

multifarious process in terms of the taxonomy,  the

"code work" and interfaces are all taken care of. There will be various

competing providers of the classification schemes, and the markets will

decide what ones thrive. The architecture allows these to be made

available to the user.

In the exact same vein, it opens the market for enhanced providers of

filters and algorithms, and various processes that sort, map, and

predictively analyze the result sets imported from other tools.

In this way, the distributed system greatly changes the economics of

the many search engine development areas.

By isolating the various components of the "search engine', this new

system will foster greater customization and innovation - The components

being the aggregation, the relevance processes (either symbolic,

conceptual, authoriative, socially computed, or a blend) and the sorting and

filtering algorithms.

The distributed system also changes the politics of search indexes,

social networks, and any kind of structured resource.

The current system does not aid those that seek exhaustive,

authoritative, definative document sets...not sets of possible matches.

When the search result set represents a complete, spam-free, and is

clear of off-topic content, then it can be seen as a datum in and of

itself.

It becomes an index, of a certian kind of query intent.

This datum then can be meaningfully combined, or summarized and

combined with other complimentary, or supplimentary resources to allow this

strucuted index to be used by other resources to provide greated depth

and breadth to the search experience.


As long as there is still noise and potentially bad results, Google's results don't represent knowledge.

The user has to turn that information into knowledge by going through and evaluating the results.

The other fundamental limitation of Google is it's ignorance of the

intent of the searcher.

When searching for "jaguar", it gives some possible document lists.

But, to result of the "relevance" process, and the sorting process

cannot be considered aything but an attempt at possible relevance.

The sorting the results as a SET in this case is informationally

valueless, because the results collect from various contexts of the term -

the cat, the car, the sportsclub.

Its true that a subset of the results represent value to that searcher.

But it is the laborious job of the searcher to define that subset for

himself, unfortunately one document at a time. And, worst of all, even

after the searcher has essentially sorted the search results based on

relevancy for his intent, the information is usually uncaptured, or

completely lost, and even the implicit indirect clues lying in

user-behaviour that could be used to corrolate to the user's intent and satisfaction

with the results are horded by a company that can't benefit from it's

value, due to it's antiquated strategic central position as aggregator.

So, it is definately the classification that is the largest improvement

of all.

The distributed platform doesn't directly offer any solution itself to

classify all objects by mechanical processes.

But, it does allow systems of communication by PEOPLE, who after all

are the final judge of the merit of a search results for a query intent.

The searcher is the final judge of relevancy, why not make them the

primary judge?

After all, keyword density, pagerank, anchor links, are only

CORROLATIONS with authority and usefullness, they do not represent usefulness

themselves.

And, the more the rigid rules are made apparent to manipulators, the more they can

be gamed in that fashion.

Google suffers an adversarial classificaiton problem, there's always a

way in - spam, or investment in white hat SEO.

By lowering costs to build ontologies, (the dewey decimal systems of

tomorrow), and advanced search processes and filters - distributed

indexing has an indirect and pronounced effect on search relevance and cost.

Crawling the web is now a redundant process.  Just like other examples

of peer to peer technilogy making processes obsolete such as ripping

your own CD, when you can just download the MP3 from a p2p network.

Storing for the first time is done. Its just re-getting, finding from one of

many shared resources.

Looking beyond the page -

References - Google only makes it's final sorting decision based on

popularity, not the degree to which a document references other

authoritative sources.

That means a document that is highly referenced will have priority over

those that are not.

 All else being equal, this is a good technique for determining a

corrolation to authority.

Some searchers may only want to search sources that themselves are well

documented, well-cited, and employ rich sets of references within their

document.

Having references within their document, to other authoritative

documents is a corrolation not only with relevance, but also authority in the

true sense of the word.

It is true that Google does give some benefit to those linking out to

sites with Good trust rank, and that outgoing links can effect the SEO

results, but this is more of a way to avoid the spam filter than

something that will cause a surge in the rankings.

Basically, there is no way for the searcher to specify the preference for rich

references in the documents they search. This is a limitation.

Not all queries require a document heavy in citiations, but, with spam,

and shoddy content, made-for-AdSense pages, the option to filter

documents without citations and refences is a necessary one for true

knowledge retrievers.

Document Density (of Topics or Concepts)

Just as contemporary web "Page" based search engines use "keyword

density" to estimate or predict the likely relevance of a resulting page,

the new system of searching "resources" will be largely influenced by

that resources proportion of documents pertaining to the given topic.

For instance, one resource may have fewer overall results, but have

results almost completely dedicated to your intended information.

THis is certainly a more time-efficient resource to begin with, and

resources of this type make better resources for beginners and experts

(perhaps not the same ones, but denser resources will be better in both

instances)

Sparse resources will many times actually give more results.

Anyone who's gotten over 1 million Google results for a query probably

hadn't made time to go through them all, and probably realizes that

this would be useless.

Sparse results mean that although the resource contains results, since

it is a miniscule part of its document set, it is unlikely to have

well-structured indexes and supplemental indexes pertaining to the topic

and is also likely to lack useful query-intent clarification procedures

to your specific intent.

The sparse resource may have fewer results, or may have more results.

THe more results usually contain garbage, spam, and "lesser sources",

one that probably aren't worth of informational content, but are

on-topic nonetheless.

Even if the sparse collection contains truly more "good resources", the

fact that they are essentially HIDDEN in the result set with other junk

makes it a less valueable resource.

The dense sources have none of these distractions.

The sparse sources may have more 'good resources', but they don't have

any classification system, or sub-indicies, or query clarification

tools to let a person search with tools that pertain to their domain.

In a sense, sparse sourcss are just one area of a tangled web. The

larger tangled web isn't structured conceptually, and neither will be the

'good resource' page results, however numerous they are.

Roadmap to Realizing the Open Platform's Promise

2006-08-30T17:40:00.000-07:00

Roadmap to Realizing the Open Platform's Promise

We've seen what can happen when powerfully integrated technologies can mash together complimentary data and processing components from various independent sources, and extend the functions to a larger set of capabilities.

We're watching web pages grow up to become services, or to use services. We're seeing web
software operate anywhere.

The networking and (meta)informational structures are both experiencing wide development.

Until now, there has not been serious discussion related to implementing a completely open data store, along with it's open components for serving, retrieving, analysing, and publishing data from the tool set.

The time has arrived for a myriad of open data stores to be supported to become a resource more vast, powerful, and leveling than anything the world has never seen.

What is the Opportunity?

The opportunity is to combine the recent developments in social computing and interoperable document exchange formats to create a scalable document and resource repository to make a large amount of documents available to be structured, tagged, classified and processed by users of the open public resource.

This technique combines the meta-data promised from the semantic web, the community tagging made popular by Web 2.0, as well as the distribution system from the Handle system, and ontological classification systems such as the Ontological Web Language.

Most organizations won't aim to house all of the data that is available.

Even for the ones that attempt to 'grab it all', they won't be able to organize or structure it on their own.

That is why the distributed system for summarizing other resources is so important.

Some groups will attempt the current model of trying to grab everything, (and doing very little human sorting and tagging with that unmanagable amount. The amount is especially unmanageable when it is behind a black box to the rest of the world.) Current data stores suffer from too much data, and not enough knowledge.

Many other groups will find that there are many companies that are actually already pretty good at "getting everything".

They just aren't good at finding everything when you need it. Without structure, organization or even an attempt at classification, it's really a tangled and unmanagable web.

New specialty "search engines" will find their true value-add by concentrating on storing only documents within certain domains of knowledge, or only indexing portions of documents, or structural relationships among documents.

I hesitate to continue the usage of the word "search engine". The term is dubious, a search engine consists of many components including:

Document / Resource Index
*Algorithm for relevance
Algoritm for Sorting by Influence or Authority

* One great way for relevance beyond keyword searching, in fact, the only INTELLIGENT way to do it, is by using human-created conceptually structured Ontologies (Ontological Web Language - OWL)

The major problem of today is that the index is locked down, only documents are given.

The algorithm doesn't work with real conceptual structures, it deals mostly with keyword density.

Thus, authority is still very easy to fake, since it is determined probabalistically.

The engines never know who to trust, or what is what, or what belongs where - so they take votes, and often, the crowd is wrong, and manipulators of the crowd can cheat the results, whether they see it that way or not.

The current "engines" treat the world's knowledge as nothing more than a string of symbols without any kind of anchor to a structured conceptual framework.

The influence algorithms will always remain a vital part of information retrieval technology, and companies such as Google will continue to thrive on their intelligent processing techniques.

What the opportunity is, to level the playing field related to processing large document stores, thusly enabling distributed ontologies to be created that can be applied to large amounts of important documents.

The new service doesn't promise to get every last document in the world. It promises to know what the ones that are important for what topic. Where they belong, what they are about, and what documents are about them.

It also allows anyone to process as much of the raw data in the index as they wish, which they can summarize, and then offer their summaries as part of their own searchable resource.

What are the limitations of the current system?

When google crawls altavista, it treats it like every other web page, based on it's keywords.

Never mind the fact that altavista happens to contain billions of web sites available by plugging in keywords.

This is only treated for how it appears as a "site" to a browser, not how it offers it's "information" to its users.

The same problem occurs with specialized search engines and resources.

Many times, a document is not the best resource for a query, and a resource such as a specialized search engine is an entirely viable return for a query. It may be best to think of these resources as complimentary results that enrich rather than replace the current search tools.

Something like Google won't be able to handle this, because every resource is just another page to them.

Whether you can search 1 document, or 10 billion, google goes by the external pages only.

So, you're missing out. You can't go where Google hasn't gone. Nevermind the fact that Google never really knows where it's going. It doesn't know what it's pages refer to, it could be intended as fiction, or fact, or it could be presented as fact but be otherwise. In any case, it's all just a list of symbols as far as Google is concerned.

"The wisdom of crowds...isn't."

The benefit is for other groups to query large engines with specialized queries, or indexing certain components, and then summarize this data, and make it available to their own searchers (and other data banks).

This can be thought of as pre-processed data.

How would you like your information retrieval oracle to bring you this message?

"We've already search everywhere for what you want, and we've found that these are the places that have the most organized occurences in their query results. SEARCH HERE"

I don't know about you, but thats the kind of information I'm looking to retrieve.

Its the classification, stupid.

What allows this to happen now?

Well, every network is effected by itself as well as other external networks and users.

The web industry has made real strides in software interoperability.

Most of the techniques that comprise the new "open platform" have been known for at least 5 to 10 years, if not much earlier in a more general form.

Hardware directly allows the information to scale, and hardware cost reduction has also allowed more users to come on board.

Someone could write a book on the emerging formats and hardware and user patterns, and what drove what.

I think it's not so important as to who is driving, but where the destination is going to be.

Lets look at this further.

What are the components?

The first compontent is a general index of resources. A resource could be a single document, a small organized group of lists, or a vast index with various levels of structure.

The network architecture for system to synchronize and communication will be implemented by the Handle system, similar to what manages the DNS servers in a non-central way.

Unlike the DNS server, there will be more layers than just the IP address and domain name.

There will be a Topic Index Layer, Various Levels of Index Broker Layers, as well the Site Broker Layer (what we currently think of as a "search engine"), and of course the documents.

This is basically a minimum of 4 tiers, with the index brokers having many levels of organization possible.

(Remember, the classification systems, known as ontologies form another independent portion, as well as the lexicon(s) used by the searcher and ontologists.)

The topic broker is the "oracle" that aims to point you in the right direction, even if that is only to another search engine of search engines. (don't worry, its going to be automatic)

The index brokers will tell you what search engine(s) you should use.

The site brokers will act as a current search engine, but they will be often be evaluated based on their own aggregate scores.

Think of search engines themselves being scored by their own keyword or pagerank scores, based on how good they are at finding certain things. This may include things like relevant document density, (similar to keyword density) as well as document structure, which ensures that the document is not a dangling page, but part of a larger knowledge web.

And, finally, my personal favorite layer, the final end result documents we were looking for all along.

This may sound complex, but it's truly how the open platform has already been planned.

Who is going to do this?

The Open Information Platform will take many players working in concert to achieve it's goals.

The first layer is one or more topic brokers.

These will be the ones who try to get everything, but unlike Google, they don't pretend to be an expert about everything.

If another index knows more than they do, they'll tell you, and actually bring you there automatically.

The index brokers will specialize in topics, or restricted indexing of specific components of web documents, either over a broad or narrow set of documents.

Other index brokers will be indexers of other indexers.

The site brokers act like today's modern search engine, in that they return documents, but they operate with a DEFINITE ontology, with authority (may be custom configurable and come from yet another 3rd party (actually, it would be a 6th party, but who's counting)

This allows these site brokers to return documents based on authoritative models of relevancy, not just keyword heuristics.

Then there are the documents.

I didn't want to leave these guys out.

These guys will interface with the ontologies in both explicit and implicit ways, both on and off their sites.

Parts of the new meta-data exchange will be enhanced by extra tagging, and "trackbacks" that let you know what places index, cache, warehouse, summarize or monitor that document.

This is primilary used for the other engines to get alerted when an update occurs, rather than having to chech 3 times a week like Google does.

Documents, their additional object naming characteristics (documents become entities beyond their simple location.)

A location can always be down, or moved, or abandoned.

The idea of a name or a document being "down", is just as absurd as the concept of "Alexander the Great" being "down" on all areas of the web, like wikipedia and all other engines.

Other ontological information related to the documents will be stored by third party ontology providers that work using the document names and extended properties.

So, to say it again, who are the players?

The architects. They make the standards and solutions. Or, at least the core system of protocols to communicate the objects of the repositories, their names, and the locations of their extended data and relationships.

Actually extending the data and the relationships among objects is the domain of the ontologists.

So, there are many layers, including the Network, the ontologies, as well as the interfaces, and various processing and sorting algorithms for relevance, influence and credibility.

The topic and index brokers. They go for everything within their domain, and specialize in redirecting people to the best resources, not the best documents.

As well, this new open platform will give rise to the potential to process, research, summarize, map, maintain and support the ever-increasing wealth this information can bring when made open.

Effects on the World :

This technology and it's integration will do more than transform the world of "IT", or the "internet".

Its scope will ripple to reach every corner of the earth.

Time is inherently more valuable to someone having access to this toolset.

This is a large change in the relative value between time and capital, or other physical property.

Time will continue to leapfrog in value, and not the least for the underprivilaged.

To put it simply, the implementation of this Idea is the Trickle Down theory of Knowledge promulgation.

Nothing, and I mean nothing will do more to close the digital divide, than this format.

Forget about putting computers in schools and villiages.

When this open platform reaches it's inflection point, the hardware will invest in itself.

This concept is truly the Reaganomics of philanthrophy through growth in to economies rich and poor.

No other media format will level the world as much as this.

And our society will never be quite the same.

Propagation Strategy.

So, who are the early adopters? How do we deal with issues of existing organizations not wanting to lose their competitive edge using their closed networks?

There are many groups who need to become more aware of how close we are to achieving this open system of Information Retrieval.

They are:

The Bloggers - leading the front line in Tech, Business and Politics. These are the scouts and rangers of the Movement.

The Academic world - This group has engineered the infrastructure and will process and shape it's progendy. They are the research labs.

The Corporate World - The ones who brought all the components together, and will eventually adopt and integrate this system to their offerings, either as a server, or client, or a complex blend. (The prosumer, to use a Alvin Toffler word.)

Government - You guessed it, the State. No one can invest in infrastructure like the Government. And, no one stands to gain more from the industry this platform will incubate.

There are also other groups, such as non-profits, non-government organizations, philanthropic organizations, as well as political groups who can benefit from this framework.

Challeges -

The very naming of a concept this vast is highly problematic.

Saying that it is an upgrade to the "web" is quite demeaning, because the web only applies to HTML documents made primarily for display purposes.

Terms like web 2.0, semantic web, social computing, active networks, distributed indexing may talk about one benefit, or allude to an evolution of sorts, but they all fall short in describing it.

Neural Name Net ?

Hey, it's got something going for it.

Everyone of the terms has something going for it, but misses others.

Other challenges will be dealing with resistance from existing providers of information retrieval technology, and large data warehouses like myspace, google, amazon, and all the rest of the capitalist machines.

Personally, my latest codename for the project is S.C.O.P.E -

Social Computing Open Protocol Environment

It also has a meaning as in Scope, the tool that one uses to scope.

And, the act of scoping.

As well as the scope of what is being surveyed, as well as the grammatical idea of scope, which determines the scope of a modifier - this is similar to meta-data.

[I had originally made a reference to the parable of david and goliath, and tried to change the story to fit my needs..

It turns out this was ill-conceived after a marathon blogging session.

The correct parable is the Tower of Babbel, for humanity to build a tower to reach the heavens.]

One one side is The old way of communications, with his war chest of cash, his data, his algorithms, his designers, his mathematicians, his venture capital ties, his investment banking clout, armies of shareholders behind him.

It would seem like nothing could stop this process.

But look, just next to every company is another company, with just as much cash, data, algos, and just as many people supporting his reign.

Now it seems there are a sea of companies, all seeking to take from one another, or to buy them out outright.

It would seem that nothing could get in the way of the mega-companies that combine the great Golliaths of the past.

But, in the distance, without anyone listening or paying attention, is the linguist, having nothing but his own language for cooperation.

He gives nothing but a whisper to the meekest of groups, and at that moment, every company will now either have to speak the new Language of Collaboration, or be history.

Let knowledge trickle down and let ye breathe wisdom.

MR

Wake up Web 2.0 to the Challenge

2006-08-14T16:56:00.000-07:00

Hello fellow web 2.0, semantic web, mashup mavens,

My intention is to uncover one very large-scale implementation that incorporates the fundamentals behind what people are calling, web 2.0, and how it can be extrapolated to its full potential.

First, I want to point our attention to an interesting comment made by none other than the Google duo as their famous research paper on their PageRank and Centralized Repository.

The keywords would be "Anatomy of a Large Scale Hypertextual Search Engine"

This was written at a more academic, idealistic time, where, speculation and ideology were still delivered with the intent of forthcoming academic insight and predictive precision.

Interestingly enough, the paper closes with this thought about the limitations of Google's Centralized Indexing system, and the next frontier of search, which they identify as distributed, de-centralized search indexing.

Here is the final paragraph:

Of course a distributed systems like Gloss [Gravano 94] or Harvest will often be the most efficient and elegant technical solution for indexing, but it seems difficult to convince the world to use these systems because of the high administration costs of setting up large numbers of installations. Of course, it is quite likely that reducing the administration cost drastically is possible. If that happens, and everyone starts running a distributed indexing system, searching would certainly improve drastically.

-Sergey Brin and Lawrence "PageRank" Page

As we can see, the famous duo has given us some wisdom and parting thoughts as to the most important evolution in information retrieval.

A decentralized database is clearly a beautiful evolution.

Its already happened for blogs.

Each blog is just a mini index of posts. Totally searchable on its own.

But, blogs have adopted standard document types to facilitate RSS feeds that allow them to be aggregated, processed and indexed elsewhere to be available for other users and aggregators.

Blogs have easily demonstrated one possible way to make content sharable, and indexable and readable throughout various web services.

In fact, one of the most interesting evolutions in the social networking arena can be made possible by something as simple as allowing "Friend Feeds", or "Profile Feeds", which enable the aggregation of profiles in other places.

Some aggregators will focus on being comprehensive and to cover as many major sources as possible.

Other aggregators will offer specialized databases of specific domains of knowledge.

How easy would it be to make tools to model and process the friend-o-sphere, and the web-index-o-sphere when an open distributed decentralized database was fully available to compliant warehousers willing to maintain the integrity of the system?

I would say that this is nothing but a true revolution in media reaching it's full potential.

Can the social sites that want a piece of the myspace pie afford NOT to make use of this extensible platform?

Can search engine underdogs afford not to commoditize the web index, to function more on processing, user behavior, and more calculation, and thusly turn the world of "search", upsidedown?

The new data warehouse model is not that of the "site", it is of the "service", or feed.

Sites will continue to be locations to access and process data from web services.

For more information on our Social Networking Initiatives, please visit

http://www.connectsociety.com

And for information on our web object repository framework, please join us at

http://www.linkassociation.com

Search is going to fragment into it's disciplines: compilaton, structuring, and retrieval.

Search is going to fragment into it's disciplines: compilaton, structuring, and retrieval.

Web 1.0 - going to my library, doing a keyword search for books at my library building.

web 2.0 - going to your library, using the computer to search all libraries, or only the libraries relevant to you.

- Being able to make use of one or more supplimental classifiction systems for all or part of the library. (such as dewey decimal, library of congress, or any other well-defined topical heirarchy or ontology)

- Being able to use any retrieval technique from any library in the world. (not just the ones your library offers.)

Thats just the beginning. Now that this rich data is so available upon call, the software to know what you want will get even better.

Snippets from a paper on Distributed Indexing. (Danzig) [2]

This project describes an indexing scheme that will permit thorough yet efficient searches of millions of retrieval systems.

We report here an architecture which we call distributed indexing that could unite millions of autonomous, heterogeneous retrieval systems.

We can make sophisticated users out of naive (non-expert) users without employing artificial intelligence. We do this by making it easy to create autonomous databases that specialize in particular topics and types of queries.

We say such databases are precomputed because they contain the results of executing particular queries on thousands of other databases. Precomputation avoids the need for users to broadcast their queries to hundreds or thousands of databases. It eliminates work, reduces query response time, and helps locate obscure information.

The idea behind distributed indesxing is to establish databases that summarize the holdings on particular topics of other databases.

(In the system:)

An index broker is a database that builds indices. [...] Index brokers can index the contents of a "primary database" and, in fact, other index brokers.

Primary Databases are today's single-site retrieval systems such as [search engines] and online catalogs.

Site brokers store the "generator queries" (resource descriptions) of all index brokers that index their associated database, and are responsible for keeping these index brokers' indices current.

Site brokers apply their generator queries against these updates and reliably forward appropriate changes to the index brokers that originally registered the generators.

Users of the index broker may have to contact the primary databases to retrieve the object itself.

[the primary object is not obligated to give the index broker a copy of itself, nor is it prevented from doing so.]

Our architecture extends attribute-based naming to bibliographic databases.

Topic broker replication is implemented with a flooding algorithm.

Creating a new index broker entails describing its contents and selecting a set of primary databases and index brokers to index.

The broker's "generator", the query that it registers at the site brokers of the databases that it indexes, defines an index broker.

...topic broker, the one logically centralized component of the system (site brokers of primary databases also register a generator and an abstract with the topic broker.)

The topic Broker that first registered an index broker informs the index broker when a pertinent new database or index broker is created.

Executing a query involves a sequence of steps that eventually identifies or retrieves a set of relevant objects. These steps are query specification, query translation, broker location, broker querying, primary database location, primary database querying, and object retrieval.

[end notes from Danzig]

We need more than a few aspiring research mega-labs to be crunching this data.

The world years to breathe knowledge.

Toward Fellowship and Advancement,

MR

References:

1. Anatomy of a Large-Scale Hypertextual web Search Engine
Brin, Page. 1996

2. Distributed Indexing: A Scalable Mechanism for Distributed Information Retrieval
Danzig, Ahn, Noll, Obraczka. (danzig@usc.edu)

http://www.techcrunch.com/2006/08/08/web-20-the-24-minute-documentary/trackback/