Self Sourcing

Information Value Analysis* is a term I came up with after years of presenting IBM’s collaboration software strategy. The company was making large investments in knowledge management technologies and we were discussing the need for better systems with our customers. One business concern that continually came up in conversation with my customers was how to identify and measure the value of intellectual capital produced by the knowledge workers of an organization.

Historically, businesses have spent a tremendous amount of energy trying to determine the location of corporate knowledge among the company employees. In the 1990s there were traditionally two ways to go about this process. The first was to create a profiling system. The profiling system would combine human resource information with a set of proclaimed skills that an employee would share about him/herself. The employee and managers would author the profile record. Then the information would be shared with the rest of the organization.

This approach led to a variety of problems. The profile tended to include biased and inaccurate information. Contributors who wanted to appear as high achievers would pad their profiles to make them look like stronger employees. Counter-intuitively, some employees would do the opposite and purposely obscure their abilities for fear of being bothered and overwhelmed by others in the organization looking for their assistance.

Another difficult problem was encouraging employees to continually maintain the data. Since employees are generally mobile within an organization and new challenges require different skills, many employee’s profiles would expire almost as soon as they were written. This was the nail in the coffin for manually fed profiling systems. They are still around but they are generally an expensive exercise in folly.

The second attempt to derive this information was through automated systems. Instead of having employees profile themselves, information about their abilities could be derived from searching through the documents they author and handle. By directing a search engine to ‘spider’ through documents written, edited, or forwarded by an employee, the system could make several determinations about the subject. Enough to make a much more accurate picture of the expertise within the organization.

IBM attempted this around 2000 with its Lotus Discovery Server product. The Discovery Server would produce a corporate taxonomy which included key concepts and terms that the business deemed valuable. The system would then search through all available documents to map relevant terms in a document to the taxonomy which they called a K-Map. In turn, the system would assign a value to the document based on it’s perceived usefulness by the organization.  Usefulness was determined through employee interaction with the document. The number of links to and from the document increased it’s usefulness. So did the amount of times the document was opened, forwarded, and cited. Furthermore, to avoid the expiration of the document’s relevance, the system tracked updates and revisions.

Tracking popularity and interaction with a document in the context of the organization’s taxonomy was a brilliant leap forward. It was much less biased and much more reliable than the arbitrary self-promotion profiling systems that came before. However, the system was terribly flawed. The system needed to be massively scalable and would require an overwhelming amount of processing power for the end result. It proved to be economically unfeasible.

Privacy was another issue. The system was only as good as the data that you fed it. IBM designers encouraged customers to spider through employee email because they logically decided that email applications contained the most relevant data. Although the system was only looking for key phrases and concepts that aligned to the K-Map taxonomy, there were very few customers who felt comfortable having the system riffle through their email.

So, with both manual and automated profiling systems failing to deliver the promise of measuring and identifying the intellectual capital of unstructured data, companies surrendered to the realization that extracting that information was too complicated, flawed, and expensive.

Then something brilliant yet simple happened in the commercial market space. Applications engaged in crowd sourcing started to appear. It is likely that many of these application designers were not aware of crowd sourcing when they started building their applications and they stumbled into it with the help of the new design philosophy of Web 2.0. Wikis, social news, and social bookmarking sites provide the best examples of crowd sourcing but it is the news and bookmarking sites that broke the ceiling on Information Value Analysis.

The concept of on-line social bookmarking was very simple in it’s inception. It solved the problem of accessing favorite websites without the need for a specific computer or web browser. By placing the bookmarks on the World Wide Web, users could access them anywhere. This was enough of an incentive to get a large consumer audience for the product.

Yet it was the analysis that you could derive from the behavior of thousands of users that became extremely useful. By observing the bookmarking choices of a community, you could extrapolate the value of the target website to that community. Better yet, the cost of gathering that data was almost zero because the workload was distributed across the community.

Social News Websites were originally invented to remove the editorial bias from news publications. Companies such as Digg realized that the perceived value of an article would determine it’s placement on the newspage and many articles that were significant to audiences were buried and went unread. To remove the determination of the value of the article from the publisher, Digg built a site of news article links that readers could rank by voting. This act of voting on the news assigned the real value to the information to the Digg community.

This technique of identifying value through user interaction is the key to solving the profile and contribution dilemma that haunted companies in the 1990s. Not only do these systems work more efficiently, they are a fraction of the cost to build and maintain.

In the consumer marketplace, companies such as Facebook are well aware of the power of identifying the value of information. Their success is further strengthened by their ability to reach infinite numbers of communities and return information that each instance of community finds relevant. The design of social networking tools took social news one step farther by creating fluid communities instead of one community that defines itself into isolation.

Now, with these social software tools available in the market, companies can revisit the conversation around knowledge management, profiling, and expertise location. Of course, companies should look at tools they can bring in house or at least subscribe to as a private service. Once the infrastructure is set up, the next step is to create a positive social networking culture within the organization. This will happen automatically when the content of the social networking system reaches a critical mass of information. Once workers realize that a social network can be the fastest means of identifying and retrieveing valuable information, participation will start to expand exponentially. Companies that realize how Social Networking tools can not only manage our social lives but act as a gateway to solving age old business problems will find themselves with a critical market advantage.

-Marc Dreyfus

*[The term is also used by engineers to decide the cost/value of uncertainty within a study. The avoid confusion, the full name of the concept is ‘Unstructured Information Value Analysis.’]