Indexing knowledge – Designing search engines for conflict and peace research

Software Being Developed to Monitor Opinions of U.S. – New York Times:
A consortium of major universities, using Homeland Security Department money, is developing software that would let the government monitor negative opinions of the United States or its leaders in newspapers and other publications overseas.

There’s an interesting article on the New York Times today on semantic analysis of textual information. The objective is not just to aggregate information, but qualitatively measure the tone of the article.

Ultimately, the government could in a semiautomated way track a statement by specific individuals abroad or track reports by particular foreign news outlets or journalists, rating comments about American policies or officials.

What’s interesting is that this technology does not capture that which is communicated through audio and video sites such as Yahoo! Video, YouTube (now owned by Google) and Google’s own Google Video. With the exponential growth in digital audio and video content, most of which is to be found in sites such as Flickr and those above, along with pod-casts, textual analysis alone may not be able to give a full view of social opinion is formed and is promoted, the interplay of youth / pop culture into mainstream political culture, subaltern movements etc.

The millions of dollars that are going into this tool, ergo, will be for the development of a sophisticated semantic analysis engine for text – that may have limited use in determining the root causes of terrorism and animosity against the US in cultures that are driven by, say, communications using mobile phones (SMS, MMS), or by a generation of youth who communicate using short videos and podcasts. Ironically, these may be the very people / population groups that the US wants to monitor the opinions of.

One way around this may be through cutting edge mashups of podcast, photo and online video searching tools. Imagine for instance a combination of:

  • Podzinger – a website that uses speech-to-text technology to create a text index of the audio, which enables users to find content within podcasts and jump directly to the point where their keyword is spoken.
  • Riya – A photo and image search engine that searches for content based on similarity
  • Clusty – Clusty queries several top search engines, combines the results, and generates an ordered list based on comparative ranking. This “metasearch” approach helps raise the best results to the top and push search engine spam to the bottom. Instead of delivering millions of search results in one long list, our search engine groups similar results together into clusters.
  • The Living LibraryThe Dropping Knowledge Initiative’s gargantuan collection of opinions and ideas from leading thinkers from across the globe.

A mashup that combined the above would leave Google trailing far behind in intelligent searches of textual as well as multimedia, and would, by extension, be far more powerful than the Homeland Security Department funded software tools mentioned in the NYT article.

Furthermore, such a mashup would be a significant leap forward in search technologies, providing a platform for the collation and analysis of information for a variety of purposes beyond a parochial interest in opinions for or against the US. For instance, such a tool, as has been envisaged in Peace Tools, could provide a phenomenally powerful foundation for peace negotiations and online dispute resolution processes, mapping peace processes and conflict transformation, and in general, honing the use of ICT in peacebuilding.

2 thoughts on “Indexing knowledge – Designing search engines for conflict and peace research

  1. One easy way to assist search engines in this area would be to adopt metadata standards for conflict and peace research into any publications put online. The United States Institute of Peace has provided funding for a project housed at the University of Pittsburgh to build a peace and conflict resolution thesarus to assist librarians (and mere web users) catalog and retrieve peace and conflict related material (

    What good is a thesarus? Well, a few search engines (e.g. Thunderstone) can use a thesarus to filter search terms and rank search results. Having common terms to desicribe similar concepts and ideas goes a long way to making the information retrievable. The UPitt thesarus is currently being tested by the International Network for the Promotion of Rule of Law ( as a possible way of organizing and retrieving material relevant to that community.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s