Collective Knowledge Systems

Collective Knowledge Systems: Learning from Everyone Who’s Been There, Done That

The dawn of the Web (Web 1.0) revolutionized publishing and content consumption, with the Web browser as the universal interface to publishing and ecommerce sites. But production power still resided with publishers, with consumers as passive recipients. After the dot-com collapse, the consumer became a producer. 

The masters of Web 2.0 — WikiPedia, Flickr, Google, and Amazon — thrived by harnessing the activity of consumers contributing content or intent. The power to inform, share, discover, or sell things was now powered directly by millions of consumers.  “Collective intelligence” meant the information value that could be algorithmically derived from that user participation. This was also the beginning of what was called the “Social Web,” in which user-generated content, and its ability to get noticed, comes from a conversation among users — not centrally managed publishing.

Having done an early attempt at collective intelligence with Intraspect, which targeted the enterprise, Tom was excited by the possibility of aggregating the knowledge of internet consumers. Once again, he sought  to find new ways to aggregate the knowledge of people working online into a common knowledge base. The domain he chose was travel.

People spend a lot of time researching travel. Having good information about where to go, what to do, and where to stay can make the difference between an unforgettable experience and a waste of precious time off. In 2005 Tom had just spent a year traveling, and he learned first-hand that the very best source of information for these questions is people who have been there. But you can’t just ask your friends. There are thousands of great places to go and millions of places to stay; your friends haven’t been to all these places, and even if they had it would take too long to ask them all. Tom wondered: How we could create a system in which you could, in effect, ask the world of travelers for their advice? As with Intraspect, the solution didn’t lie in searching what was already there. It laid in building a system in which people are motivated to create the content needed, with increasing returns as more people participate.

With that goal in mind, RealTravel was born. The site provided an environment for a community of travel enthusiasts to create beautiful travel journals of their adventures, share them with friends and family, and find other like-minded travelers. People looking for information about where to go, where to stay, or what to do could learn from the authentic experiences of those who had been there.

Any collective knowledge system has to solve two basic problems: how to motivate people to contribute high-quality content, and how to harvest value from that content for people who are learning from the collective knowledge base. RealTravel solved the problem of motivating users to contribute content by providing a beautiful publishing platform for sharing travel adventures. Its design goal was to give every traveler the same platform as a professional travel writer publishing in a slick travel magazine, fueled by the capabilities of Web 2.0. RealTravel called them travel blogs, and people could easily put together ongoing stories with itineraries, photos, reviews, journals, and conversations with people who followed them. As people shared their experiences, a collective knowledge base of user-generated travel advice took shape.

To solve the problem of harvesting value from collected travel experiences, RealTravel used semantic computing and AI. Everything was indexed by location and time, so it would be easy to find all of the travel experiences about a place (not just the name but its geopolitical and geospatial location on the globe, which was new to the Web back then). The time index allowed people to follow adventures over time and also know what it is like to be somewhere for a particular season or event. For example, travelers could browse a portfolio of destination photos taken by many travellers during a popular festival. Second, the company added authoritative, published information to augment the user’s content about places to go and things to do. That meant that while you were exploring someone’s trip, you could find out the price of a hotel or read a relevant piece from a travel book. Third, users could annotate their adventures with tags that were gently curated to offer useful social indices based on what people love about travel, such as Nature, Beach, or OffTheBeatenPath. Fourth, the stories that travellers wrote were “read” by unsupervised machine-learning algorithms that could cluster blogs into categories which weren’t covered by tags. Tom and his team were surprised to find that the machine-learning algorithms discovered categories such as travel “for the adventure of food,” or “for religious enlightenment,” which the human editorial team had missed.

The resulting knowledge base of travel experiences could be browsed along any of its indices: location, time, traveller, tags, and content types (journal entries, photos, hotels, restaurants, things to do, etc.). A dedicated search engine could combine any of these indices for highly targeted search, and allow generalizations up and down the location hierarchy (country, state, region, city, neighborhood). There was also a recommendation engine that interviewed the researcher on their travel goals and constraints, and returned personalized recommendations with stories from bloggers who had been there and who matched the researchers’ profiles. Altogether the site applied technologies from asynchronous collaboration, knowledge management, social networks, GIS and mapping, tagging, content management, and machine learning.  

Within a year of its release in 2006, RealTravel had millions of monthly users and pretty good coverage of the major travel destinations around the world. It ultimately dissolved into a consolidation of the online travel industry. Today, most of the information that could be found in RealTravel is provided by TripAdvisor, which expanded from hotel reviews and now dominates the category. But it doesn’t read like Condé Nast Traveler.

The experience of building RealTravel gave Tom insights about combining the Semantic Web with the Social Web which led to a keynote for the Semantic Web conference and a journal article which outlined generalized the principles of collective knowledge systems.

Tom is still an inveterate traveller and continues to document his adventures through photography. “The world is a book and those who do not travel read only one page.” – St. Augustine