What is Free Associate?

Free Associate is a system that creates an autonomous "personal context agent" that monitors flows of networked information. It assists the user by creating a personalized metadata framework for the creation and management of individualized knowledge structures.

Why is this needed?

Keyword searching is a brute-force way of finding interesting content. It doesn't work well, and it's not efficient. In addition, when you use a search engine, the results are presented in list form, which doesn't contextualize the results at all. The other option is surfing until you pick up an interesting thread or site, and then stumble upon something you like. In both cases, the search for interesting or relevant content is dependent on you seeking it out. Then once you've found interesting content, there are very few digital tools for organizing this information into a coherent and useful personal knowledge structure.

How does it work?

Free Associate is composed of two parts: an information harvester and an information visualizer.

The information harvester is an application that you run on a personal server. It has three main responsibilities: maintaining your personal digital knowledge library, building intelligent content filters based on your knowledge library, and then seeking out additional information to bring to your attention.

The information visualizer presents the results from the information harvester in a graphically intuitive method based on contexts and similarities, and allows you to do more with the results of the information than just click on the link. You can also rearrange the visual patterns from the results on the screen or add/delete other pieces of information from the screen.

Give me an example!

Let's say you were tracking news in Iraq. Your first options are to go to Google and type "iraq" or visit your favorite news sites, but neither approach gives you a concise overview at a glance. Using the agent, the harvester would surf multiple sites in advance. The visualizer would take the results and present a current visual summary based on your established preferences.

The visualizer is also designed to help tie together content elements by content and relevance in a source-agnostic manner. The harvester and visualizer are capable of storing any network-accessible data for inclusion in personal knowledge patterns. To return to our Iraq example, the Free Associate user could group together web pages, RSS news feeds, emails, PDFs, Word docs, and video clips related to the topic of Iraq. In a way, Free Associate acts as a "dynamic desktop" for organizing diverse networked content sources into a coherent personally relevant knowledge framework.

What else could it track?

• search engines
• text files (PDFs, Word .DOCs) stored on the user's server
• online web services (Amazon, Syndic8)
• RSS/XML news feeds
• Blog feeds
• email accounts
• instant messaging accounts
• calendar data (via WebDAV)
• address book data (via WebDAV)
• databases
• other Free Associate systems (if it gets that far!)

• travel planning
• monitoring web pages for updates
• to-do lists
• project management
• Internet memes
• presentation notes
• books and reading lists
• tracking OS/application patches and dependencies
• mapping out abstract ideas
• link management

The goal is to allow you to build complex patterns of information that combines Internet-based information with your own notes, comments, and thoughts. Once you've built these patterns, you can send them out to people so they can view your results.

If the concept proves viable, eventually the system would enable real-time interaction within shared "data spaces" based on content areas and interested communities.

What's the geeky stuff behind it?

The harvester is a PHP/MySQL-based web application running on a Linux server. It is capable of scanning RSS feeds, XML feeds, web pages, POP3 email accounts, instant messaging accounts, and almost any other standards-compliant data feed. It uses a Bayes algorithm to sort results based on your previous requirements and requests. The visualizer is a Flash MX application. Communication between the harvester and visualizer is via SOAP/XML. All components require a connection to the Internet.

What's been done so far?

The harvester is built and running and currently is maintaining a large database of automatically harvested web pages and RSS content. Next steps include integrating support for more data sources and file types, expanding the scope and reach of the harvester 'bot, and improving the intelligence of the Bayesian/LSI filtering mechanisms.

The visualizer is in an alpha development stage but is largely functional. Users can create information patterns, add/modify/delete content, access Google through the interface, and share the knowledge patterns via email. Next steps include completing the interaction toolset, expanding direct interaction with the harvester application, and finding new ways to make information display more visually coherent.

Eric has completed the EMMA program at the KMT/Utrecht School of the Arts. Further development of the system will be done as time permits. Suggestions and offers of collaboration are welcome, please feel free to contact me if you have any interest in the project or in the ongoing development process.

Available documentation:
• Project proposal (final)
• Exegesis form
• Running prototype
• August 2003 Thesis
  (accepted for EMMA degree August 28th, 2003)

Who is behind this project?

Eric Miller created Free Associate as his individual project in partial fulfillment of the MA requirements for the EMMA-DMD degree program at the Utrecht School of the Arts.