Note5

Can We Organize Our Information Better, Please?

Jun. 27, 2010

As users of the internet in this modern age, we have no shortage of options to both store and consume information. Does that mean that keeping track of our information is easy? Not really. The proof in this lies in the fact that we have so many options for storing the content we produce and consume. We keep our videos on YouTube, photos on Flickr, our current thoughts on Twitter, and our written content everywhere from Google docs to Blogger to Scribd to Slideshare. If you’re paying attention to trends, you’re aware of several variations on these themes as well.

Each of these services is great. They all have a ton of content that finds it’s way to the world. However, with a great volume of content, the means of organizing that content so it can be easily found is a struggle. How do these services accomplish organizing content? There are four main methods in use today.

1. Hierarchy – folders and trees
2. Tagging – concept application
3. Filtering – attributes
4. Search – when all else fails

Each of these methods of helps to structure information to a certain level of efficacy, but there are problems with each of these approaches.

Hierarchy – the problem of paradigm

The perspective with which we understand information is as unique as the person interpreting it. One man’s trash is another’s treasure. The same thing has different meaning depending on who comes in contact with it.

Hierarchies are very powerful means of structuring information. We all do it, making differentiation and parent-child relationships to understand the information we need to process. Enough with abstract statements, let’s talk about something more concrete – kinds of animals. If we try to understand kinds of animals, we might start by making a differentiation between animals that are domestic, and those that are wild. In doing this, we make two sibling branches of information. Then, for the domestic animals, we can make another differentiation between domestic animals that are pets, and animals that are agricultural. Here, we made a sub-distinction and created a parent child relationship between the domestic animals and the group of animals that are pets and used for agricultural purposes. We could continue to make categories of sub distinctions, but I think you get the point. Doing this creates a structured tree to help compartmentalize an entity of data.

The advantage of using a hierarchy is that it creates a very efficient means of organizing a large volume of information so it can more easily be found. In addition, with each step into another sub-distinction, meaning is collected. In the example of organizing animals into the structure above, we can see that a dog is a kind of pet that is domestic. It all sounds great. If the problem we’re trying to solve is that of keeping track of information and building meaning so that information can be properly understood, a hierarchy seems like a perfect solution.

Not so fast. Part of the problem with using a hierarchy is that people wanting to learn more about animals might not want to start with the same set of distinctions we make. If I want to learn about what kinds of animals are OK to eat, the structure mentioned above is not very helpful. Both wild and domestic animals have members that are save and unsafe to eat. Finding the information for a specific purpose that doesn’t match the purpose of the tree becomes completely inefficient. The paradigm is different.

Any tree of distinction can be a valid means of organizing information, but a single tree is most certainly not the exclusive way to organize it.

Tagging – the problem of misleading semantics

Tagging is an answer to the short comings of a classical hierarchical structure of information. With tagging, you can attribute a concept to an entity, and thereby connect things that don’t necessarily belong in the same ordered structure. Now, even with animals in different branches of a tree – domestic and wild – any where they are, they can be labeled “edible” or “inedible”.

Another advantage to using tags to provide structure to information is that tags can be free form. There is no restriction to the kind of concept that can be applied. This allows us to arbitrarily apply our own understanding to information. With tags, we place things into our own paradigms, however they may be structured.

While tags bring flexibility of understanding to the table, they have their own set of problems. Tags used exclusively for structuring data neglect to recognize a true and natural set of relationships that may exist for the piece of information. For example, if we are looking for animals that are “edible” we could find a specific kind of fish tagged as such, but we do not get to see how it might be categorized, and what other kinds of things might be naturally related.

In addition, the context and meaning of the information is lost by virtue of bringing things that are unrelated together. For example, if paper, chickens, glycerine, and tree bark are all tagged as “edible” the relationship is more of a synthetic coincidental type than a real connected relationship. If the things have nothing else in common, we only know that these things are edible, and nothing more. The value of context in which to understand these pieces of data is lost.

Terms can be overloaded too. By tagging something “Paris” are we going to find photos of the Eiffel Tower, a restaurant menu, or photos of celebrities? This, however, is the classic problem of semantics. As always, this can be overcome by sufficient clarification, but tagging is very susceptible to misunderstanding.

Tagging information helps us to customize the way we understand information in our own way, but it arbitrarily builds relationships between things that may not be naturally related. The result allows us to build synthetic structures around information, but in doing so, information about real and genuine structure is lost.

Filtering – the problem of detail without context

Information tends to have attributes. Color, size, shape, and cost are all common examples of items that might be in a catalog. By looking at attributes and filtering through them, in essence, we can build our own hierarchy based on properties that really exist for an object of information. Here, the power of a classic hierarchy is harnessed, allowing you to quickly find an item of information through a sea of others by making distinctions.

The power of tagging is invoked here as well. After all, tagging is itself a means of attribution. In this case, the tags are not as arbitrary, but relate directly to the properties of an item. In a way, attributes of an item of data are tags, but do not synthetic.

The problem here is that attributes are the details of an item. They are the things that belong to that item, the children of the hierarchy. A set of filters consisting of “leather”, “blue”, “has a buckle” requires context. These could be properties of a shoe, a belt, or a car. Filtering by attribute requires some other kind of structure to be meaningful. The details of an item do not tell us the context in which that item is properly placed.

While filtering information by attribute allows us to use the power of a hierarchy to find information in a custom categorization, the meaning to be understood from the organization of that information is neglected, and the value lost without it.

Search – the problem of knowing the right question to ask

When we can’t find information in any way that makes logical sense to us individually, what do we do? Search. By simply typing what we want to find into a text field, and relying on the mystic oracle to give us an answer, we delegate the work involved in finding information to an algorithm. Search is what happens when all else fails. We throw our hands in the air and ask for help. Fortunately, we are blessed with some reasonably good results. Search is a very effective way to find information.

Even though it is clear that search is not a means of organizing information, it bears recognition since it is the tool of choice for finding it. When the path to find that information is unclear and convoluted, search is how we find it. The problem of course, is that the mileage you get when searching varies depending on the words you type in the text field. If you’re too general, you get too many results. If you’re too specific, you find nothing. Constructing the right phrase to search for is an art form to find your intended result. Therein lies the challenge.

In addition, the results returned may be incomplete. The relevancy and accuracy are a matter of judgment placed in the hands of the almighty search index. The quality of that index is only as good as the places in which the program peers.

Indeed, search is a tool we all use to find what we’re looking for first. The simple fact today is that there is so much information available for consumption, that it is impossible to find what we are looking for without help. This last resort has become our first attempt.

Resolution?

Certainly, the challenges for organizing the information we are concerned about are daunting. While Note5 doesn’t make claim to cure the ails for these problems we all face, it does make an attempt at making it better.

While the ideas are still in embryo, the direction is clear. Note5 recognizes the absolute necessity to organize information in new ways. Stay tuned!