Continuing with part 3 of our 3 part series in the technology of Yapmap, this post will talk about the most obvious of the technology innovations, our user interface and the diagram of the conversations we call the Visual Results Map.
The easiest way to tell that YapMap is different is to look at a results page. It’s different, it's visual. There have been attempts to build visual search interfaces in the past like SearchMe, and Google recently added its Instant Previews feature to provide a visual component. Both of these attempts at visual search display an actual screenshot (or portion of a screenshot) of the search result.
When building the interface for YapMap, we decided to take a different approach. We viewed screenshot presentations as too dense and thus hard to peruse and simple page results as two sparse and contextually barren to understand and triage possible items of interest. An analgous example is that Google Maps doesn't show you all details at every level of zoom. You need the right amount of detail, for the right level of zoom. We built the YapMap results map diagrams at a level of abstraction that we think provides the appropriate level of detail for understanding, triaging, and exploring discussion search results.
To build the visual results page was more difficult than we ever would have imagined. A discussion thread is a very complex informational construct, to come up with the right level of abstraction and show the relationships between the posts, we had to create a new way to view the results and, more importantly, had to create the data structures to support the detail and relationships shown in the maps.
Building a 100 million Maps
When we started building the interface we looked at various types of technologies. Flash, HTML5/Canvas, VML/SVG were all considered and prototyped. Flash got nixed because the interface would have had to been monolithic, good Flash computer science types are hard to come by, and frankly it seemed like video playback has been the only that has saved it from dying long ago. HTML5 and Canvas features were very interesting. However lack of penetration and page load rendering stopped us from utilizing those technologies. VML/SVG was actually the most promising of the new technologies but was eliminated due to the lack of pervasive embedding support. Ultimately we went with a basic css sprite paradigm. While painful to construct efficiently, it allowed us to build compact pages that were effectively functional across all browsers. (If you’re interested take a look at our code and see if you can figure out how it is built… and tell us what you think.) We now have more than one hundred million maps built on this technology.
Another key feature of our visual paradigm is providing snippets overlays throughout the conversation. As you explore the conversation, the responses need to come back very fast. That meant building an efficient retrieval system that seamlessly returns the required information. While we started on Project Voldemort to provide this functionality, we ultimately found that Riak had better latency parameters. Having many users actually tell us that they didn’t realize calls were going back to our server for each overlay was the ultimate proof that we solved this problem effectively.
Lots of results in a Compact Space in a Compact File
People often give one of two responses about the YapMap interface: "each result takes more space than usual" and "there are a magnitude more results on each page". The reality is that both are true. Providing more context does take up more space than a simple text based search engine. However, this also allows us to actually serve up a number of results for each possible major match. If you think about it in terms of unique destinations, each one of our results pages usually includes 10-20 times more destinations than a traditional result page. This means the end user can understand better and then pick the best destination to drill in. All of these links could make YapMap results pages large or slow to return. We hope you actually have the opposite experience. We’ve done our best to make the page small and speedy, spending substantial time optimizing page construction as well as our backend infrastructure.
We love elegant interfaces. Simplicity that belies complexity is one of our core design principles. We think the YapMap results page is an example of this set of goals. The goal is to allow the user to the learn the interface at their own speed. Things like subsection filtering, author filtering and alternative coloring schemes are all available to be discovered where and when the user desires that functionality.