Dynamic text

My final project proposal, submitted to my Human 2.0 class, taught by the Media Lab’s Hugh Herr (see his TED talk here).

A Dynamic Representation of Text

This project is founded on the notion that the conscious and rational part of our brain (the voice inside our head) is, while powerful, severely limited in bandwidth and fatigue rate. When we read, there is an ongoing process of converting the symbols of text into a thought, which must then must be synthesized into our mental model of the world. The conversion processes at work consume a lot of mental energy. Also glucose.

The current representation of text – static and abstract – is a poor medium of transferring knowledge. There are two layers of abstraction between text on a page and the thing it refers to. The earliest written languages involved a one-to-one mapping between concepts and symbols. Chinese is a good example. Each animal, object, emotion, landform, food, place had it’s own representation. Later, alphabetic systems developed. These involve an alphabet that can combine to form symbolic representations (words) which themselves represent the concept being referred to. This doubly-nested abstraction is why we take “language” classes until we’re 18 year old, and often well beyond.

Thus, it is difficult for many people to focus and read a long document in one sitting, because they quickly lose focus, read
the same sentence over and over, etc. Learning tends to be a constant struggle against more primitive brain structures, which insist vehemently that the markings on a page or screen are a) not going to kill you and b) not food, so why are you still sitting there!? (Go pick berries or reproduce!) Fortunately it’s not easy to fool those parts of the brain. No one gathers up their mental energy to sit down and watch Netflix, because video is a dynamic medium that tells a story and engages the primitive structures of our brain, while still (potentially) being intellectually stimulating for our hight functioning brain structures. Dynamic media bring the two parts of the brain into synchronization.

Spritz is an attempt to address the weaknesses of static text. They took the 1-dimensional string of text and broke it up into 0-dimensional points (words) that were shown one at a time. This is not a bad idea, and the verdict is still out on whether it improves reading speed without adversely affecting retention.

This project takes a dramatically different approach. We will be splaying 1D text into a 2D plane, and augmenting it with font modifications, color, and potentially animation. This gives the effect of watching the construction of a body of text into a coherent Gestalt, and also allows for persistence of the content. That is, a reader can pause the program and re-read an earlier part, which can be easily found by leveraging the brain’s immense spatial reasoning and memory faculties.

The text can be linguistically parsed using Python’s Natural Language Toolkit. It can then be sequentially displayed on a screen in a manner similar to RSA Animate, but without the additional sketches. The parse tree of a given sentence can guide decisions on how and where to display the text in the 2D display area. For instance, if a sentence contains a list, there can be arrows pointing to
each of the three elements, with all additional modifiers to those elements dangling from those branches. As the reader progresses through the reading, it splays itself into a 2D structure that can then be explored at the reader’s leisure. Playback can be paused, sped up, and slowed
down. Eventually, content-awareness can be incorporated, so words (especially adjectives and verbs) can demonstrate the qualities or actions they themselves describe. The logical endpoint of this project is the ability to create functional and interactive animations of any scene or
concept described in text, but that’s way down the road.