Kyle Matthew’s recent thread caught my attention. He said:
“This idea of prompts as a camera in ideaspace is very useful for coming up with prompts. It’s put me in the mind of prompts as positioning a camera. I’ve spent a lot of time drone camering so thinking through a camera moving around in 3d space is one I’m really familiar with…
e.g. I’ve been playing with prompts that dynamically move the “camera” around — e.g. describe a social role every 150 years moving back in time”
Kyle (who then rolled this insight into Flying Drones in Latent Space) was citing Venkatesh Rao’s A Camera, Not an Engine:
“Any magic in the output is a strong function of magic in the input, not the processing. The Webb telescope doesn’t manufacture those amazing pictures. It sees them “out there.” In infrared. All the associated image processing adds is a kind of hallucinatory false coloring to bring them into our visual range.
If it sounds like I’m setting up a “data camera” mental model for AI, it’s because I am. But as you’ll see, that makes what’s going on more interesting, not less. Unliketraditional photography, which killed a whole world of tedious photorealistic painting, but like the Webb telescope, “data photography” reveals worlds within worlds we’ve never even suspected existed, let alone been able to see, in piles of data too large for us to ever hold in our heads.”
This perspectives reminded me of a construct in Greg Egan’s Diaspora. As Google’s Bard describes it:
“This construct, referred to as a “knowledge space,” is a vast and dynamic realm where information is organized and interconnected in a manner that mirrors the physical world.
Imagine a boundless expanse of space, where each point represents a discrete piece of knowledge. These points are not randomly scattered but instead form intricate patterns and relationships, resembling the intricate structures of physical matter. Just as atoms are bound together by chemical bonds, so too are pieces of knowledge linked by the connections between them.
These connections are not rigid or static but rather fluid and dynamic, reflecting the ever-evolving nature of knowledge. As new information is acquired, the knowledge space adapts and grows, with new points being added and existing connections being modified or strengthened.
This three-dimensional representation of knowledge offers a unique perspective on the vastness and interconnectedness of human understanding. It allows us to visualize how different concepts relate to each other, how new knowledge emerges from existing ideas, and how seemingly disparate fields can be brought together under a unified framework.
The knowledge space in Diaspora is not merely a theoretical concept; it plays a pivotal role in the novel’s narrative. As a digital mind, Yatima, the protagonist, perceives and interacts with the world through this knowledge space. It serves as Yatima’s mental map, guiding its thoughts, informing its decisions, and shaping its understanding of the universe.”
Epistemologically orientated constructs like this are, like the metaverse itself, not a new idea. But, as Venkat describes, the ongoing Cambrian-style explosion in this space is different enough to be of real interest.
This conceptual cascade occurred at the same time as I began to toy with Metamind’s Roam Research plugin. Metamind is a Bengaluru-based designer-engineer duo pioneering the medium of content graphs and developing protocols for multiplayer, graph-based socio-economic activities. Their Roam plugin is a first step on that journey and it’s designed to allow one to comprehend Roam graph changes over time.
After installing it, I wanted to stress test the plugin’s mechanics and UX. Essentially, I needed a way to simulate sustained, long term use. The answer? Set up a dummy Roam graph, install an autotag plugin—something that automates the enforcement of semantics—and use an LLM to generate graph inputs. So, I began prompting ChatGPT and asking questions about the Second World War, passing the responses in, and triggering updates via the Metamind plugin periodically. The latter in turn gave me insight into new, modified and renamed pages that occur as a result of the autotagging.
Somewhere along the way I read both Kyle and Venkat’s thoughts and realised that I was, in effect, speedrunning the sensemaking that occurs when one is exposed to new facets of a reality. That capability could turnout to be very consequential.
If a sufficiently annotated reality is included in an LLM’s training data, then one can create simple prompt protocols, feed the responses to a knowledge graph, comprehend the changes that occur to the graph’s topology over certain increments, and vicariously experience an opinionated sense of a particular reality over a particular period of time. Instead of the one hundred and fifty years that Kyle cites above, it becomes possible to make smaller circles and replay history at increments of one hundred and fifty days. Or hours. Or minutes.
Take the Second World War. B.H. Liddell Hart authored perhaps the best strategic account of its events. In it, he describes how close the Blitz was to overcoming British resistance and morale. In it, he describes how near Nazi Germany was to obliterating Allied forces at Dunkirk, post-Blitzkrieg. “If Hitler only knew…” But he didn’t. And that is how it goes.
History hinges on what its characters don’t see as much as what they do. This won’t change. Yet, armed with an appropriately trained LLM, a knowledge graph, and a medium for comprehending knowledge graph changes over time, our ability to understand the extent of a historical character’s perspective and the rationale of their choices is taken to a new level. Movies, books and other traditional cultural mediums may allow us to replay history. This new combinatorial evolution of technology, though, allows history to be roleplayed.
I could combine a timeline—like this one which charts the history of the physics of dynamics—with Kuhnian paradigms and use the historical roleplaying combo described above to step through time and experience the terraforming of the scientific terrain. I could use this collection to see where the paradigm shifts emerged, how they spread, and where their flow was most and least viscous, and why. Taking a leap from the study of genetics, I could silence a particular node and chart the impact of the severed connections. I could become a non-player character, placing a shroud of awareness over my insight, perhaps by limiting the in- and out-degrees of my connectivity to the graph embedding I was roleplaying, and simulating an impoverished understanding.
What does reality look like from the perspective of such varied roles? Soon—perhaps sooner than we think—we may have a way of finding out.