What do we mean when we say something is a kind of thing? I’ve been wrestling with that question a great deal of late, partly because I think the role of the ontologist transcends the application of knowledge graphs, especially as I’ve watched LLMs and Llamas become a bigger part of the discussion.
Graphs are truly wondrous thing because there are very few static data structures that can’t be represented as a graph of some sort. Additionally, with a few minor modifications, there are very few dynamic data structures that can’t be represented in the same way. Whether you’re talking RDF graphs or labelled property graphs, you can represent hash tables (dictionaries), arrays, simple and complex data types, trees, relational database tables, and so forth as a graph. As such, graphs are the molecules of the data world.
However, things get slightly more complex when you start talking about LLMs. Certainly, anyone who’s spent time in natural language processing is familiar with the intimate relationship that graphs have with entity enrichment or extraction, but as the goalposts in the computer world have shifted more and more recently of late, it’s another form of a graph - neural networks, more generally recurrent neural networks, or RNNs, that have taken centre stage.
Such neural networks make use of graphs of interconnections between “electronic neurons” that can decompose complex structures from text to images and sounds to code fragments, turn these into features, and then determine the weight of these features by using training data to indicate patterns to be found in these features (using a principle called activation, which is worth exploring, but not now.
The upshot of this is that neural networks essentially create a giant, highly compressed hologram of information that describes conceptual relationships based on where they tend to cluster as patterns in a ridiculously high number of dimensions. You can describe concepts as things or even get to the point of describing concepts as classes of things simply because human language tends to make similar use of classes and concepts. Classification emerges spontaneously by looking at a large enough corpus, but what those classifications are tends to be left to the mercy of the training corpora. If your corpora never mentions horses, for instance, asking the model to identify and classify horses becomes largely an exercise in attempting inferencing without a lot of prompting that indirectly feeds that concept into the chat environment.
Moreover, that knowledge is not retained in the model itself but only ancillarily in the “attention” or context that this model uses when dealing with multiple sessions of users. You can sharpen the knowledge of the model in a certain direction through the use of specialized Loras, but these loras do not alter the fundamental structure of the LLM but simply augment it with temporary specialized knowledge.
What’s most important here is that a given concept doesn’t have a fixed identifier but rather has what would be considered a sheaf of complex eigenvectors that describe regions in that very big space. Categorization becomes stochastic or probabilistic at that point.
The closest analogy of what that looks like is to consider JPEG files. These files use compression algorithms to turn a given picture into a series of wave functions, and it is the aggregate of those wave functions that superimposed on one another, which then reproduce the image. It’s not a perfect representation, and you cannot point to a bit in a JPEG representation and say that it corresponds to a given pixel. Does that mean that JPEG images are useless? No, of course not. It simply means that any JPEG image will always be just an approximation of the source image, though it may be a very close approximation.
This holds for LLMs as well. Boundaries are fuzzy and, in many cases, are likely not even contiguous, primarily because linguistic patterns are also fuzzy and not necessarily contiguous.
This distinction between the clearcut boundaries inherent in most knowledge graph representations and the ability to absolutely assign a particular meaning (via annotations of graph structures with other graph structures) simply has no counterpart in the LLM world. Indeed, inferencing on a graph is comparatively trivial concerning the identification and manipulation of subgraphs. Inferencing on an LLM requires building another layer of abstraction that is very expensive to maintain, both from a power and a compute cost perspective.
Does this mean, then, that ontology is dead with the emergence of AIs? No.
Rethinking Ontology
An ontology, when you get right down to it, can be thought of as the components of a language: its core concepts and relationships, grammar, rules and restrictions, and even its mappings to other languages. You can think of an ontology as a language toolkit. Some ontologies are tiny, with perhaps a dozen classes and relationships, usually handling very specialized tasks (SKOS, RDFS, and SHACL all come to mind). Others are vast, sprawling constellations intended to describe thousands of classes and hundreds of thousands of relationships, with perhaps trillions of expressions of differing types.
However, this view of language is a very dry and logical perspective, as if diagrammed in a Language Arts class (or, more accurately, a book on symbolic logic). It is no surprise that it is also a form that is especially easy for machines to consume. Human language is far messier, verbose, and unnaturally qualified, with many unstated referentialisms (this, that, these, etc.).
LLMs generally work with fairly arbitrary JSON structures, essentially doing some ontological mapping between descriptive processes (narratives) and structural descriptions. For instance, I laid out a character in a GPT-based game I’m developing, which the Assistant then recast as the following:
Elspeth d'Arcy, as a character in the New Albion game world, is a dynamic and complex figure with a blend of intellectual prowess, magical skill, and an authoritative presence. Here's a detailed description:
Appearance: Elspeth is a striking woman with an above-average height and a petite build, giving her a poised and elegant stature. She has short brunette hair styled in a bob, which frames her face neatly and adds to her sharp, professional look.
Attire: Her official attire is a unique blend of functionality and style, befitting her role as an inspector. It includes a woman's version of a watch officer's uniform: a soft beret, a crisp white blouse, and breeches. She wears a grayish-blue great coat over a brown bodice, complemented by a feminine blue cravat. This ensemble not only signifies her position but also her readiness to engage in investigative work. Her formal wear is conservative but stylish, favoring gowns in yellows, greens, and fall colors, while her casual attire is a slightly relaxed variation of her official uniform.
Role and Skills: Elspeth serves as a chief investigator for the Duke of Emanibeth. She specializes in solving complex and challenging crimes, often utilizing her skills in forensic magic. Her role demands a blend of analytical thinking, magical ability, and a keen eye for detail.
Personality: She is characterized as an INTP (Introverted, Intuitive, Thinking, Perceiving) personality type, which reflects in her open, engaging, and curious nature. Elspeth is observant and quiet, yet well-mannered, with an exceptional ability to read both people and scenes. Her integrity is high, guiding her actions and decisions.
Influence and Relationships: Elspeth is well-liked and respected in her circle, known for her dedication to justice. However, her role as an inspector has inevitably made her some enemies, particularly in the criminal underworld.
Magical Prowess: As a mage, Elspeth brings an extra dimension to her investigative work. Her focus on forensic magic suggests a deep understanding of using magical means to uncover clues and solve mysteries, making her an invaluable asset in her field.
Elspeth's character combines elements of fantasy and Victorian-era detective work, creating a rich and engaging persona that is both unique and deeply rooted in her world's lore.
From this, I then asked ChatGPT to create a JSON file for storing relevant information (this has gone through a few iterations). The resulting file came out looking something like the following:
{
"@type": "Character",
"name": "Elspeth d'Arcy",
"title": "Lady Inspector",
"vocation": "Inspector",
"characterClass": "Mage",
"socialClass": "Aristocracy",
"age": "Early 30s",
"level":6,
"imagePrompt": "Unless otherwise indicated, Elspeth wears her official attire.",
"alignment": "Neutral Good",
"gender":"Female",
"species":"Human",
"appearance": {
"height": "5' 6\"",
"build": "Petite",
"facial_hair": "",
"hair": "Short brunette hair styled in a bob",
"accessory": ""
},
"attire": {
"official": "A woman's version of the watch's officer uniform, which includes a soft beret, white blouse, breeches, a grayish-blue great coat, a brown bodice, and a feminine blue cravat. ",
"formal":"Conservative but stylish gowns in yellows, greens and fall colors.",
"casual": "a soft beret, white blouse, brown or dark green skirt, a grayish-blue great coat, a brown bodice, and a blue chocker with a pentacle pendant."
},
"description": {
"role": "She acts as a chief investigator for the Duke of Emanibeth, where she is called in to solve the more intractable crimes. She has short brunette hair kept in a bob, and wears She is a skilled magician, focusing primarily on forensic magic.",
"residence": "D'Arcy Hall, Mayfair, Elondim",
"background": "A pretty woman of above average height, and poised structure. She acts as a chief investigator for the Duke of Emanibeth, where she is called in to solve the more intractable crimes. She has short brunette hair kept in a bob, and wears a woman's version of the watch's officer uniform, which includes a soft beret, white blouse, breeches, a grayish-blue great coat, a brown bodice, and a feminine blue cravat. She is a skilled magician, focusing primarily on forensic magic.",
"relationship_with_Elspeth": "self",
"personalityType":"INTP",
"personality":"Open,engaging,curious, observant, quiet, well-mannered, but good at reading people as well as a scene. She has a high sense of integrity",
"influence": "Elspeth is generally well liked and respected, and works hard to achieve justice. As an inspector, however, she has made enemies, especially in the underworld."
},
"attributes": {
"strength": 11,
"intelligence": 18,
"wisdom": 16,
"constitution": 12,
"dexterity": 14,
"charisma": 16
}
}
The structure here illustrates a particular ontology. I don’t have a schema for that ontology, and I doubt one exists. Instead, ChatGPT has synthesized this from examples both in the chats involved and from external content that made up its corpus.
Having built up such a document, it is possible to reverse engineer a basic schema in either JSON schema or in other schema languages (SHACL, XSD, others). As with Generative AI in general, such a synthesis is usually an iterative process that involves identifying data that needs to be captured and variations of that data from instance to instance.
ChatGPT is surprisingly schema-aware concerning RDF, and schema.org, in particular, seems to be its go-to schema when working with RDF. With the following prompt,
Given the {previously referenced} JSON document, create the closest schema.org based Turtle document to it as you can.
the AI assistant produced the following Turtle output:
@prefix schema: <http://schema.org/> .
<http://example.org/ElspethDArcy>
a schema:Person ;
schema:name "Elspeth d'Arcy" ;
schema:jobTitle "Lady Inspector" ;
schema:hasOccupation <http://example.org/Occupation/Inspector> ;
schema:memberOf <http://example.org/SocialClass/Aristocracy> ;
schema:age "Early 30s" ;
schema:gender "Female" ;
schema:nationality "Human" ;
schema:height "5' 6\"" ;
schema:weight "Petite" ;
schema:hairColor "Brunette" ;
schema:address <http://example.org/Residence/DArcyHallMayfairElondim> ;
schema:affiliation <http://example.org/Affiliation/DukeOfEmanibeth> ;
schema:knows <http://example.org/Person/ElspethDArcy> .
<http://example.org/Residence/DArcyHallMayfairElondim>
a schema:Place ;
schema:name "D'Arcy Hall, Mayfair, Elondim" .
<http://example.org/Occupation/Inspector>
a schema:Role ;
schema:name "Inspector" ;
schema:roleName "Mage" .
<http://example.org/SocialClass/Aristocracy>
a schema:Organization ;
schema:name "Aristocracy" .
<http://example.org/Person/ElspethDArcy>
a schema:Person ;
schema:name "Elspeth d'Arcy" ;
schema:sameAs <http://example.org/ElspethDArcy> .
<http://example.org/Affiliation/DukeOfEmanibeth>
a schema:Organization ;
schema:name "Duke of Emanibeth" .
# Attributes are represented as blank nodes as they don't have individual URIs
[ a schema:QuantitativeValue ;
schema:name "Strength" ;
schema:value "11" ] .
[ a schema:QuantitativeValue ;
schema:name "Intelligence" ;
schema:value "18" ] .
[ a schema:QuantitativeValue ;
schema:name "Wisdom" ;
schema:value "16" ] .
[ a schema:QuantitativeValue ;
schema:name "Constitution" ;
schema:value "12" ] .
[ a schema:QuantitativeValue ;
schema:name "Dexterity" ;
schema:value "14" ] .
[ a schema:QuantitativeValue ;
schema:name "Charisma" ;
schema:value "16" ] .
The assistant didn’t capture everything (some concepts simply don’t have equivalencies in schema.org), but it captured a surprising amount. However, a bit of inspection also reveals that not all relationships are captured, most notably, the connection is lost between <http://example.org/Person/ElspethDArcy>
and the attributes. This can be added with relative ease by rewriting as:
<http://example.org/Person/ElspethDArcy>
<http://example.org/Property/attributes> (
[ a schema:QuantitativeValue ;
schema:name "Strength" ;
schema:value "11" ]
[ a schema:QuantitativeValue ;
schema:name "Intelligence" ;
schema:value "18" ]
[ a schema:QuantitativeValue ;
schema:name "Wisdom" ;
schema:value "16" ]
...
).
The key point remains, however, that Generative can usually get you 90% of the way there, but expect that there will be some work with the remaining 10%.
On the other hand, if you have an established JSON structure but don’t yet have an RDF schema, you can also use ChatGPT to get you much closer to fidelity with that original document. Given the prompt:
For the given JSON document, create an arbitrary Turtle file that best represents the same information, generating namespaces and associated prefixes as required.
ChatGPT cogitates for a bit, then delivers the following:
@prefix char: <http://example.org/character/> .
@prefix attr: <http://example.org/attributes/> .
@prefix app: <http://example.org/appearance/> .
@prefix attire: <http://example.org/attire/> .
@prefix desc: <http://example.org/description/> .
char:ElspethDArcy
a char:Character ;
char:name "Elspeth d'Arcy" ;
char:title "Lady Inspector" ;
char:vocation "Inspector" ;
char:characterClass "Mage" ;
char:socialClass "Aristocracy" ;
char:age "Early 30s" ;
char:level 6 ;
char:imagePrompt "Unless otherwise indicated, Elspeth wears her official attire." ;
char:alignment "Neutral Good" ;
char:gender "Female" ;
char:species "Human" ;
char:appearance app:ElspethAppearance ;
char:attire attire:ElspethAttire ;
char:description desc:ElspethDescription ;
char:attributes attr:ElspethAttributes .
app:ElspethAppearance
a app:Appearance ;
app:height "5' 6\"" ;
app:build "Petite" ;
app:facialHair "" ;
app:hair "Short brunette hair styled in a bob" ;
app:accessory "" .
attire:ElspethAttire
a attire:Attire ;
attire:official "A woman's version of the watch's officer uniform, which includes a soft beret, white blouse, breeches, a grayish-blue great coat, a brown bodice, and a feminine blue cravat." ;
attire:formal "Conservative but stylish gowns in yellows, greens and fall colors." ;
attire:casual "a soft beret, white blouse, brown or dark green skirt, a grayish-blue great coat, a brown bodice, and a blue chocker with a pentacle pendant." .
desc:ElspethDescription
a desc:Description ;
desc:role "She acts as a chief investigator for the Duke of Emanibeth, where she is called in to solve the more intractable crimes. She has short brunette hair kept in a bob, and wears She is a skilled magician, focusing primarily on forensic magic." ;
desc:residence "D'Arcy Hall, Mayfair, Elondim" ;
desc:background "A pretty woman of above average height, and poised structure. She acts as a chief investigator for the Duke of Emanibeth, where she is called in to solve the more intractable crimes. She has short brunette hair kept in a bob, and wears a woman's version of the watch's officer uniform, which includes a soft beret, white blouse, breeches, a grayish-blue great coat, a brown bodice, and a feminine blue cravat. She is a skilled magician, focusing primarily on forensic magic." ;
desc:relationshipWithSelf "self" ;
desc:personalityType "INTP" ;
desc:personality "Open,engaging,curious, observant, quiet, well-mannered, but good at reading people as well as a scene. She has a high sense of integrity" ;
desc:influence "Elspeth is generally well liked and respected, and works hard to achieve justice. As an inspector, however, she has made enemies, especially in the underworld." .
attr:ElspethAttributes
a attr:Attributes ;
attr:strength 11 ;
attr:intelligence 18 ;
attr:wisdom 16 ;
attr:constitution 12 ;
attr:dexterity 14 ;
attr:charisma 16 .
This suggests that ChatGPT, Copilot, or Bard is more useful in the ontology design phase (at least for now), especially if you tend to be a bottom up designer if you like working with exemplars.
It also suggests, intriguingly, that the assistant’s view of namespaces is surprisingly much like my own - that namespaces should reflect class organizational structures such as desc:residence
, and attr:wisdom
. In other words, the natural language organization that ChatGPT uses, when not otherwise constrained, seems to align pretty closely with both JSON and XML.
I believe there is a lesson in this. We have a tendency, as designers of ontologies, to want to impose a grand vision or order on a language, forgetting, in the process, that language has a certain deep logic to it of its own. A class is, when you get right down to it, simply a template for that organization.
Moreover, design work is where computer assistants shine, not in performing the act, but in suggesting that organization that seems to flow most naturally from one expression (JSON, for instance) to another (Turtle, as above). This suggests that one of the key roles of the ontologists is to be better in tune with language itself, and then, like a gardener, to better shape it subtly, rather than being like a prideful architect striving to build a monument to the ego who then discovers that their work, while impressive, proves brittle in day to day life.
The Role of Ontologists
This brings me to the crux of this article. I have come to believe that organizations, whether an organization of people, an organization of cells, or an organization of words, all share certain characteristics, the most fundamental of which is language. A knowledge graph is one reflection of that language, a map to ensure that concepts that are the same across that organization (and how fuzzy an idea it is) that share similar attributes can be expressed via similar behaviours. This canonical ontology can be informed by external stimuli (outside languages), but it also has a cohesiveness that needs stewardship to keep from becoming unruly.
It is imperative that the best thing that an ontologist can do is not so much to create such a master ontology but rather to discover it. This is an important distinction. While there are often great similarities between organizations, the differences between them are typically as (or more important), and as such a one-size-fits-all approach to ontology design will all too often fail to capture significant nuances and may even prove too constraining in the end, making the ontology useless as the organization evolves.
At the same time, there are specialized ontologies that can be used in an annotation fashion (via different strategies) on assertions on objects themselves, either via reification or similar approaches. Publishing ontologies such as prov-o or the like can be applied as an annotation layer for managing quality control, provenance, and publication state. Such operational ontologies may be derived via inheritance when describing class-wide structures or appended as reified annotations to objects or expressions (assertions).
Additionally, it is worth noting that simple types (datatypes or literal types in RDF parlance) can be used to manage dimensional analysis. Similarly, because any given resource is, in essence, a referential pointer, expression subjects and objects can incorporate lists and references to data structures (arrays, complex objects, token rings, and so forth), shapes as templates can be invoked without the overhead of formally declaring classes and/or properties.
As such, from a purely pragmatic standpoint, one role of the ontologist is to unearth underlying patterns in models (through inheritance, aggregation, or metaphor) that can keep ontologies from exploding with object and relationship types. Another role is to identify transformations on canonical forms to generate instances working with common external schemas (such as schema.org, NIEM, FIBO, or similar systems).
These issues - cross-ontology modelling and ontological transformations - can be done by AI assistance, but the computing costs can be very expensive. However, the process of developing the generalized transformations themselves (think XSLT here or Turtle update) is very much one that can be augmented via assistants, often by baking transformations into the schema metadata as configurational information (that can also be stored within the graph).
Indeed, one of the principal goals of any ontologist is to move as much of the actual mechanics of processing into the graph itself, to make the processing declarative and tokenized rather than imperative. This is where an ontologist shows their mettle. This is also where languages such as Turtle, SPARQL, and SHACL can prove most valuable by facilitating declarative modes of architecture (which can be generalized first then specialized upon refinement) rather than imperative modes (which tend to be specialized first and only building out to more general cases when resources can be allocated to do so). Imperative programming is fragile when dealing with trillions of expressions; declarative is not and indeed fosters both parallelism and distributed computing.
One can argue that ensuring governance is another role, but I’m not sure that’s completely the case. Governance, as is so often practised, is seen as an imperative, human-in-the-middle activity. However, it can be argued that while some of the role of governance comes down to data hygiene, most governance involves identifying patterns that indicate fraud or similar deliberate uses of disinformation and then implementing filters to catch resources when they trip these patterns. This is an ontological function.
Conclusion
I can’t tell you, in five years or even one year, what will be the dominant mechanism for building, querying, and validating graphs. From an ontological perspective, it doesn’t matter. The ontologist is the person who is responsible for making the graphs usable by both specialists and lay audiences, for protecting the linguistic integrity of the graphs, and for transforming graphs into forms that others can best utilize. That much will not change, even in the era of the LLM.
Kurt Cagle is the editor of The Ontologist.