Knowledge Graph Convolutional Networks: ML Over Reasoned Knowledge
Knowledge Graph Convolutional Networks: ML Over Reasoned Knowledge
Let's take a look at knowledge graph convolutional networks and explore Machine Learning over reasoned knowledge.
Join the DZone community and get the full member experience.Join For Free
Bias comes in a variety of forms, all of them potentially damaging to the efficacy of your ML algorithm. Read how Alegion's Chief Data Scientist discusses the source of most headlines about AI failures here.
Why Haven't Machines Taken Over the World Yet?
It seems quite clear that machines should be able to outperform humans in many more tasks than they currently can, or at least that they should be able to make truly smart predictions. I'm sure we can all relate to a moment when an app made us a recommendation that didn't make any logical sense. You only have to look as far as the recommender system that recommended the product you only just bought, or the spam filter that stole a reply from someone you messaged. In practice, we find that we can't trust machines with decision-making on our behalf.
Intuitively, it feels like our machines are missing something, some capability, such that if we aren't very careful about how we teach them then they'll miss the blindingly obvious. Why is this?
If our machines are missing key elements, then there must be unexplored territory that we can leverage to improve the performance of our models. We can start by examining the abilities that we have ourselves.
There are a few key skills that humans possess that are clearly necessary for making good decisions: learning  and logical reasoning  on top of a pre-existing knowledge base (human memory) .
In mainstream "AI," of these three, we've only really cracked learning.
Skill 1: Learning
The underlying and well-recognized pitfall of doing learning alone is that the learning is shallow; the result is always a complicated black-box mapping from a flat set of symptoms to a conclusion. Learning machines aren't encoding an understanding of the deeper causes of the outcomes they see in their training data. This means that learning on its own is inclined to miss the point.
We ask, "Is there a way to give our machines the other skills that we possess, that they lack?" Armed with those skills, they can make decisions that we can trust.
Skill 2: Being Logical
Deduction or reasoning is second-nature to humans. We use a mental model of rules to predict what will happen in our surroundings. Take this example:
My dinner is in the pan, the pan is on the stove, and the food is simmering. This implies that the pan is hot. Given that the pan is hot, if I touch the pan it'll burn my hand. Therefore the pan is dangerous.
This line of reasoning is obvious to us (except for those odd occasions where you forget and you do burn yourself). In fact, this kind of deduction is so fundamental to our lives that it's hard to spot when we're using it.
Reasoning is central to our existence and to human intelligence. Consequently, it's useful to build tools that can automate that thinking for us at great scale and complexity. That's just the start, and there's even more to be gained from it.
Get More From Less
Deduction takes place over a certain set of facts and concludes with the generation of new facts. This means that given information and the rules that govern a domain, exponentially more information can be derived.
This sounds particularly enticing in the world of Machine Learning, where the size of a dataset is often the performance bottleneck. Reasoning promises to let additional knowledge be derived from the existing. This radically augments the value of the data already available.
The trouble is, as it stands, the vast majority of machine learning models cannot perform any kind of deduction.
Ideally, we want our learning systems to make use of the rules that govern a domain and correlate the outcomes of those rules. In this way, they can gain an understanding of the dynamics of the field.
Skill 3: Storing Knowledge
Reasoning relies upon the context of data. This is necessary in order to automatically know in which particular circumstances a new fact can be inferred. If we created a formula for this, we would say:
Data + Context = Knowledge
What Do We Really Mean by Context?
Context,in this case, means the information governing the structure of data. To use reasoning for a given datapoint, we need to know what type of data it is and how it is inter-related to other datapoints.
The information of the real world isn't made up of a set of column names that we can put into a table. Tabulation is a method we've been using for many years to de-complicate the problem domain. Unfortunately, this process removes the important context: the inter-relations between data-points. It makes sense to throw away structural information if you don't have a technique that can use it. But, if you can use it, then you can build a much more intelligent solution.
We can consider how we store knowledge in our own heads. People don't put their memories in an ordered system of filing cabinets in the back of their heads (desirable as that might seem). Instead, we hold a web of connected experience: a knowledge base. For our intelligent systems that component is called a knowledge graph. This technology enables structured, contextual data storage and reasoning across its knowledge content.
Skills 1+2+3 = Learning Using Reasoned Knowledge
We want to combine all three skills together. The natural progression is to shift from learning over flat, non-contextual data to learning over reasoned, contextual knowledge. There are numerous benefits that we can derive from this:
- Learningbased on facts inferred via reasoning
- Implicitly embed the context of each datapoint into the learned model for better decision-making
- Prediction that may generalize beyond the scope of the training data according to the logical rules of the domain
- A transparent and queryable graph of the underlying domain knowledge
- Reduced quantity of training data required
How Do We Do It?
To learn based on the reasoned knowledge, we need to learn over a knowledge graph. Graph learning is a new research area, where some of the most promising models are Graph Convolutional Networks (GCN). Knowledge graph learning research is in its absolute infancy.
In general, we need an architecture as depicted: a learner asks questions of a knowledge graph via logical reasoning. In this way, any inference of facts made by the reasoner can be performed as and when the learner demands information from the knowledge graph. As a result, the knowledge graph isn't required to store all of the possible facts that the reasoner might infer; they can be generated at query-time.
Published at DZone with permission of James Fletcher , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.