Hello world,
I am probably not the most qualified to give a course on AI, but I have been following the topic for some years now, and I will also be publishing some tooling capabilities for it, so I thought it was a good idea to make sure everyone is on the same page regarding the AI concepts and keywords being used.
During the last few months, as I coded my way through new capabilities for my module, I was discussing AI topics and found out that many very good people, sometimes even those working with AI, were not aware of the terms I was using, or the concepts I was trying to develop or learn about.
AI is a vast subject and most of the time it is used as a buzzword that is synonymous with Large Language Model (LLM). So, in order to make sure we are discussing the same things in the next posts I am going to cover, I will run you through the different elements below.
If someone finds a better article than this one to explain the key concepts, feel free to add it in the comments.
I am trying to explain it via an oversimplified story of how one concept evolved into another and how they build on each other, so you can better understand their limits and remember the story better than a list of words with definitions.
The Genesis of AI
The tools being used are, contrary to belief, not fundamentally different from what has been used for many years in the past; the main difference is the scale at which such systems are built.
Not so long ago, the term “data science” was used in the field of developing some sort of intelligent capability. The idea was to use algorithms, a set of rules implemented to generate a deterministic outcome out of set parameters, to try to predict values.
By defining different algorithms with different parameters, we were building models. These models have to be trained on some data (where you already know the expected value) to make sure that the algorithms you have implemented are predicting the expected value, or something very close to it.
The way you set different parameters to change the algorithm and create models is sometimes referred to as weighting the model. You put a certain weight on the parameters of the model.
In the past, I was very excited about that field; being a data scientist was the hottest job of the decade. It requires you to understand math, probability, and coding in order to build these algorithms and therefore derive a model. For anyone willing to learn about that and try things out for themselves, there used to be a big hype around Kaggle. On the website, you used to have datasets available for testing, some competitions, and an exchange among peers working on creating algorithms.
During that time, the competitions started to draw attention to a specific set of algorithms that very often won, or at least finished in the top 5% of every competition.
These algorithms were the Neural Network algorithms; the idea behind them is to recreate how the brain works by setting up nodes and edges. The concept is fairly understandable: you feed some sort of data to the neural network, the different data points are scattered across the different edges, and when reaching a node, the data is transformed, and so on, until it reaches the final layer. Each node weighs the data point to transform it. The end result is checked against an expected value, and depending on how close it is, this is where the model (neural network) is rewarded or penalized.
The process of checking whether the network needs to be rewarded or penalized based on the result is called training. There are several ways to do that: these techniques are called Reinforcement Learning (RL), RLHF (Reinforcement Learning from Human Feedback), or, most commonly, backpropagation, where you feed back the error so the model can learn to adjust the weights of the nodes for each mistake.
You can even just loop through that process using data that is labelled for consumption — for example, you tell the model this is the weight, the size, and the location of an animal, and you expect it to tell you what it is. This is supervised learning.
Or you can simply feed it all the data points and let the model find the pattern to get the answer. This is unsupervised learning.
Neural Networks started to show up and were really good at some challenges, especially finding patterns and discrepancies in large volumes of data.
Most of the time of the data scientist, or data engineer, at that time was spent cleaning the data and getting it into the right format. You could not simply pass any data to the system; you needed to prepare it.
Typically these tasks are label encoding and one-hot-encoding (changing text into numbers that can be used by the model — we also have vectors* that represent a tokenized word, its position in the text, and other attributes that can be represented in that vector*), normalization (making sure your data is all on a certain scale), standardization (adjusting your data points to normalize their distribution), feature selection (picking the data points you want to feed in so you don’t introduce noise), and feature engineering (developing new data points based on existing ones).
Shortly after this, people even developed Deep Learning, which was basically adding more and more layers of nodes and edges to make the calculations even more complex. More nodes means more parameters can be processed, with more accuracy.
* vector: a series of numbers where each position means something. An easy example is a spatial vector, where [x,y,z] represents a position in 3D space.
When you train these models, you basically adjust these nodes, or parameters, to output a better result. Once trained, you can use that model, which is what is called inference. Inference is simply using a developed model with new data points (not the ones you trained it with) to output a result.
I should say that there are a lot of different neural networks that exist and are specialized in different tasks — for vision (Convolutional Neural Network (CNN)), or for time series analysis (Recurrent Neural Network (RNN)).
However, the one that gained the most popularity was the Large Language Model (LLM); this one is specialized in semantics.
The reason it became so popular is obviously its utility: through language, the model looks like it can think and articulate concepts, when what it actually does is predict the next word. And by word, I mean token. You’ve heard that term a lot, but what is it really?
You may still remember, from a few paragraphs ago, that you need to prepare your data before feeding it to the model. That is still true: a model does not understand words, it understands numbers, or vector-shaped numbers, that represent words or parts of words. This is what a token represents. The big breakthrough came from a paper by Google that introduced the transformer; this capability allows words to be split into tokens. Before that, when preparing the data — during training, or before inference — you were processing one word at a time.
Now the transformer scans all the text or images that you have and groups the data by the most frequently appearing pairs. So in English, “and” or “ing” are three-character sets that are very common, so they will most likely become two separate tokens.
Now you should have a good idea of what the LLM stack is doing and where it comes from. You get text, then you transform it into something that can feed the model (already in place in your LLM), then the LLM uses its nodes and edges to weight the data and, based on its existing training, it will output the expected result. (By the way, you’ve just used inference.)
To give an overview of the process from raw text to the result output, I asked AI to generate this diagram as a representation of the different steps.
The current AI era
What changed when ChatGPT (GPT: Generative Pre-trained Transformer) came out mostly came down to the computational power the model was using. The first version of ChatGPT used 175 billion parameters, so it could really understand nuance and a very large number of topics.
It was not fundamentally new, but it used one of the oldest tricks in the book: throw a huge amount of power and a huge amount of data at training the model. The “theory” was that more parameters = better results. And it worked pretty well, to a certain extent. (Disclosure: there are ongoing lawsuits and widespread allegations that data used to train the main models included copyrighted work used without permission, hidden behind a technology layer; no company has been held liable for it to date.)
So you get your first version of the chatbot we had at the start. What changed afterwards?
The thing is, you may have a large amount of data, but at the beginning you have a bit too much of it, especially the bad kind. Feeding the model with everything you can find on the internet is a good strategy to start with, but there is a lot of wrong information on the internet, so you end up with data quality issues.
Also, you may have a large amount of compute (power + data storage), but you don’t have an unlimited amount. So your model isn’t able to store everything, and it can also mix things up, because elements that look similar may mean different things. For example, with the word “terrific,” it can mean something really good or really bad — the model just sees them as the same vector in its representation.
So the first round of iteration to make the model better was to use better data, more annotated data. This is where the data-labelling job has grown a lot in recent years.
Also, since storage is always a constraint, engineers tried to essentially model a (large) neural network to encapsulate the whole knowledge of the internet — but you will never have enough space for that. Because the model tries to fit everything into the number of nodes (parameters) it has, as it runs out of space it will try to pair a calculation made for “red” on the same node as “7,” because during their transformation they somehow seem similar enough. This is called superposition in the model.
This is one of the few reasons the model hallucinates: it finds patterns and relationships in unrelated topics.
To accommodate some aspects of this, models are designed to use a certain number of tokens during a session — this is what the context window is about. You’ve probably heard about the million-token windows that newer models can use; this is the limit on the number of tokens the AI can reasonably hold, and beyond that, either the model loses some of them in the depths of its layers, or it starts to superpose them and hallucinate.
Now we had our new LLMs, but they were not yet in their final form. Having a chatbot is nice and cozy, but what would be really useful is if it could take actions — and this is where the harness and MCP were introduced, giving rise to the term “agentic AI.”
As models are able to generate predictive text or images, they could also use predictive invocation of tools. The commands to run a Python or JavaScript script are, after all, just text. So the capability to use tools, and to have a certain way to interact with them, was developed via the harness. It tells the AI model what type of process it should carry out when performing certain actions. These actions can be an API* call, or just a script that the AI model has in memory.
At some point, this is what made the LLM capable of performing tasks and turned it into an agent. Anthropic was the first to propose a scalable way to not only let the LLM use its own tools, but also let you provide your own tools to the LLM — these are called MCPs (Model Context Protocol). An MCP is a tool that tells the AI model to execute certain actions and expect a certain response. It is usually a wrapper around an API* definition that the AI can understand, thanks to some wrapping functionality offered by the MCP library (e.g., the mcp library in Python).
Usually an MCP runs on a server that the AI model can invoke and interact with, essentially performing API calls on your behalf.
*API: Application Programming Interface, which offers endpoints (URLs) that allow you to request, create, or modify a resource.
Now that the AI model has a harness (a protocol, or set of rules, defining its behavior around tools) and a way to add more tools via MCP configuration, it can be used to automate some tasks. However, it quickly became evident that the context window limitation means you can’t really feed it too much. Also, it wasn’t really following your setup — it was mostly following the patterns it already had from its training.
To address this, two main elements were developed and introduced over the last year: markdown files (Skills, Agents, etc.) and knowledge graphs.
By using a SKILLS.md file, for example, you can tell your AI model (Claude, in this case) that you want a certain pattern to be followed — specific instructions to shape the answer into a better fit for your expected results. It will try to follow these directions over the general answers it normally gives; however, this eats into your tokens and context window.
The knowledge graph, or context graph, is basically a sort of database that gives the AI access to information about the setup you currently have on your system. Knowledge graphs let the model ask questions about your specific setup, and the useful part is that they’re structured in a way the AI can query. Think of it as a database, but with a data model that isn’t rows and columns, but rather a graph of nodes and edges — each node represents an entity, and each edge, their relationship. (I have a dedicated article on how I built a knowledge graph for an AEP(Adobe Experience Platform) implementation.) By being queryable, the model can use only the data it needs to answer the question, instead of loading all of your documents into its context window.
Lastly, the main buzzword that’s been around is creating loops, or agentic workflows. These capabilities are basically a cron job, a task executed at a specific time, or a service that listens for certain events in order to run a script that invokes AI to perform a certain set of tasks.
These concepts aren’t new — we’ve been automating a lot of our work already in the past. The advantage AI brings is that you don’t need structured data to perform these tasks; you can simply let AI understand the elements by itself, based on your settings.
In the past, this required standardized processes to be in place, with expected inputs and outputs for each part of the workflow. Now AI can take unstructured data (a JIRA ticket, an event payload that differs from one to the next, etc.), understand its meaning, and trigger a certain type of workflow. What you need to do is build a harness around these tasks so you have expected behavior in place. However, due to AI’s probabilistic behavior, you can’t be sure exactly what’s going to happen or how it’s going to behave. Without particular guardrails, it will try its best to work around any unexpected outputs to complete its task, whereas in the past, the workflow would have simply failed.
Is that good behavior? Does it make things more robust, or more unpredictable? It’s up to you to decide.
Future of AI
You now have a picture of the current state of AI, and hopefully a better understanding of how everything has played out.
What’s the future of AI? Honestly, no one knows. One thing is for sure: current LLMs are fundamentally flawed in terms of comprehension or thinking patterns. They don’t actually comprehend, they don’t think — they are just very, very good at finding patterns and executing tools. They are very good predictive algorithms, finding the next token based on a specific input.
Would there be a better way to do it? Probably. The way harnesses are built or configured can obviously be improved. The knowledge base can be made more specific to certain domains. You can add even more algorithms or models dedicated to specific tasks.
In the past, during the data science era, the best results were often achieved with ensemble algorithms — a set of multiple algorithm models used together, not just one, but a combination. It wasn’t perfect, but the combination of different models was, on average, bringing better results. Will this be developed further in the future? Instead of one very large language model, will we have a set of smaller models, each specialized in certain tasks, that the harness will invoke depending on the need? In my opinion, probably.
I will leave you with three personal opinions on where AI will evolve in the future — feel free to call me out on them, whether they come true or not :).
- General AI model use in companies will not be from closed source models.
We have seen recently how open source models are catching up with closed source models. These models can run locally (in a private server farm) and answer general questions and perform the same tasks in more or less the same way as the famous frontier models (the latest version of AI models).
Instead of paying hundreds of millions of dollars to companies like OpenAI or Anthropic, some companies will just build their own data center to hold that model and run their internal setup. Because the setup runs locally, sensitive data never has to leave the company.
The job of an AI engineer will be to build the correct harness for these models and make sure the setup has enough capacity and power to be used internally, probably building some internal MCP tools to bring the model internal capabilities. - AI model capability will just improve incrementally.
The time when we were seeing large changes from one model to another is now gone. The latest improvements made by frontier models have not revolutionized the way they answer questions.
The main gains from AI training have already been made; the last percentage points of improvement are harder to achieve, and are most likely more linked to a better harness, a better mix of models (ensemble of models), than to just adding more compute and/or power as it used to be in the past.
It is also not sustainable to get better via getting bigger. Most people have seen the improvement of models in the past and were speculating that it would only keep getting better because it was improving exponentially. However, most people don’t account for the fact that the investment in data centers was also increasing exponentially. - Replacing the workforce with AI will not happen.
AI is a fabulous tool, and it can help you achieve more, and sometimes more efficiently. For people who never coded or never learned the technicality of some tools, it feels like magic, and (software) engineering looks like it’ll be dead in 5 years.
For most people who actually work with it, they constantly see the flaws in the code produced, the gaps in the application architecture that weren’t accounted for.
For me, it’s more of a “Google Search 2.0” moment. When Google introduced search as we know it, we were all blown away because knowledge became so easily accessible. Now we’ve removed the step of reading the article and understanding it before building something.
Is it great? I am not so sure. It definitely helps you build your proof of concept, but going into production with actual customers requires so much knowledge (and therefore context) that I do not see that being possible… or you need to pay a lot of money.
The price of tokens has a ceiling: if AI companies make it too high, bigger customers will build their own service instead of relying on a third party (also for legal and security reasons); if you make it too low, then the cost of the inference (calling the model to get an output) will be too high and it will not be sustainable.
It is fundamentally an amazing tool which will change the workplace and how we approach tasks. However, as humans, I believe we should fundamentally think about whether we want to replace all the tasks and let a third party application run them.
I hope this article was helpful to you, and don’t hesitate to leave a comment if I missed anything or made a mistake.