Table of Contents
Every System Has a Tech Stack
In the world of software development, every system or application has a tech stack—a set of tools and technologies that work together to build a system or application.
For Example, a web application consists of a stack that includes the front-end and back-end framework, database, and hosting.
Agentic AI Tech Stack
Similarly, there is a tech stack for developing and maintaining Agentic AI systems. Let’s break down the stack and explore some popular industry options for each.
1. AI Model
If the brain is the CPU of humans, then AI models are the CPU of the AI agent that help them to reason.
Currently, LLMs are being used as the brain of these Agentic AI systems. In the future, perhaps more powerful AI models will emerge to replace LLMs.
Popular Options
- Open Source
- Mistral (Widely used)
- Llama
- BLOOM
- Falcon
- Vicuna-13B
2. AI Agent Framework
Let’s say you build an AI system that makes human lives much easier and many people are using it. For that, you have to do everything by yourself, such as memory management, tool binding, AI model integration, etc. Now, if you decide to upgrade to the latest AI model, you would have to alter the entire system because every AI model has a different way of being used.
But if you use a framework, you don’t have to do anything; everything will be handled by the framework, e.g., model implementation, memory management, tools, testing, and monitoring.
Popular Options
3. Memory Management
If I ask you what you ate this morning, you might know what you actually ate. How do you know that? It’s because of your memory, where past experiences are stored. Our past experiences always influence the present workings of our brain.
For example, if a dog bit you in the past and a few months later you pass by a dog, you will feel fear because of the past experience stored in your memory. But if you didn’t have that experience in the past, you wouldn’t feel that fear.
Similarly, AI agents have memory that helps them store past experiences and take those experiences into account in their present workings.
For example, in the past, your agent may have learned that a user is male or female. In the future, it will consider that when interacting, such as recommending skincare routines based on gender.
But the problem is that currently, the brain of agents, LLMs, does not have an infinite context window to remember everything about the past. So, we need a way to manage memory by considering the LLM’s context window and how the agent uses that memory in the present.
Some agentic frameworks have native support for managing memory, but there are also additional separate tools that you can use with these frameworks or independently
Popular Options
4. Vector Databases
AI agents often need data to use in their operations. The storage and retrieval of such data is often done with the help of vector databases.
The types of data they may need include past chats, FAQs, documentation, and proprietary datasets, among others
Popular options
- Open Source
- Qdrant (Widely Used)
- Milvus (Widely Used)
- Weaviate (Widely Used)
- Pgvector (Widely Used)
- Elasticsearch
- Chroma (good for learning projects)
- Faiss
- Closed Source
- Pinecone (widely used)
- Azure Cosmos DB
- MongoDB Atlas
5. Tools for Agents
Tools are actions that an agent can take.
For example, web search, running code, ticket booking, using a calculator, fetching data from databases, etc.
Tool is just a function that does something and returns results to agent back. You can create any tool for your agent according to the nature of your agent and bind it with your agent. You can also use existing apis for that tool instead of developing your own.
Popular Options
- Web Search
6. Sandboxes
Sandboxes are isolated environments where you can execute the outputs of agents safely. For example, executing code, etc.
Popular options
- Executing Code
8. Monitoring and Observing
Monitoring is critical for tracking the performance of AI agents in real-time to ensure they are working as expected.