Introduction to Agents in LLMs
Agents are becoming very popular in today’s world. I wanted to write a simple enough introduction to agents.
What is an LLM Agent?
LLMs excel at understanding and responding to prompts. Agents are AI systems that elevate this capability by enabling LLMs to make decisions and take action autonomously. In simple terms, Agents are like a fusion of LLM chains and tools.
So let’s look at an code example to better understand. We will use Langchain for this which is a popular tool for LLM application development —
Tools are interfaces that an agent, chain, or LLM can use to interact with the world. As we know that the LLM is trained on some data to predict next word. Now, to perform certain actions which it is not very capable of, it might start hallucination like some recent topic or a mathematical calculations. Let’s say we have 3 tools — Wikipedia, Retrieval/ RAG from a set of documents and Arxiv for this example. Now there are various tools that are supported by Langchain which can be found here like the ones mentioned earlier.
# arxiv tool
from langchain_community.utilities import ArxivAPIWrapper
from langchain_community.tools import ArxivQueryRun
arxiv_wrapper = ArxivAPIWrapper(
top_k_results = 1
, doc_count_chars_max = 200
)
arxiv = ArxivQueryRun(
arxiv_wrapper = arxiv_wrapper
)
arxiv.name
> 'arxiv'# wikipedia tool
from langchain_community.tools import WikipediaQueryRun
from langchain_community.utilities import WikipediaAPIWrapper
api_wrapper = WikipediaAPIWrapper(
top_k_results = 1
, doc_count_chars_max = 200
)
wiki = WikipediaQueryRun(api_wrapper = api_wrapper)
wiki.name
> 'wikipedia'In simple terms, retreival/RAG applications, as pictured below, has an LLM, a collection of documents, and supporting infrastructure to improve information retrieval and answer construction. The RAG pipeline looks at the database for concepts and data that seem similar to the question being asked, extracts the data from a vector database and reformulates the data into an answer that is tailored to the question asked. This makes RAG a powerful tool for companies looking to harness their existing data repositories for enhanced decision-making and information access.
We talked about the chunking process and a simple RAG application here as well. We have talked about FAISS earlier here which is essentially a Approximate Nearest Neighbour search store. I used Ollama for generating the embeddings.
What is Ollama?
We might not have access to OpenAI APIs or a cloud environment like AWS to run LLMs. What if there was a simple and easy way to run small LLMs locally with zero hassle + not getting charged where we can start to just play with the LLMs and build POCs. Enter Ollama, a platform that makes local development with open-source large language models a breeze. With Ollama, everything you need to run an LLM — model weights and all of the config — is packaged into a single Modelfile. Think Docker for LLMs. And its extremely easy. You just have to go download it from the official website — https://ollama.com/ , install it and in the terminal run the command like — ollama run gemma to download the opensource LLM. And following is an example of how you can use it. Hurray ! You are able to run your own LLM.
# retrieval tool
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings.ollama import OllamaEmbeddings
from langchain.tools.retriever import create_retriever_tool
# load the data from the website
loader = WebBaseLoader("https://docs.smith.langchain.com/")
docs = loader.load()
# Chunking the data into consumable size
documents = RecursiveCharacterTextSplitter(chunk_size = 1000, chunk_overlap = 200).split_documents(docs)
# Saving the data in FAISS data store
vectordb = FAISS.from_documents(documents, OllamaEmbeddings(model = "llama3:8b"))
retreiver = vectordb.as_retriever()
retrieval_tool = create_retriever_tool(
retreiver
, "langsmith_search"
, "Search for information about LangSmith. For any questions about LangSmith, you must use this tool"
)
retrieval_tool.name
> "langsmith_search"Now let’s build everything together — The LLM, the ReAct Prompt, the tools and put everything to a Agent.
ReAct prompts LLMs to generate verbal reasoning traces and actions for a task. This allows the system to perform dynamic reasoning to create, maintain, and adjust plans for acting while also enabling interaction to external environments (e.g., Wikipedia) to incorporate additional information into the reasoning. We will be using Prompt Template from langchain hub.
from langchain_community.llms import Ollama
from langchain import hub
# Loading the LLM
llm = Ollama(model = "llama3:8b")
# ReAct framework Prompt
prompt = hub.pull("hwchase17/react")
# tools
tools = [wiki, arxiv, retrieval_tool]
# Construct the ReAct agent
agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent = agent, tools = tools, verbose = True)Let’s look at this example where I wrote the query and the agent smartly chose the Wikipedia tool first as an action and searched for the query. Then it gave thought that it has everything it needs and needs to summarize. Lastly, provides the final concise answer.
Let’s search something related to the documents saved in the data store.
The final one searching something related to the Arxiv tool.
Hope this introduction did help in understanding the basics of Agents. Now agents are being integrated using another tool called Langgraph which we will talk about some other day.
Please do provide your feedback in form of responses and claps :)
References —
[1] https://youtu.be/2_gSXyt2108?si=spGei7XBVBE9DFIC
[2] https://learn.deeplearning.ai/courses/ai-agents-in-langgraph/
