Using agents

This is an agent specifically optimized for doing retrieval when necessary and also holding a conversation.

To start, we will set up the retriever we want to use, and then turn it into a retriever tool. Next, we will use the high level constructor for this type of agent. Finally, we will walk through how to construct a conversational retrieval agent from components.

Setup

Dependencies

We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any ChatModel or LLM, Embeddings, and VectorStore or Retriever.

We’ll use the following packages:

npm install --save langchain @langchain/openai

We need to set our environment variable for OpenAI:

export OPENAI_API_KEY=YOUR_KEY

LangSmith

Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with LangSmith.

Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=YOUR_KEY

The Retriever

To start, we need a retriever to use! The code here is mostly just example code. Feel free to use your own retriever and skip to the section on creating a retriever tool.

import { TextLoader } from "langchain/document_loaders/fs/text";

const loader = new TextLoader("../../../../../examples/state_of_the_union.txt");
const documents = await loader.load();

import { CharacterTextSplitter } from "langchain/text_splitter";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings } from "@langchain/openai";

const textSplitter = new CharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 0,
});
const texts = await textSplitter.splitDocuments(documents);
console.log("texts.length", texts.length);
const embeddings = new OpenAIEmbeddings();
const db = await MemoryVectorStore.fromDocuments(texts, embeddings);

texts.length 41

const retriever = db.asRetriever();

Retriever Tool

Now we need to create a tool for our retriever. The main things we need to pass in are a name for the retriever as well as a description. These will both be used by the language model, so they should be informative.

import { createRetrieverTool } from "langchain/tools/retriever";

const tool = createRetrieverTool(retriever, {
  name: "search_state_of_union",
  description:
    "Searches and returns excerpts from the 2022 State of the Union.",
});
const tools = [tool];

Agent Constructor

Here, we will use the high level createOpenaiToolsAgent API to construct the agent.

Notice that beside the list of tools, the only thing we need to pass in is a language model to use. Under the hood, this agent is using the OpenAI tool-calling capabilities, so we need to use a ChatOpenAI model.

import { pull } from "langchain/hub";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = await pull<ChatPromptTemplate>("hwchase17/openai-tools-agent");
prompt.promptMessages;

[
  SystemMessagePromptTemplate {
    lc_serializable: true,
    lc_kwargs: {
      prompt: PromptTemplate {
        lc_serializable: true,
        lc_kwargs: {
          template: "You are a helpful assistant",
          inputVariables: [],
          templateFormat: "f-string",
          partialVariables: {}
        },
        lc_runnable: true,
        name: undefined,
        lc_namespace: [ "langchain_core", "prompts", "prompt" ],
        inputVariables: [],
        outputParser: undefined,
        partialVariables: {},
        template: "You are a helpful assistant",
        templateFormat: "f-string",
        validateTemplate: true
      }
    },
    lc_runnable: true,
    name: undefined,
    lc_namespace: [ "langchain_core", "prompts", "chat" ],
    prompt: PromptTemplate {
      lc_serializable: true,
      lc_kwargs: {
        template: "You are a helpful assistant",
        inputVariables: [],
        templateFormat: "f-string",
        partialVariables: {}
      },
      lc_runnable: true,
      name: undefined,
      lc_namespace: [ "langchain_core", "prompts", "prompt" ],
      inputVariables: [],
      outputParser: undefined,
      partialVariables: {},
      template: "You are a helpful assistant",
      templateFormat: "f-string",
      validateTemplate: true
    }
  },
  MessagesPlaceholder {
    lc_serializable: true,
    lc_kwargs: { optional: true, variableName: "chat_history" },
    lc_runnable: true,
    name: undefined,
    lc_namespace: [ "langchain_core", "prompts", "chat" ],
    variableName: "chat_history",
    optional: true
  },
  HumanMessagePromptTemplate {
    lc_serializable: true,
    lc_kwargs: {
      prompt: PromptTemplate {
        lc_serializable: true,
        lc_kwargs: {
          template: "{input}",
          inputVariables: [Array],
          templateFormat: "f-string",
          partialVariables: {}
        },
        lc_runnable: true,
        name: undefined,
        lc_namespace: [ "langchain_core", "prompts", "prompt" ],
        inputVariables: [ "input" ],
        outputParser: undefined,
        partialVariables: {},
        template: "{input}",
        templateFormat: "f-string",
        validateTemplate: true
      }
    },
    lc_runnable: true,
    name: undefined,
    lc_namespace: [ "langchain_core", "prompts", "chat" ],
    prompt: PromptTemplate {
      lc_serializable: true,
      lc_kwargs: {
        template: "{input}",
        inputVariables: [ "input" ],
        templateFormat: "f-string",
        partialVariables: {}
      },
      lc_runnable: true,
      name: undefined,
      lc_namespace: [ "langchain_core", "prompts", "prompt" ],
      inputVariables: [ "input" ],
      outputParser: undefined,
      partialVariables: {},
      template: "{input}",
      templateFormat: "f-string",
      validateTemplate: true
    }
  },
  MessagesPlaceholder {
    lc_serializable: true,
    lc_kwargs: { optional: false, variableName: "agent_scratchpad" },
    lc_runnable: true,
    name: undefined,
    lc_namespace: [ "langchain_core", "prompts", "chat" ],
    variableName: "agent_scratchpad",
    optional: false
  }
]

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({ temperature: 0 });

import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";

const agent = await createOpenAIToolsAgent({
  llm,
  tools,
  prompt,
});
const agentExecutor = new AgentExecutor({
  agent,
  tools,
});

We can now try it out!

const result1 = await agentExecutor.invoke({ input: "hi im bob" });

result1.output;

"Hello Bob! How can I assist you today?"

Notice that it now does retrieval

const result2 = await agentExecutor.invoke({
  input: `what did the president say about ketanji brown jackson in the most recent state of the union? The current date is ${new Date().toDateString()}`,
});

result2.output;

"In the most recent State of the Union, the President mentioned Ketanji Brown Jackson as his nominee "... 176 more characters

See a LangSmith trace for the run above here

Notice that the follow up question asks about information previously retrieved, so no need to do another retrieval

const result3 = await agentExecutor.invoke({
  input:
    "how long ago did the president nominate ketanji brown jackson? Use all the tools to find the answer.",
});

result3.output;

"The president nominated Ketanji Brown Jackson 4 days ago."

See a LangSmith trace for the run above here

For more on how to use agents with retrievers and other tools, head to the Agents section.