How to Build Deep Agents for Enterprise Search with NVIDIA AI-Q and LangChain

While consumer AI offers powerful capabilities, workplace tools often suffer from disjointed data and limited context. Built with LangChain, the NVIDIA AI-Q blueprint is an open source template that bridges this gap. LangChain recently introduced an enterprise agent platform built with NVIDIA AI to support scalable, production-ready agent development.

This tutorial, available as an NVIDIA launchable, shows developers how to use the AI-Q blueprint to create a deep research agent that tops leaderboards and can be connected to enterprise systems. The blueprint uses the best of open and frontier LLMs, is optimized using the NVIDIA NeMo Agent Toolkit, and monitored with LangSmith. The result: faster time-to-production for agentic search apps that keep business data exactly where it belongs—private and in a secure environment.

The NVIDIA AI-Q blueprint and NeMo Agent Toolkit are both part of the broader NVIDIA Agent Toolkit, a collection of tools, models and runtimes for building, evaluating and optimizing safe, long-running autonomous agents. 

What you’ll build: A deep agent

You will learn:

Set up

How to build long-running data agents with NVIDIA and LangChain

Install and run the blueprint

Clone the repository and configure your API keys. Copy the environment template first.

Open deploy/.env and fill in the required values.

The NVIDIA_API_KEY grants access to NVIDIA-hosted models like Nemotron 3 Nano. The TAVILY_API_KEY enables web search.

Next, build and start the full stack. Starting multiple containers at once means the first build can take a few minutes, based on your internet connection and hardware specs.

This launches three services:

Once all services report healthy, open http://localhost:3000. Figure 1, below, shows the AI-Q Research Assistant chat interface, where you type a research query and watch the agent work in real time.

Customize AI-Q: Workflow, tracing and model configuration

Open configs/config_web_docker.yml. This single file controls the LLMs, tools, agents, and workflow configuration.

The llms section declares named models. Notice the enable_thinking flag—it toggles chain-of-thought reasoning on or off for Nemotron. The following example declares three LLMs with different roles:

nemotron_llm_non_thinking handles fast responses where chain-of-thought adds latency without benefit. nemotron_llm enables thinking mode with a 100K context window for the agents that need multi-step reasoning. gpt-5.2 adds a frontier model for orchestration. 

The blueprint consists of both a shallow and deep research agent. The following configuration shows both:

The shallow research agent runs a bounded tool-calling loop—up to 10 LLM turns and 5 tool calls—then returns a concise answer with citations. Simple questions like “What is CUDA?” resolve in seconds. The deep research agent uses a LangChain deep agent with a ToDo list, file system, and sub-agents to produce long-form, citation-backed reports.

To keep all inference on-premise, change the orchestrator_llm to point at a self-hosted model.

Monitor the traces

To monitor AI‑Q agents, enable LangSmith tracing so each query generates a full execution trace, including LangChain tool calls and model usage. Add your LANGSMITH_API_KEY to your deploy/.env and add the telemetry section to the config file:

Each query generates a trace that captures the full execution path.

Shallow research sample query:

Deep research sample query:

Expand the trace to inspect each node. The tool calls to web search are especially useful for debugging—you can see exactly what query the agent sent and what results came back. Beyond individual traces, use LangSmith to track latency, token usage, and error rates over time, and set alerts for regressions.

Optimize a deep agent

To tune the deep research agent for your domain, start by examining how it assembles its sub-agents. The deep research agent uses the create_deep_agent factory from LangChain’s deepagents library.

The factory wires together the orchestrator LLM, the tools, and two sub-agents.

Context management is central to how deep agents work. The planner agent produces a JSON research plan. The researcher agent receives only this plan — not the orchestrator’s thinking tokens or the planner’s internal reasoning. By passing only a structured payload, we reduce the token bloat and prevent the “lost in the middle” phenomenon, where LLMs forget critical instructions buried deep in massive context windows. This isolation keeps each sub-agent focused. The following example shows a planner output for a query about retrieval-augmented generation (RAG) versus long-context approaches:

This architecture has been tuned to perform well on both Deep Research Bench and Deep Research Bench II.

To customize the agent for your domain, edit the prompt templates in src/aiq_aira/agents/deep_researcher/prompts/. For example, open planner.j2 and instruct the planner to keep outlines to three sections or fewer for more focused reports. You could also add additional debug logging to inspect the intermediate state (like /planner_output.md) to see how your prompt changes affect the context passed between sub-agents.

Add a data source

The blueprint implements every tool as a NeMo Agent Toolkit function. To connect a new enterprise data source, implement a NeMo Agent Toolkit function and reference it in the config.

Step 1: Implement the NeMo Agent Toolkit function

The following example connects to an internal knowledge base API:

NeMo Agent Toolkit validates the config fields at startup, so misconfigurations fail fast. The agent will use the function’s docstring to decide when to call the tool.

Step 2: Reference the tool in the config

Declare the new tool under functions, then add it to each agent’s tools list:

You don’t need to change any agent code. The agent discovers the new tool’s name and description automatically, and the LLM calls it when a query matches. Use this same pattern to hook into your own enterprise systems or leverage MCP (Model Context Protocol) to grant your agents access to existing tools. This ensures your research stack remains private, and deeply integrated with the data that matters most to your organization.

Going further

By extending and building on the NVIDIA AI-Q blueprint, developers are able to bring a best-in-class LangChain deep agent architecture to their enterprise. To go further, review:

The NVIDIA AI-Q Blueprint is being integrated by partners across the ecosystem, including: Aible, Amdocs, Cloudera, Cohesity, Dell, Distyl, H2O.ai, HPE, IBM, JFrog, LangChain,  ServiceNow, and VAST.