AI Agents as Model Interpreters: How Healthcare Institutions Bridge the Gap Between Complex Analytics and Clinical Decision-Making

Healthcare institutions generate terabytes of operational data daily. Emergency department visits, bed occupancy, infection rates, and staffing patterns all feed into predictive models built by data scientists. The problem is not the models. The problem is the interface between a complex epidemiological simulation and a clinician who needs an answer in 30 seconds.

EpidemIA, a multi-agent system from researchers at IRCCS Sant’Orsola in Bologna, treats agents as translation layers. Instead of asking clinicians to learn R or Python, the system accepts natural language queries and orchestrates calls to a suite of epidemiological models. The agents handle parameter extraction, model selection, execution, and result formatting. The clinician gets a chart or a prediction without touching code.

This is not a chatbot wrapper. It is an orchestration problem with state management, tool boundaries, and observability requirements that matter when the output influences patient care.

The Interface Problem

Healthcare data pipelines produce batch outputs: risk scores, forecasts, and statistical summaries. Clinical workflows operate in real time. A bed manager needs to know if the emergency department will overflow in six hours. A clinician needs to know if a patient’s symptom cluster matches an emerging infection pattern.

The gap is not technical literacy. It is context switching. A domain expert should not need to remember which Python library handles time-series decomposition or how to tune a compartmental model’s parameters. The agent layer absorbs that friction.

EpidemIA uses two subagents:

Planner Agent: Parses the natural language query, identifies the required model or analysis, and extracts parameters.
Executor Agent: Calls the appropriate tool from a modular suite, handles errors, and formats the output for human consumption.

Both agents run on a local open-weights LLM. No data leaves the institution. This is a hard requirement in healthcare, where sending patient data to a third-party API is a compliance failure.

Architecture and Orchestration Flow

The system runs on a single-GPU workstation. The LLM backbone is local. The model suite includes:

Statistical analysis tools (time-series decomposition, correlation matrices)
Epidemiological simulators (SIR, SEIR, and custom compartmental models)
Visualization generators (matplotlib, seaborn wrappers)

When a user submits a query like “Predict emergency department visits for the next week based on the last three months,” the flow looks like this:

Query Parsing: The Planner Agent extracts intent (prediction), target variable (ED visits), time window (next week), and historical context (last three months).
Model Selection: The Planner identifies the appropriate forecasting model. If the query is ambiguous, it asks clarifying questions.
Parameter Extraction: The Planner pulls parameters from the query or defaults to sensible values.
Execution: The Executor Agent calls the model, passing parameters and data references.
Error Handling: If the model fails (missing data, invalid parameters), the Executor reports the error and suggests fixes.
Formatting: The Executor converts raw output (arrays, DataFrames) into a chart or summary table.
Response: The user sees a visualization and a natural language explanation.

The agents do not invent data. They do not hallucinate parameters. They call deterministic tools and return the results. The LLM handles language understanding and response generation. The tools handle computation.

State Management and Tool Boundaries

Each query is stateless by default. The agents do not maintain conversation history unless explicitly designed to do so. This simplifies debugging and reduces the risk of context drift.

Tool boundaries are strict. The Executor Agent cannot modify the underlying models. It can only call them with specified parameters. If a model requires a new feature, a human expert adds it to the suite. The agent does not write code at runtime.

This design choice trades flexibility for safety. In a high-stakes environment, you do not want an agent inventing a new epidemiological model on the fly. You want it to use validated tools built by domain experts.

The modular suite is a directory of Python scripts. Each script exposes a function with a defined signature. The Executor Agent knows the function names and parameter types. When the Planner requests a forecast, the Executor calls forecast_ed_visits(data, horizon, method) and returns the result.

Validation and Observability

The system logs every query, every tool call, and every output. Logs include:

User query (sanitized to remove patient identifiers)
Planner’s interpretation (intent, parameters)
Executor’s tool call (function name, arguments)
Model output (raw data, not just the formatted response)
Execution time and error messages

This audit trail is critical. If a clinician makes a decision based on an agent’s output and the outcome is poor, the institution needs to reconstruct what happened. Did the agent misinterpret the query? Did the model receive bad data? Did the user misunderstand the output?

Validation happens at three layers:

Input Validation: The Planner checks that extracted parameters are within valid ranges. If a user asks for a forecast 10 years into the future, the agent flags it as unrealistic.
Model Validation: Each tool includes sanity checks. If a compartmental model produces negative population values, it raises an error.
Output Validation: The Executor checks that the output matches expected types and ranges before formatting.

If any validation fails, the agent explains the problem and asks for clarification. It does not guess.

Security Boundaries

Running a local LLM solves the data exfiltration problem but introduces new risks. The model can still generate harmful outputs if prompted adversarially. The system mitigates this with:

Prompt Filtering: The Planner rejects queries that attempt to manipulate the LLM (jailbreaking, prompt injection).
Tool Sandboxing: The Executor runs model scripts in a restricted environment. Scripts cannot access the network or write to arbitrary file paths.
Output Sanitization: The Executor strips any content that looks like code execution attempts or file system commands.

The agents do not have write access to the production database. They read from a read-only replica updated nightly. If an agent somehow generates a malicious SQL query, it cannot execute.

Deployment Shape

The system is a Python application running on a single workstation. The LLM (an open-weights model like Llama or Mistral) runs in a Docker container with GPU passthrough. The model suite lives in a separate container with no network access.

A simple web UI (Flask or FastAPI) accepts queries and displays results. Clinicians access it through a browser. The UI enforces authentication and role-based access control. Not every user can run every model.

Scaling is vertical. If query volume increases, the institution adds more GPUs or deploys multiple workstations. Horizontal scaling (distributing queries across a cluster) is possible but requires a message queue and shared storage for model outputs.

The deployment is intentionally simple. Healthcare IT environments are conservative. A system that requires Kubernetes and a service mesh will not get approved. A system that runs on a single box with a web interface will.

Failure Modes

The system fails in predictable ways:

Ambiguous Queries: If the Planner cannot extract clear intent, it asks for clarification. If the user keeps submitting vague queries, the agent eventually gives up and suggests contacting a data scientist.
Model Errors: If a model crashes (out of memory, invalid input), the Executor logs the error and reports it to the user. The agent does not retry automatically. A human investigates.
Hallucinated Parameters: If the LLM extracts a parameter that does not exist, the Executor rejects the tool call and asks the Planner to try again. After three failures, the agent escalates to a human.
Slow Execution: Some models take minutes to run. The UI shows a progress indicator. If execution exceeds a timeout (configurable, default 10 minutes), the agent kills the process and reports a timeout error.

The system does not fail silently. Every error produces a log entry and a user-facing message.

Trade-offs and Limitations

Aspect	Choice	Trade-off
LLM Hosting	Local open-weights model	Lower accuracy than GPT-4, but no data leaves the institution
Tool Flexibility	Fixed suite, no runtime code generation	Safer, but requires human intervention to add new models
State Management	Stateless queries	Simpler debugging, but no multi-turn conversations
Scaling	Vertical (single workstation)	Easier deployment, but limited throughput
Validation	Strict input/output checks	Fewer errors, but more rejected queries

The system is not a general-purpose AI assistant. It is a domain-specific orchestration layer. It does not answer medical questions. It does not recommend treatments. It runs models and presents results.

When to Use This Pattern

This architecture makes sense when:

You have a library of validated models that domain experts trust.
The models are too complex for end users to run directly.
Data cannot leave your infrastructure (compliance, security).
Queries are predictable enough that a fixed tool suite covers most cases.
You need an audit trail for every decision.

It does not make sense when:

You need real-time responses (sub-second latency). Model execution takes time.
Your models change frequently. Updating the tool suite requires redeployment.
Users need exploratory analysis. The agent cannot invent new analyses on the fly.
You have a small team and cannot maintain a local LLM infrastructure.

Code Example: Tool Call Interface

Here is how the Executor Agent calls a forecasting tool. The tool is a Python function in the model suite.

# tools/forecast.py
import pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing

def forecast_ed_visits(data_path: str, horizon: int, method: str = "exponential_smoothing") -> dict:
    """
    Forecast emergency department visits.
    
    Args:
        data_path: Path to CSV with columns [date, visits]
        horizon: Number of days to forecast
        method: Forecasting method (exponential_smoothing, arima)
    
    Returns:
        dict with keys: forecast (list), confidence_interval (list of tuples)
    """
    df = pd.read_csv(data_path, parse_dates=["date"])
    df = df.set_index("date").sort_index()
    
    if method == "exponential_smoothing":
        model = ExponentialSmoothing(df["visits"], seasonal="add", seasonal_periods=7)
        fit = model.fit()
        forecast = fit.forecast(steps=horizon)
        # Simplified confidence interval
        ci = [(f * 0.9, f * 1.1) for f in forecast]
        return {"forecast": forecast.tolist(), "confidence_interval": ci}
    else:
        raise ValueError(f"Unknown method: {method}")

# executor_agent.py
import importlib
import json

def execute_tool(tool_name: str, params: dict) -> dict:
    """
    Execute a tool from the suite.
    
    Args:
        tool_name: Name of the tool module (e.g., "forecast")
        params: Dictionary of parameters
    
    Returns:
        Tool output as a dictionary
    """
    try:
        module = importlib.import_module(f"tools.{tool_name}")
        func = getattr(module, f"{tool_name}_ed_visits")
        result = func(**params)
        return {"status": "success", "data": result}
    except Exception as e:
        return {"status": "error", "message": str(e)}

# Example call
params = {
    "data_path": "/data/ed_visits.csv",
    "horizon": 7,
    "method": "exponential_smoothing"
}
output = execute_tool("forecast", params)
print(json.dumps(output, indent=2))

The Executor does not parse the tool’s output beyond checking for errors. It passes the raw result to the formatting layer, which generates a chart or summary.

Technical Verdict

Use this pattern when you need to operationalize a stable library of complex models for non-technical users in a high-stakes, compliance-heavy environment. The agent layer reduces friction without introducing the risks of general-purpose code generation.

Avoid it if your models change weekly, your users need exploratory freedom, or you lack the infrastructure to run local LLMs. In those cases, a traditional BI dashboard or a managed notebook environment is simpler.

The key insight is that agents do not need to be creative. They need to be reliable translators between human intent and deterministic tools. In healthcare, reliability beats flexibility.

Agentic AI, minus the noise.