# Toward Reliable AI Agents for Network Management

## Abstract

Turning general-purpose LLMs into agents we trust with network configuration, planning, and troubleshooting requires solving two entangled problems: how do we build agents that stay correct on the long-tail queries operators actually ask, and how do we evaluate them rigorously enough to know whether progress is real? I'll present two complementary systems. MeshAgent (SIGMETRICS '26) attacks the build side: rather than curating ever-larger task-specific corpora—an approach our user study shows does not scale—it extracts reusable structural and behavioral invariants from a handful of sample queries, encodes them as constraints that guide LLM generation and validation, and adds an abstention mechanism for low-confidence cases, reaching 95–100% accuracy with as few as 14 constraints and improving over baselines by up to 26%. NetArena (ICLR '26) attacks the evaluation side: today's static, expert-curated networking benchmarks are typically under 300 queries and statistically unreliable, so NetArena introduces a unified state–action abstraction, plugs into real emulators like Mininet and Kubernetes, and dynamically generates unlimited queries with automatic correctness, safety, and latency checks—shrinking cross-agent confidence-interval overlap from 85% to 0% and revealing that even top agents average only 13–38% on realistic tasks. Together, they chart a practical path: execution-grounded benchmarks that expose where agents fail, and constraint-guided workflows that make them reliable enough to deploy.

## Bio

Kevin Hsieh is a Principal Researcher at Microsoft Research in Redmond, where his work spans the intersection of AI and systems/networking. His current research focuses on AI agents for network management and zero-trust network security, building on a broader research track in distributed ML systems, federated learning, and data-drift mitigation for production ML. He holds a Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University. Before CMU, he spent six years as an engineer and manager at MediaTek in Taiwan shipping systems software for smartphone chips.