Putting agentic AI to the test

Bill Doerrfeld | February 28, 2025

The latest edition of DirectorPlus explores how New Relic is utilizing agentic AI and what the results have been.


A lot is being said about agentic AI right now. But is it all hype?


I recently synced with Camden Swita, Head of AI and ML Innovation at New Relic, on this loaded topic for LeadDev — we dove into the details to explore how New Relic developers are actually putting it to the test and seeing real-world results.


This one is loaded with on-the-ground insights. For instance, I never dreamed image-to-text models would outperform raw text for things like navigating a database schema!


Success with agentic AI involves tracking the right metrics, bridging deterministic and probabilistic AI, getting input from various domain experts, and discovering new optimizations, says Camden.


Finding success with AI is a moving target, especially with mounting quality concerns. So, it's fascinating to see how agentic AI is being approached in real-world settings.


Check out the full story on LeadDev! And be sure to subscribe to DirectorPlus if you'd like this monthly column in your inbox. 


Featured image credit: Cash Macanaya on Unsplash.


Read: AI agents and the missing link in DevOps

Sign up to the DirectorPlus newsletter for a new story in your inbox each month featuring on-the-ground insights from some of the industry's top engineering leaders. Or, if you would like to be featured in the series, please read this.

Other Blog Posts

How AI Agents Change API Interaction opening slide doerrfeld bill 2026 nordic apis
By Bill Doerrfeld • October 18, 2026
Here are my slides from my opening keynote at Nordic APIs Summit 2026, exploring how AI agents are changing API interaction patterns.
By Bill Doerrfeld • October 17, 2026
My latest research for AI Security University looks at AI tools and resources for pen-testers, red teamers, and bug bounty hunters.
By Bill Doerrfeld • October 16, 2026
My commissioned feature for Arthur D. Little's PRISM publication analyzes the big shifts underway now that generating code is becoming trivial.
By Bill Doerrfeld • October 11, 2026
Was happy to participate as a speaker and moderator at SecureMaine on October 8th, 2026.
By Bill Doerrfeld • October 5, 2026
My latest for LeadDev interviews a Meta engineer who built an agentic system to optimize their capacity planning, leading to real power use savings.
By Bill Doerrfeld • September 30, 2026
I set out expecting to write an article about how every enterprise was jumping on trendy alternative or "neo" clouds. That ended up not being the case.
By Bill Doerrfeld • September 28, 2026
Join the virtual AI Pen-Testing Conference to explore how AI is changing penetration testing, red teaming, and the security testing of AI systems.
By Bill Doerrfeld • September 28, 2026
Counting down to Nordic APIs Summit 2026 in Stockholm, featuring 60+ speakers exploring agentic AI, APIs, MCP, and security.
By Bill Doerrfeld • September 28, 2026
I covered a Yale and GitHub report on LeadDev that explores developer attitudes on climate change and curbing IT waste.
By Bill Doerrfeld • September 15, 2026
Does your agent really need access to every tool call under the sun?