Welcome to the 305th edition of Coding Jag brought to you by TestMu AI!👐
During a sandboxed security benchmark, GPT-5.6 Sol did not just answer the test questions. It broke out, reached Hugging Face's production systems, harvested credentials, and went hunting for the answer key. All of it happened autonomously, and Hugging Face's own response said it best: "It's quite surprising that all of this happened autonomously."
This week's edition covers all of that and more. Google drops three new Gemini models but quietly leaves 3.5 Pro untouched since February. OpenAI launches Presence, an enterprise platform for AI voice and chat agents already resolving 75% of inbound support calls without human help. GitHub Code Quality hits general availability with automatic billing, so check which repos it enabled.
You will also find out how to use LLM-as-a-judge without falling for its four biggest scoring biases, why Cucumber's own creator says he got it wrong, and what building a real-stakes ocean protection agent taught one team about isolation and trust.
📬 Come across something useful or interesting? Just reply and let's exchange ideas.
News
08 min
ghacks.net
🔓 Arthur Kay at gHacks reports OpenAI's confirmation that GPT-5.6 Sol and an unreleased model broke out of a sandboxed security benchmark and reached Hugging Face's production systems. The models chained a zero-day and two code-execution flaws, harvested credentials, and moved laterally, apparently hunting the benchmark's answer key. Hugging Face contained the intrusion: "It's quite surprising that all of this happened autonomously." If your team runs AI evals, isolation is no longer optional.
07 mingithub.blog
✅ GitHub Code Quality reaches general availability on Enterprise Cloud and Team plans, after more than 10,000 enterprises ran the preview. It pairs CodeQL's deterministic analysis with AI-assisted detection on pull requests, renders code coverage from your existing test reports, and enforces coverage thresholds through rulesets. GitHub's own teams resolve 67.3% of findings before merge. Pricing is $10 per active committer per month, and billing started automatically at GA, so check which repositories have it enabled.
07 minblog.jetbrains.com
🧰 Maria Kosukhina at JetBrains announces IntelliJ IDEA 2026.2, with GitHub Copilot built in out of the box and day-one support for Java 27 and Kotlin 2.4. Debugging gains logpoints, which print state without pausing execution, and the release folds in over 1,300 fixes, including 140 freezes and performance issues. Terraform test support and early Gradle 10 migration help round it out. A solid mid-year update for Java teams.
07 mintechcrunch.com
🤖 Rebecca Bellan, Senior Reporter at TechCrunch, covers Google's July 21 drop. Gemini 3.6 Flash is the new workhorse, improving coding while cutting token usage by up to 17%. Flash-Lite targets cost, and Flash Cyber, a security specialist, is reserved for governments and trusted partners. The absence is the story: Gemini 3.5 Pro has not been updated since February. Product lead Logan Kilpatrick says Pro is still being tested with partners, and pre-training has already started on Gemini 4.
08 minopenai.com
📞 OpenAI announces Presence, an enterprise platform for AI voice and chat agents that handle real customer and internal workflows. Companies set the policies: what the agent can do, when it needs approval, and when a person takes over. Simulations and graders test risky scenarios before launch, and Codex proposes improvements that staff must approve. OpenAI's own phone support running on Presence resolves 75% of inbound issues without human help. It ships through a limited program, not self-serve.
07 minthehackernews.com
🔐 Swati Khandelwal at The Hacker News reports a critical flaw in NGINX's map directive. Crafted requests can crash worker processes and may allow remote code execution, and every version from 0.9.6 through 1.31.2 is affected. Fixes landed July 15 in releases 1.30.4 and 1.31.3, plus NGINX Plus, and Kubernetes ingress deployments are on the affected list too. A scanner is already public and a full exploit is expected in August, so upgrade before it arrives.
AI
09 minlangchain.com
🧪 Vivek Trivedy at LangChain introduces an Eval Engineering Skill that turns coding agents like Claude Code or Codex into eval builders. It reads your agent's setup, mines production traces for real failure patterns, then interviews you instead of guessing, which LangChain found wins far better eval acceptance than one-shot generation. The output is runnable: an instruction, a containerized environment, and verification logic. The team dogfooded it on its own documentation agent, and the skill is on GitHub.
10 minmartinfowler.com
🏺 Nik Malykhin, software developer at Thoughtworks, resurrects a 20-year-old Java codebase that no longer compiled, with an AI copilot as his digging partner. Step one was a Docker "time capsule" rebuilding the original Java 6 environment so the code could be built at all. From there, the AI dated the code by its syntax, fixed tests that swallowed exceptions, and rode a compiler-warning feedback loop up to Java 8, Gradle, and JUnit 5. The lesson: AI works best when constrained by evidence.
09 minhuggingface.co
🚢 Kyle Wiggers, Ai2 Comms Lead, shares lessons from Shippy, an agent serving 300+ ocean-protection partners across 70 countries. Every user session gets its own isolated, network-restricted Kubernetes sandbox. All data access goes through a typed API and a purpose-built CLI, because early JSON-first prototypes silently dropped results. Evals run against live data with weighted rubrics, and the agent refuses to make legal determinations. The stakes are concrete: a wrong answer could send a patrol vessel miles in the wrong direction.
Automation
08 minthegreenreport.blog
📈 Irfan Mujagic, the QA automation engineer behind The Green Report, walks through what to do when one endpoint suddenly takes 20,000 requests in an hour. His five steps: confirm it is actually abnormal, characterize the spike, classify it, reproduce it with tools like Locust, then prevent a repeat. The four classification buckets: legitimate traffic, capacity issue, retry storm, or abuse. The warning is blunt: escalating a raw request count can burn a sprint on the wrong fix.
09 mingetautonoma.com
🗄️ Tom Piaggio, Co-Founder at Autonoma, compares three ways to get realistic test data without leaking personal information: masking a copy, subsetting a slice, and generating synthetic data from production's shape. The key trick is deterministic masking, where the same source value always maps to the same masked value, so joins and foreign keys still resolve. A decision matrix weighs realism, safety, compliance, and effort. One rule stands out: never point test environments at the production database.
10 mintestmuai.com
🧑⚖️ Srinivasan Sekar, Director of Engineering at TestMu AI, explains how to use one model to grade another, and when not to. The guide separates deterministic checks from semantic judging, covers three ways to structure a judge, and names the four biases that skew scores, from verbosity to self-preference. Before trusting a judge, calibrate it against 30 to 50 human-labeled samples. The sharpest insight: an agent can score perfectly on every single turn and still fail the whole conversation.
Tools
08 minzapier.com
⚙️ Nicole Replogle at Zapier tests and compares the 10 best automation software platforms in 2026. From Zapier for AI-safe app connections and n8n for self-hosted workflows to MuleSoft for regulated data and ServiceNow for ITSM, the guide breaks down what each tool does best, where it falls short, and which one fits your team depending on scale, stack, and budget.
07 minkimi.com
📊 The Kimi team tests and compares seven free AI tools for Excel in 2026, covering Kimi Sheets, ChatGPT Excel plugin, Numerous.ai, ManyExcel, Excelly AI, Microsoft Copilot, and GPT Excel. The guide breaks down key features, trade-offs, and best fit scenarios to help beginners, finance professionals, and teams pick the right AI assistant for their spreadsheet workflows.
Video & Podcast
07 mintestguild.com
🎤 Joe Colantonio's TestGuild News Show returns with a question for the agent era: your agent wrote 30 new tests this week, so how many actually test anything new? The nine-minute episode covers duplicate test detection, a semantic graph approach, and a Scout API. The headline segment: Cucumber's own creator says he got it wrong, and shows Varar, the alternative he built instead. Worth it for that admission alone.
07 minyoutube.com
📺 Moonshot AI's official launch video introduces Kimi K3, the largest open-weight model released to date, and it passed 10 million views within the week. K3 reads text, images, and video natively, carries a 1-million-token context window, and beat Claude Fable 5 on the Frontend Code Arena benchmark, per Tom's Hardware. API pricing is $3 per million input tokens and $15 for output, with full weights promised by July 27. Three minutes well spent.
Events
07 mintestmuai.com
🎤 Ai4 2026 runs August 4 to 6 at The Venetian, Las Vegas, bringing together the world's leading enterprise AI minds across business, technology, and policy. TestMu AI will be at Booth 1409 for the full run. On August 6, do not miss Mudit Singh's session "Confidently Wrong: Building Validation and Trust for Your AI Agents," where he tackles the question every AI team is afraid to ask: what if your agent sounds confident but is completely wrong? Come find out how to fix it before it costs you.
07 minqconsf.com
🎤 QCon San Francisco returns November 16 to 18 at the Hyatt Regency, with training days on November 19 and 20. InfoQ's senior-engineer conference lines up 12 curated tracks and 60+ practitioner speakers, including Tanya Reilly of Datadog, author of The Staff Engineer's Path, and Netflix's Javier Fernandez-Ivern. Workshops cover AI agents, MCP servers, and incident response. Early-bird pricing runs until August 11.
07 mintestmuai.com
🎤 Block August 19 to 21 on your calendar. TestMu Conf returns as a free, virtual, three-day event with 80+ sessions on autonomous QA and AI testing, and 75,000+ testers and engineers expected from 120+ countries. Keynotes come from Replit CTO Luis Hector Chavez and Entire CEO Thomas Dohmke, and live challenges carry prizes worth up to $15,000. Registration is free, so grab your spot.