Agentic-Ai on

I Reached Hacker Rank by Building an Operator, Not by Grinding Boxes

Thu, 02 Jul 2026 00:00:00 +0000

Twenty-nine days ago I had never owned a machine on Hack The Box. In a month I hit Hacker rank. In the same window of time, I received an email from Anthropic that told me my account had been approved for their Cybersecurity Verification Program, and that the default safeguards blocking dual-use offensive-security work had been lifted for my organization.

Two milestones, one stretch of time. The interesting thing is what they say together, and how I earned the Hacker rank.

The Rank Is Not the Story

In January of 2021 I joined Hack The Box. Back then you had to hack your own invitation, you couldn’t just sign up. It took me around four hours to do so, and I was so happy when I succeeded. I love problem solving and have always leaned into things that challenge me. Up until June of 2026, I had only completed Tier 0 and Tier 1 of the Starting Point machines. On May 26th of this year I started back up on Hack The Box and achieved Hacker rank by the second week of June. I did not grind these boxes the way I would have done five years ago, with a terminal, a notebook, and my own two hands doing every step. I built an operator, called it Harkonnen, and connected it to HTB through their API. Over the last five years I have built up a collection of over 50 cybersecurity books that I purchased through Fanatical and Humble Bundle. As I mentioned in previous posts, I converted all this knowledge to markdowns and used them to build skills, tooling, and scripts. I wanted to prove out my theory that arming an AI harness with a specific body of knowledge and pointing it at a specific target could achieve good results. This idea is similar to my work with my Oracle HCM body of knowledge. My local AI harness reasons about a target, selects and chains its own tooling, and drives multi-stage attack paths while I supervise the decisions that carry risk.

The harness runs a six-phase method on every engagement: observe the target, think through the attack surface, plan the path, execute, verify the result, and record what it learned. Recon feeds foothold. Foothold feeds privilege escalation. Each stage writes its own notes, so the next stage starts from evidence instead of from scratch. I orchestrated the flow. The system did the repetitive reach.

Harkonnen Learns as It Goes

The body of knowledge was the starting point, not the finish line. The part I care about most is what happens after the box is done.

At the end of every session Harkonnen captures what it just learned. Not a log dump, a review: what worked on this machine, what wasted time, what pattern is worth keeping. From that review it writes new knowledge into its own store, and when a technique is repeatable it turns that technique into a skill, a script, or a piece of tooling it can reach for next time. The failure that cost me time tonight becomes a checked step tomorrow. The system is editing itself to be faster than it was an hour ago.

That is the difference between a tool and a system that compounds. A tool does the same thing every time you run it. Harkonnen does not run the same way twice, because every engagement leaves it with more skills, tighter tooling, and knowledge it did not have before. I did not have to teach it the second machine the way I taught it the first. It taught itself the parts it had already seen.

The learning does not stay on my disk. Harkonnen saves its knowledge to a GitHub repo and opens pull requests against it, so every gain is versioned, reviewable, and permanent. I read the PR, I see exactly what it decided to remember and why, and I merge it. The improvement is not a black box I have to trust. It is a diff I can read.

Not all of what Harkonnen knows comes from the boxes. I built a live intelligence feed into it, a KEV and AI-security news aggregator that pulls from trusted sources: CISA’s Known Exploited Vulnerabilities catalog and a curated set of AI-security outlets. The point is that Harkonnen is not learning from a snapshot frozen the day I fed it 50 books. It sees what is being exploited in the wild this week, and it sees what is happening to AI systems specifically, because the systems it defends and attacks are increasingly AI systems themselves.

The Skill Moved

For a long time the scarce skill in offensive security was memorization. Know the exploit, know the flag, know the one misconfiguration that opens the machine. The person who had walked the most paths was the winner.

That is now changing. When a system can hold the technique, the scarce skill becomes something else: designing the harness that wields the technique, setting the boundaries it operates inside, and knowing exactly which decision the human keeps. Which is systems engineering and security research in the same job.

This is not a tool that replaces the operator. It is a tool that raises what one operator can direct. The exploit knowledge still matters, the same way arithmetic still matters after the calculator. It just stops being the bottleneck.

Why the Anthropic Approval Belongs in the Same Post

Building an autonomous offensive-security operator runs straight into a wall by design. Frontier models are shipped with safeguards that block exactly this class of work, because most people asking a model to write an exploit are not authorized to. That is the correct default.

The Cybersecurity Verification Program is how a legitimate operator gets past that default the right way. I described my use cases. Anthropic reviewed them, verified I am a real security practitioner doing authorized work, and lifted the dual-use safeguards for my account. Vulnerability exploitation and offensive tooling are no longer blocked by default for my organization.

So the two milestones are not a coincidence stacked next to each other. They are two sides of the same claim. Hack The Box measured the capability by results. Anthropic verified the operator by review. One is proof the system works. The other is proof I am trusted to point it.

Where This Goes

Security work is moving from “an expert runs the tools” to “an expert directs a system that runs the tools.” The autonomous piece is coming whether the industry is ready or not. The open question is who is trusted to point these systems, and who understands where the rails have to be.

That is the seat I have been building toward, and now I have evidence on both sides of it. The capability, verified by a rank earned in under a month. The operator, verified by the vendor building the frontier.

The tools are getting autonomous. The scarce skill is knowing where to put the human. That part was never the model’s job to figure out. It is mine.

Auditing My Own Threat Briefing Tool Before I Trusted It

Mon, 29 Jun 2026 00:00:00 +0000

I found an open-source project that builds a weekly cybersecurity briefing for non-technical employees. It pulls a threat-intel newsletter, some linked articles, a bit of video commentary, sends all of it to a model, and writes out a clean HTML page you can drop on a company intranet. Useful idea. I wanted my own version, tuned for a security operations audience instead of general staff, and I wanted it as a reusable skill rather than a standalone app.

So I built it. Then, before I let myself trust it, I ran a real risk assessment on the thing I had just built. That second step is the whole point of this post, so let me start there and work backward.

The Tool Is Not the Interesting Part

The briefing generator is straightforward. It fetches the current threat intel live, the newsletter feed plus authoritative sources like the CISA Known Exploited Vulnerabilities catalog, then writes a single self-contained HTML briefing with the sections a SOC analyst actually wants: an executive summary, the week’s exploited and critical CVEs with a source link on every row, indicators of compromise, a MITRE ATT&CK mapping, and a prioritized action list. Every claim carries a source URL, and if I cannot source something, the rule is to omit it rather than invent it.

You can see one for yourself: here is an example briefing the tool produced , generated from real sources.

I made one deliberate choice that matters later. The original needs a runtime, four package dependencies, and a standing API key sitting in a file. My version needs none of that. It uses the fetch and search tools the assistant already has, so it runs with no install and no secret on disk. Hold that thought, because removing the key turned out to be a security decision, not just a convenience one.

That is the build. It is fine. It is not what I want to show you.

Auditing Your Own Build Is the Real Skill

Here is the move most people skip. When you build something that touches untrusted input, the valuable discipline is not shipping it. It is turning the same scrutiny you would aim at someone else’s system onto your own, before anyone is depending on it.

So I ran the briefing tool through a NIST SP 800-30 risk assessment, the same standard I would use to assess a client system. Not a checklist I filled in to feel thorough. An actual audit: characterize the system, map where data and trust flow, catalog the controls that genuinely exist, then work the threat scenarios honestly and rate the residual risk after those controls.

The honest finding came fast, and it was not flattering. This tool ingests untrusted, attacker-influenceable text, the newsletter body, the linked articles, the search results, and feeds it straight into an AI context, then writes an HTML file a human will open. That is the textbook setup for two problems. The first is indirect prompt injection: a poisoned article could try to steer what the briefing says. The second is stored cross-site scripting: a malicious headline could smuggle a script tag into the output HTML and run when someone opens the file. Both rated as my top risks, and both were real in the version I had just written.

This is the part I want a hiring manager or a client to see. Not a clean sheet that says everything is fine. A build where the author found the sharp edges himself and said so plainly.

You can read the whole assessment for yourself: here is the full NIST SP 800-30 risk assessment , with the system characterization, the threat matrix, a per-tool check for known vulnerabilities and recent breaches, and the residual risk after the fixes. It carries an overall grade, because a finding a hiring manager can act on needs a number, not an impression.

The Audit Has to Change the Build, or It Is Theater

A risk assessment that sits in a folder next to the tool is a document. A risk assessment that changes the tool is engineering. So the findings went straight back into the build.

The audit told me to treat every piece of fetched text as untrusted data and never as instructions, so that became a hard rule in the skill: summarize what the feed says, never obey it. It told me to escape every fetched string before writing it into HTML, so that became a rule too, with a check afterward that greps the output for an injected script tag and throws the result away if one slipped through. It told me to add a content security policy to the output so that even a missed escape cannot execute, so the template now ships with one. And the one risk I cannot fully engineer away, a model being subtly steered by poisoned input, is handled the honest way: a human reads the briefing before it goes anywhere. The tool does not get to publish itself.

The thing I shipped is not the thing I first wrote. It is the thing the audit corrected. That gap between the two is the build log.

Why the Missing API Key Was a Security Win

I mentioned I dropped the standing API key. The audit is where that paid off. A whole category of risk, a secret sitting on disk that could be read by the model or leaked into the output, simply does not exist in my version, because there is no secret to steal. Removing the four package dependencies did the same thing for supply-chain risk: libraries you do not install cannot carry a vulnerability into your tool.

This is worth saying out loud because it runs against instinct. The smaller, more boring build, no runtime, no dependencies, no key, was also the more secure one. Most of the time, security is not a feature you add. It is surface area you refuse to take on in the first place.

What I Want You to Take From This

If you are building anything that puts untrusted text in front of a model, the briefing tool is a fine pattern to copy. But copy the second half too. Build the thing, then audit your own thing against a real standard, find where it is exposed, and let the findings rewrite the build before you trust it.

The residual risk on mine, after the fixes, is low to medium and acceptable for an internal, human-reviewed briefing. I can say that with a number behind it because I did the work to earn the number, not because the tool looked clean on the surface. That is the difference between something that demos well and something you can actually run.

I did not bolt security on at the end. Auditing it was the build.

The Loop Is the Skill

Sat, 20 Jun 2026 00:00:00 +0000

As I was going through the deeplearning.ai Agentic AI course, Andrew Ng put a loop on the screen: build a quick prototype, run it, look at where it actually fails, fix the component that is failing most, and repeat. He called it the disciplined development process for agentic systems.

If you are learning to build AI infrastructure, that loop is the thing to learn first. Not a framework, not a model, not a clever prompt. The loop. Everything else is detail you hang off it once it is running.

What follows is what the course taught me about that loop, in the order I would teach it to someone starting out. I will be honest about one thing as I go, I had already been working with most of this before I took the course. These patterns are what you converge on when you build the real thing, which means you can converge on them too.

Start With the Loop, Not the Tools

Most people learning agentic AI start by collecting tools: a framework, a vector database, an agent library. That is backwards. The first skill is the development loop itself, because it is the thing that turns a demo into a system.

The loop is four moves. Build the smallest version that runs. Run it on real input. Find where it actually failed, not where you assumed it would. Fix the one component that is failing most, then run it again. You are not trying to make it perfect on the first pass. You are trying to make the next failure visible.

The reason this matters more than any tool: agentic workflows are multi-step, so errors compound. A small failure early corrupts everything downstream, and all you see at the end is a broken result with no idea where it broke. The loop is how you keep the failures local enough to fix. Skip it, and you are debugging a black box.

So before you reach for anything else, build the habit of running the loop. It is cheap, it is unglamorous, and it is most of the job.

Error Analysis Is Where the Work Lives

Here is the move inside the loop that beginners skip: you do not improve an agentic system by staring at the final output and guessing.

Andrew is relentless about this. When a multi-step system produces a bad result, the instinct is to tweak the final prompt until the output looks better. That is polishing the part you can see while the real break sits three steps upstream. The discipline is error analysis: trace each failure back to the component that caused it, find the bottleneck, and spend your effort there instead of on the part that was already fine.

This is a posture, not a tool. You are training yourself to ask “which component failed” before you ask “how do I make the output nicer.” Once you have that habit, the loop has somewhere to point. Without it, you are just rerunning the prototype and hoping.

When working with the PAI system, this is the principle I had reached on my own, from the trust side rather than the measurement side. The system does not get to say work is finished. It has to demonstrate it, with evidence captured the moment the claim is made. “Should work” is not an answer it is allowed to give. Same refusal, two directions: Andrew refuses to believe an unverified result because guessing does not improve a system, and I refuse to believe one because an unverified claim is worthless to the person depending on it.

Evals Turn Guessing Into Measuring

The deepest thing the course teaches is not a pattern. It is that you cannot improve what you do not measure.

An eval is just a repeatable test of whether your system did the right thing. Once you have one, the loop stops being vibes. You change a component, you run the eval, and you know whether it got better or worse instead of guessing from one impressive-looking output. The build-run-fix loop and error analysis tell you where to spend effort. Evals tell you whether the effort worked.

This is the part that feels like overhead when you start and turns out to be the whole game. A demo that works once proves nothing. A system you can trust is one where every fix is measured and every past failure stays measured. That is the difference between something that demos well and something you can run in front of real users at volume.

The Three Moves Most Beginners May Have Not Reached Yet

The course pointed at three practices that go beyond a basic verify-it-ran loop. If you are learning, these are the next things to build, in order.

The first is the agent judge. Verifying that work is correct is not the same as judging whether it is good. A deliverable can pass every technical check and still be the wrong thing to hand to whoever asked for it. The fix is a second agent whose only job is to evaluate the first one’s output against a standard, and running a few of them with different lenses so one agent’s blind spot does not become the system’s. Verification asks “is this true.” A judge asks “is this good.”

The second is adversarial verification. It is one thing to confirm a finding. It is another to actively try to break it. A separate agent, prompted to refute rather than agree, catches the plausible-but-wrong conclusion that a confirming pass waves through. The finding survives only if the skeptic cannot knock it down.

The third is the one everyone underrates: evaluation that accumulates. A system that checks its work and then forgets cannot tell you whether it is getting better or quietly getting worse. The fix is a test set that grows, where every real failure becomes a permanent check that runs on every future change. You cannot improve what you do not measure, and you cannot measure what you throw away after each task.

None of this is complicated. This is the difference between a system that produces work and one you can trust at volume.

Build the Loop Before You Build the Agent

So if there is one thing to take from all of this, it is the order. You do not need permission to build the real thing, and you do not need to wait for a course or a certificate to hand you the patterns. Nobody is keeping them behind a gate. Prototype, watch where it actually fails, fix the part that fails most, verify, repeat. You learn that loop by running it. The reason these principles read as familiar to me is that I had already hit them on real work before the course put names to them. That is what it feels like when you learn something the right way: the words arrive after the understanding, not before.

Start with the cheapest discipline that works. The discipline was never supposed to live in the model. Make your AI write down what it is about to do, then make it prove it did it. That single habit is most of the work. Then, once the loop is running, get serious about evals, because measuring whether the system is actually getting better is what separates a demo from something you can trust.

Build the loop first. The agent is what you build on top of the loop.