Case study · AI-native pipeline engineering
Sourcing to submission, so my time goes to judgment.
An AI-driven job application pipeline: discovery, fit screening, resume and cover letter generation, tracking, engineered so my time goes to judgment. The bank matches and aligns my own verified experience to a job, always drawn from what I actually did.
Situation
Coming back to the industry after five years running a business meant re-entering a job market flooded with candidates. Volume alone was a problem. Dozens of listings worth pursuing, each needing a resume and cover letter built specifically for that role, tuned to that company’s priorities and that JD’s language. In a market that crowded, a generic application is invisible. Signal is the whole game, and signal is expensive to produce by hand at the volume a real search requires.
The market wasn’t the only problem. I hadn’t sat down to write an actual resume in close to ten years. Before any system could search, screen, or match anything, the raw material to build one from didn’t exist yet in any usable form.
Task
Facilitate the whole application process with AI. Source and score which roles were worth my time before spending any of it reading a JD closely. Rebuild the content itself into real evidence instead of activity. Curate and tailor that content per role, collaboratively, filling gaps and drafting cover letters together. Track every application so nothing got lost. And run several roles in parallel, so time-to-submission stayed roughly flat as the number of roles I was serious about grew.
There was a constraint underneath all of it. Re-entering a technology career after five years away meant I needed real hours back for skill development. A system that saved time only to leave me doing the same mechanical curation, faster, would have missed the point.
Action
Every stage below got built the same way. I described what I wanted in plain English, stream of consciousness, by voice. A PRD skill I’d built turned that raw description into the questions I hadn’t answered yet. Claude Code built and validated each piece incrementally from there, one function at a time. Here’s what that produced, stage by stage.
Foundation. Two things existed before any pipeline did.
-
Content. Bullets are mined from decades of prior resumes and created by hand where gaps existed, then rebuilt through several evolutions to strengthen them, first in ChatGPT, then restructured in Claude into Impact-Decision-Execution: the outcome, the decision and judgment behind it, and how it was actually done. Tagged by job relevance and competency demonstrated, scored, spanning 31 years, 26 at Microsoft and 5 running my own business.
-
Rules. A short set of rules governs how AI is allowed to use that content:
- Bank content stays verbatim.
- Every AI edit gets bolded, visible and reversible.
- A gap gets workshopped with me, directly.
- Content comes from exactly two places, the bank or a live workshop.
- Facts get verified.
The tool that enforces all of this, the thing that queries the bank, proposes a lineup, and runs the workshop, is its own build, separate from the content itself.
The pipeline, stage by stage:
-
Discovery. A scraper polls an extensible list of job sites daily. A fast filter clears the obvious disqualifiers first, location, salary, the hard rules. A second, deeper filter reads the actual job description against a real understanding of what I’m looking for, evaluating fit both ways: whether the role fits me, and whether I fit the shape of the role. Both filters publish their results to my knowledge management system, and the last step of the night pre-generates an initial resume bullet selection, a draft ready the moment I wake up.
-
Human decision. I review that queue, recommendation, draft, and blind score attached, and decide what’s actually worth my time this week.
-
Resume workshop. Builds the resume itself:
- Bullet selection, ordering, and tuning to tell the right story for this job.
- Summary construction, skills, and ATS alignment.
- A depth check on every bullet: does it actually map to what the job description asks.
- Scored, then held at a checkpoint before anything ships. I see the full lineup and why each bullet was picked.
Where the score shows a weakness, we work through it together in a workshop. Bullets get added, removed, or strengthened based on exactly what the scoring flagged.
-
Cover letter. Fills what the resume can’t reach:
- Why this company, why this job, why now.
- The proactive questions, addressing likely evaluation concerns before they’re raised.
- Construction, then editing, all in a live workshop with me. Never generated by AI.
- Scored independently, blind, then scored again as a pair with the resume, evaluating the whole application package.
-
Tracking and feedback loop. Every application is logged in the same knowledge system I use for everything else, building toward a real comparison between what progressed and what didn’t, once there’s enough volume to show a real pattern.
Here’s what that workshop step looks like in practice. One bullet in the bank was trying to do too much: blending a customer retention win, a new role I’d created, a quality score, and a revenue recovery into one overloaded sentence. In a workshop session, I told the system the real story behind it. It surfaced three separate accomplishments hiding in that one line. Split apart: the $150K revenue leak I found and built an accountability system to close; the retention improvement from 65% to 85% from a new, dedicated quality role; and the technician quality score climbing from 80% to 95% from a cradle-to-grave quality program. Same underlying material. Three specific, verifiable bullets, each one provable on its own.
The system runs multiple builds in parallel on top of a real concurrency mechanism. My own cognitive capacity is the real limit, how many resumes I actually want to work on in a day. The system itself can run far more sessions than that. Several sessions read and write the same bullet bank at the same time, and none of them can corrupt it. The mechanism, the resume system’s own base/lens/bank structure, and a real conflict it caught in production, are all in the deep-dive below.
Architecture deep-dive optional, for the technically curious
The pipeline: six stages, two kinds of ownership
Action names these stages the way a reader can follow them. The system names them the way they’re actually coded. Same pipeline, two vocabularies:
| Action | The system |
|---|---|
| Discovery | Nightly scrape, Stage 0 fit screen, and Stage 0.5 autobuild, run together overnight |
| Human decision | Review the queue |
| Resume workshop | Stage 1, assess/select/draft |
| Cover letter | Stage 2, generate/apply/verify (the drafting half) |
| Tracking and feedback loop | Stage 2’s tracker write, plus the Notion Job Tracker running underneath every stage |
3 job boards, dedupe, filter, write to Notion staging.
Headless Claude call, 5-tier rubric, dontAsk permission mode.
The essay's manual Notion-AI triage.
First-draft resume, then a second, fully separate agent scores it blind, no build context.
So the model can't grade its own homework.
I read the Go / Stretch-Go list, first draft and blind score attached.
System proposes a lineup.
Stops the session cold and shows every bullet, its score, and why it was picked.
DOCX + cover letter, then one question: did you apply?
No Notion page ID back = did not happen.
Conceptual Stage 3, interview prep, is real infrastructure, but it’s a separate system outside this pipeline.
The resume system: base, lens, bank
Knows my rules, my workflow, my failure modes. Does not know the job.
Chief of Staff / TPM / Product Management. Decides which bullets to prioritize, which story arc to emphasize for this reader.
The bullets themselves, scored and tagged, Impact-Decision-Execution structure. The only layer that took a year-plus to build.
A CoS reader and a TPM reader value the same accomplishment for different reasons, so the lens decides what to prioritize. It never touches the bank itself. The bank is scored and tagged once; every lens queries the same underlying asset differently.
Concurrency: the mechanism, and a real conflict it caught
The resume bullet bank is a single CSV, git-backed, no database. More than
one session can tailor resumes against it at once. An earlier version of this
system wrote by row position and once corrupted data when a parallel session
inserted a row above the row it was writing to. That’s the kind of bug you
only learn from once. Every bullet now carries a permanent BulletID
(B0001…) that never changes, and every write targets the ID, never a
position.
A session never touches the shared master file mid-build. It snapshots the bank at the start and journals its edits. A serialized merge step then runs a real 3-way comparison: base against the snapshot, theirs against the current master, ours against the journal, before it writes anything back. A session registry shows which build is waiting on a merge, so running several sessions in parallel works like a checked-in, checked-out discipline.
Real git history from one afternoon:
session (Aquent AI Data Engineer)
session (Aquent AI Data Engineer)
session (Aquent AI Data Engineer)
Three sessions branched from the same base version the same day. Most merges were clean. One wasn’t. The merge step caught a real collision on a bullet two sessions had both touched since the shared snapshot. It flagged the conflict and held it for a manual resolution. That’s the mechanism doing exactly the job it was built for, on a real production day.
A real PRD, including the honest counterpoint
The design document behind this mechanism (PRD_Parallel_Sessions_Bank_Concurrency.md)
does something most retrospectives skip. It names the technically cleaner
alternative and explains why it lost anyway.
Honest counterpoint (on record): if engineering time is being spent anyway, a flat CSV is a weak concurrency substrate and SQLite would make locking/atomicity/IDs free. [The chosen approach] wins on migration cost and toolchain fit today, not on technical purity.
The same document still carries open questions, dated and unresolved, raw exactly as they stood the day they were written: how to handle a conflict-resolution UX when two sessions edit the same bullet, what the retention policy should be for snapshots and journals, the exact stale-lock timeout. A real PRD keeps a decision log with rationale and a list of what still isn’t decided.
Maturity: hardened against real failures
| Type | Change |
|---|---|
| Initial commit | Nightly scraper, three sources, Notion write-up |
| Fix | Silent network failures being marked as successful runs |
| Fix | Stale-run alert missing last-run date and elapsed hours |
| Fix | launchd throttle from a wake-check restart loop |
| Fix | launchd TCC permission block, invisible until an unattended run hit it |
| Redesign | Fit-screen rubric to 5 tiers, full re-screen |
| Add | Stage 0.5 autonomous first-draft build |
Three separate unattended-automation failure modes, each caught, diagnosed, and hardened against in production. That’s the part a demo skips past to get to the feature list.
Result
- 6 conversations from 84 applications since the system went live, a little over 7%, up from 1 conversation across roughly 200 applications before it. Nearly 14 times the old response rate, and ahead of the broader tech market too, where cold applications are converting at roughly 2-3% industry-wide in 2026.
- 4-5 resumes now take 1.5-2 hours in parallel, by my own logged time, versus the 8-12 hours the old serial rate of 2-3 hours each would cost for just four. That time includes strengthening the bank itself as I go, the same bank every future resume draws from. Most of what’s left is judgment, storytelling, and content. The headless, autonomous stages are the accelerator; the thinking is still mine.
- Discovery runs the whole autonomous half unattended, overnight. A nightly scraper finds and filters listings, an autonomous fit-screen scores them against a 5-tier rubric, and a first draft with a blind, independent score are both waiting by the time I decide what’s worth my time this week. That’s why time to start working dropped from about an hour to five or ten minutes: I open an inbox already stocked with a reviewed queue, a draft, and a score.
- The concurrency mechanism has been tested by real production use. Parallel sessions against the same bank have produced a real flagged conflict, caught and resolved by hand instead of silently corrupting data. Detail in the deep-dive.
- The bank keeps compounding. Every session that strengthens a bullet writes the improved version back, so it grows stronger the more it gets used.
Learning
The bottleneck was the quality of the context I fed the AI. Every stage of this pipeline shares almost nothing in implementation: Python scraper, prompt architecture, Notion automation. They all run on the same principle. Structured personal knowledge plus AI synthesis produces output a generic tool can’t match, and the knowledge is the part that takes time to build.
The second thing I learned is where the automation line actually sits. If I can fully describe a piece of work as an SOP, no judgment call buried inside it, that’s a strong automation candidate, and that’s most of Discovery. The moment a step needs judgment, a human has to stay in the loop, and the rules around that step matter more, not less, because that’s exactly where an ungoverned AI would improvise. Structured that way, AI stops being a construction tool and becomes a real thought partner, clearing everything around the judgment so the decision gets my full attention.
The third thing I learned is that naming your own gaps in public is worth
more than it costs. I published an essay naming exactly what this system
didn’t do yet, including that it graded its own homework and that a bullet
had no permanent link back to its source. Both gaps are closed now. A second,
fully separate model scores every build blind, and a permanent BulletID
replaced the row-position addressing that once let an earlier version of the
bank corrupt data. Closing a named gap five months later is the same
discipline continuing past the point where the writing stopped.
All selected work