My First AI Judge Interview: What Could Possibly Go Wrong?
My First AI Judge Interview: What Could Possibly Go Wrong?
The Challenge & Implementation
HackerRank organized the September 2026 edition of Orchestrate, a 24-hour hackathon featuring the "Buy or Wait?" challenge. The task asked whether someone could afford an expense while meeting future commitments. HackerRank supplied the assignment and data, and I named my implementation Praxi Clew.
The build system used AI to extract facts from messages and images, then employed Python to forecast balances over 90 days and evaluate payment options. Astra handled orchestration, coordinating the AI extraction step with the Python forecasting. Antigravity, Google's agentic coding tool, implemented the code, running Gemini 3.8 as its model in the IDE. ChatGPT Astra reviewed the artifacts throughout the process.
The Submission Requirements
The submission required a code archive, recommendations for 250 requests, and a development transcript. I directed the work and increasingly insisted on evidence, limited scope, and an end to the correction cycle. The evaluation focused on whether I could explain the code at a deep level-not just direct it or evaluate what came back.
The AI Judge Interview
The 30-minute AI Judge interview featured Chakra as the judge. The interview was a voice-based session where you "walk the AI judge through your architecture, your decisions, and your tradeoffs." The conversation started with introductory questions, followed up on what I said, pushed back on answers, and used those responses to shape subsequent questions. I found the interviewer to be very intelligent.
During the interview, I repeatedly and clearly communicated that I had used AI throughout the project-this was intentional since I wanted to win. Near submission, I was reading about previous Orchestrate results and wondering whether they offered an edge. I even asked ChatGPT to help me prepare for the interview. Despite feeling concerned and nervous, I participated fully.
One critical moment occurred when I needed AI to explain the build in real time during the interview. The system built with AI needed AI to explain itself mid-interview. This assistance belongs in the account. Unfortunately, I do not have the full spoken transcript, so the questions below come from what I recorded in the chat.
Key Questions Asked
Variable Expenses: What value was used to estimate variable expenses? The submitted function used the median of up to five recent amounts in the recurring series-a reasonable choice to reduce the influence of an unusually large or small purchase, though it has limits since typical spending is not a guaranteed upper bound.
Order of Events: If salary and bills landed on the same day, what happened first? The code credited confirmed income, deducted existing expenses, then deducted the proposed purchase payment. This assumes income is available before outgoing payments that day-it does not model the exact settlement times banks use. An earlier report had stated debits came first, but inspecting the submitted function revealed credits came first, exposing a discrepancy between the explanation and the actual implementation.
Spending Changes: Python attempted combinations of up to three permitted changes-such as stopping an eligible expense or reducing it to its minimum allowed amount-and checked payment plans against the forecast, excluding protected categories.
Verification: How did we know the outputs were correct across all 250 requests? The public dataset provided 25 examples with expected answers, while the remaining 250 were the submission requests. We could verify format, permissions, schedules, and balances against the forecast, and replay exported evidence to reproduce the identical output file. However, we could not claim agreement with expected answers we had never seen-the build had already demonstrated this limitation.
In request 185, a 12% rent increase became rent of €12. The validator passed because it used the same incorrect rent as the forecast. Comparing the extracted fact with the original message exposed the error. Correcting rent from €451 to €505.12 moved the recommended payment from December to January. This illustrates why I could not answer every question with "we validated it"-validated what, against which inputs, and with what limitations?
Ownership vs. Assistance
The question about what I designed myself highlighted a tension: I design my projects, vet the implementation, and troubleshoot bugs with agents. HackerRank supplied the assignment, and AI proposed and wrote substantial parts of the technical solution. I remained responsible for the project as a whole-including whether I could explain the code, not just direct it or evaluate what came back.
Mid-build, I paused and questioned the process. We were going in circles: another confident report, another correction, another review. I requested a reset and a complete assessment of the files and requirements, and I set guardrails so a targeted fix would not become permission to change unrelated behavior. This intervention fundamentally changed the review process. We assembled the actual code, output, requirements, and extracted evidence so the next assessment could examine them together. I was simultaneously troubleshooting how we were evaluating the system and directing corrections to the system itself.
Results & Reflection
The interview ended okay but I was definitely not confident. I felt relieved simply to have completed it. After the Orchestrate event concluded, I received my score: 66.1 out of 100. My reported placement was 139 out of 3,062 participants, approximately the top 4.5%.
Interestingly, the transcript received full marks. I cannot determine which individual decisions earned those marks, but it was my strongest evaluated component. The interview, code, and output scores were relatively close. The results do not support blaming everything on interview nerves alone-the public sample results had already shown limitations: 80% agreement on payment method, but only 12% on the exact safe payment amount. The forecast needed work, and so did my ability to explain it, especially this.
For my next build, I want explanation to become a checkpoint while the implementation is still taking shape. When an agent hands me an answer, I want to be able to describe back what a component does, which inputs it uses, and where it can fail. I do not yet know exactly how I will check that against the code every time-that is what I need to figure out next. It will probably interrupt the satisfying pace of the agents firing on all cylinders, but I believe that interruption is necessary.
Looking ahead, I plan to read and review the interview rules as carefully as the build requirements, including what assistance is permitted during any live assessment. Knowing my own implementation well enough to survive questions more specific than anything I prepped for is the harder problem-and the one I actually need to solve. Next time, I am saving the receipts: the chat logs, the test outputs, every place where I need to know the reasoning, not just the result. HackerRank's transcript logging instructions are worth reading before starting, too.
This is my personal participant account. HackerRank organized Orchestrate and supplied the "Buy or Wait?" challenge and data. Scores and placement come from my result display. Antigravity assisted with implementation, and ChatGPT Astra assisted with review, interview questions, and this article. The overall effort was AI Assisted. Human Approved. Powered by NLP.
Comments
No comments yet. Start the discussion.