What Tencent's AI PM interviews reveal about the RAG knowledge gap

A post circulating in Chinese AI product manager communities this week made a pointed observation: when companies like Tencent, ByteDance, Kimi, and DeepSeek ask "What is RAG?" in AI PM interviews, most candidates answer the first third of the question and stop. The rest of the answer is where the actual job lives. That gap is worth unpacking, because it maps onto a real split between how RAG gets described in documentation and how it behaves when you are responsible for a product built on top of it. The answer everyone gives RAG, Retrieval-Augmented Generation, externalizes knowledge from model weights. Instead of encoding domain-specific or frequently-updated information into a fine-tuned model, you maintain a knowledge base alongside the model and retrieve relevant chunks at inference time. The model reasons over what it retrieves. For a product manager, the commercially useful framing is this: RAG moves your iteration surface from the model layer to the knowledge layer. Product, operations, and sales teams can update what the system knows without coordinating a retraining run. Faster feedback loops, lower dependency on ML infrastructure for routine updates. That answer is correct. It is also incomplete, and interviewers at these companies apparently know exactly where to push. Where RAG actually gets difficult The Chinese PM community post highlights two specific hard problems that separate candidates who have shipped RAG pipelines from candidates who have read about them. Chunk granularity. When you index documents into a retrieval store, you have to decide how to split them. Chunks that are too large return imprecise matches: you retrieve a full section when you needed a specific paragraph. Chunks that are too small fragment context and force the model to reconstruct meaning from pieces that do not cohere. There is no universal correct setting. It is a tuning process that depends on document type, query distribution, and what "good retrieval" means for your specific use case. The post notes that mentioning you have actually tuned chunk size in an interview is the signal that tells an interviewer you have debugged a real pipeline rather than summarized one. Context cost and latency. Every RAG call appends retrieved text to the prompt. That text costs tokens, and tokens cost time and money. In latency-sensitive applications, customer support tools, real-time assistants, the overhead of retrieval plus a large context window can break your response-time budget entirely. Knowing when that cost is worth paying, and when a different architecture is the better call, is where senior judgment shows up. The boundary question The post frames RAG boundary awareness as the actual senior-PM differentiator, and the guidance it offers is specific: Use RAG when the enterprise has private data that cannot be embedded in a public model, and when answer verifiability matters to the business. Compliance, support accuracy, legal contexts all fit. Avoid RAG when retrieval quality is hard to control due to poorly structured source documents, when the knowledge is stable enough to fine-tune instead, or when latency constraints make the retrieval overhead prohibitive. RAG with poor retrieval does not just underperform. It produces confident wrong answers. In high-stakes contexts that is worse than no answer at all, and a PM who does not understand that boundary will not catch the failure mode until it has already done damage. What this says about the AI PM role The meta-point from this discussion is that chunk-size tuning and context-cost reasoning are now being positioned as product manager knowledge, not just engineering knowledge. Two years ago, a PM was expected to understand RAG conceptually and defer implementation details to an ML engineer. The interview culture at Chinese big-tech companies has moved that line. Whether that is a sign that AI PM roles genuinely require more technical depth, or whether interview culture is overcorrecting toward engineering trivia, is a real question. The community post does not resolve it. It just coaches candidates to clear the bar as it currently exists. The question worth sitting with: if you have hired or interviewed for AI PM roles in the last year, what retrieval-pipeline depth do you actually expect candidates to own? Has the bar shifted, or is this a Chinese big-tech specific phenomenon that has not crossed over yet? Top comments (0)