Knowing AI Isn't Enough. Great AI Project Managers Connect Data, Models, and Products

AI Projects Begin with Data Architecture, Not Just Models

Looking at an AI architecture diagram on, say, Microsoft Azure, you'll realize it's not just a standalone "AI module." It's akin to an industrial kitchen where ingredients are acquired, sorted, stored, cooked, and quality-checked before being served to customers. The AI model is only a part of this assembly line.

The first block is data integration and ingestion, where data is collected in periodic batches or real-time from multiple sources: ERP, CRM, IoT sensors, internal systems, files, and APIs. For instance, a retail company predicting product demand might pull data from sales systems, inventory, loyalty programs, and even store sensors. If this "ingredient import" phase is flawed, even the best model will only learn from insufficient or incorrect data.

Next, data management and storage deals with SQL databases, NoSQL databases, data lakes, distributed storage, encryption, access control, data catalogs, and metadata - essentially your "cold storage" and "inventory ledger." You need to know where data is stored, who can access it, whether it's sensitive, and which version is used to train the model.

Following this is software development, covering backend, frontend, business logic, workflows, microservices, containers, authentication, and scalability. An AI model doesn't naturally transform into a product; it requires web apps, mobile apps, APIs, business flows, logins, and permissions. Tools like GitHub and Jenkins aid CI/CD, while Terraform facilitates Infrastructure as Code deployment.

Then we reach the AI platform, housing machine learning, NLP, generative AI, and other models. This platform must support large-scale deployment, monitoring, and lifecycle management, providing an inference layer - the "execution engine" that takes input, processes it through trained models, and returns output.

Finally, we tackle other infrastructure aspects like APIs, messaging queues, streaming services, security tools, authentication, threat detection, access control, and AI model protection layers. For instance, if an internal chatbot accesses HR documents, you can't just ask if it answers correctly; you must know who asks what, where logs are saved, and if salary data exposure is possible.

Tips for juniors: Don't attempt to learn all cloud services at once. Sketch a simplified AI project with five blocks: data entry, storage, app-model interaction, model operation, and security/monitoring. Creating this schematic view indicates substantial progress in project comprehension.

DevOps: The Foundation for More Than Mere AI Demos

A common counterargument is: "As an AI PM, why bother with DevOps? That's developer work." While you don't need to write Jenkins pipelines or Terraform modules, understanding DevOps is essential for managing timelines, release risks, testing environments, and distinguishing demos from fully-fledged products.

DevOps bridges software development and IT operations to deliver faster, more reliable, and trustworthy software. It emphasizes automation, CI/CD, infrastructure management, and real-time monitoring throughout the application's life cycle. For instance, when a feature for a product recommendation system is completed, CI/CD automates building, testing, and deployment across development, staging, and production environments, preventing risky manual copying.

An important concept is Infrastructure as Code (IaC), enabling server, network, and configuration files to be defined via code, ensuring consistent, scalable, and replicable environments. Imagine franchising a coffee shop - allowing each branch to brew based on "personal experience" would lead to chaos. IaC acts as the standardized recipe ensuring every branch has the correct setup.

DevOps also includes monitoring and logging - the system's "CCTV and logbook." When AI applications slow down, APIs time out, or users don't receive results, logs and metrics help teams detect, respond, and optimize performance swiftly. For AI PMs, these signals deeply impact release plans, SLA adherence, user experience, and business trust.

Tips: When joining a project, ask simple questions: how many environments does the team have, are releases manual or automated, where are logs viewed, how long does rollback take, who handles production incidents? You don't need to be a DevOps engineer to ask these questions, but doing so marks a leap in project management maturity.

MLOps: Ensuring Model Longevity Post-Deployment

While DevOps stabilizes software operations, MLOps keeps machine learning models from becoming obsolete after production deployment. Many new AI learners skip this as they focus on model training, accuracy, notebooks, and demos. However, within businesses, models need to perform well continually, even as data, user behavior, and business goals evolve.

MLOps extends DevOps thinking across the entire machine learning lifecycle: data collection, model training, experimentation, validation, deployment, and monitoring. It involves version control for datasets and models, automated training pipelines, reproducible experiments, and governance. For example, if last month's fraud detection model used dataset A and this week's used dataset B, you need to know which version produced which result. Otherwise, when asked, "Why did the false positive rate increase?" the team struggles to respond.

Data drift is a crucial concept - the change in data distribution over time between training and production data. Think of it like acing past exam papers but the new year's format changes entirely. You're not less capable; your practice data no longer reflects reality. AI faces similar challenges. A travel demand prediction model trained pre-pandemic might perform poorly once travel behaviors shift.

MLOps also manages model updates, scales inference workloads, continuously evaluates model performance, and executes rollbacks or updates as needed. If a new recommendation model reduces revenue, the team needs quick rollback mechanisms. If requests surge during sales, the inference system must scale. MLOps also covers traceability, auditability, documentation, and compliance with business, legal, and ethical standards.

AI PMs should be particularly vigilant here. Models in finance, healthcare, or insurance need more than "high accuracy" justification. Teams must explain which data was used, who approved it, when models changed, and whether they comply with regulations.

Tips for juniors: When writing tasks or acceptance criteria for an AI feature, don't just record "model must achieve accuracy X." Include criteria on data, versioning, monitoring, rollback, and documentation. For instance: "Model versions must be logged in the registry; the dashboard must display latency, error rate, and key metrics; rollback options for prior versions must be available." Such statements reflect AI PM-like thinking rather than simply minute taking.

GenAIOps and LLMOps: Confidence Is No Guarantee of Accuracy

Generative AI and large language models spice things up but also introduce operational complexities. GenAIOps and LLMOps are specialized MLOps branches focusing on the operationalization of generative AI models. These models are large, resource-intensive, latency-prone, ethically risky, and control-challenged.

Regarding performance and cost, GenAIOps/LLMOps employ techniques like quantization, pruning, distillation, distributing workloads between edge and cloud, which make models "leaner, faster, cheaper" while maintaining adequate quality. For example, a customer support chatbot shouldn't take 15 seconds to respond due to model heft. Users will abandon it before it displays its intelligence.

Prompt optimization, continuous evaluation, and AI red teaming are distinctive aspects. A prompt isn't just a question typed into ChatGPT; in actual products, it's a system design component. Teams need to test prompts for information leakage, user jailbreak attempts, and off-topic responses. Red teaming involves bringing in "attackers" to find weaknesses before real users or malicious actors do.

GenAIOps/LLMOps also monitor output for hallucination, bias, and toxic language. Hallucination occurs when models confidently deliver incorrect responses. For example, an internal chatbot inventing a non-existent leave policy. While seemingly trivial, if followed, the operational and legal fallout could be severe. Teams need feedback loops and retraining or parameter adjustment mechanisms based on real-world usage.

Security takes center stage: safety filters, access control, compliance checks, protection of private data, and avoidance of harmful or sensitive content generation. Real-time use cases like chatbots, virtual assistants, and enterprise search must ensure low latency, high availability, and efficient scaling.

Tips: If on a GenAI project, add these questions to your checklist: What can the model fabricate, who checks answers, does the prompt have versioning, is there content filtering, do logs capture sensitive data, can users provide feedback, what metrics assess quality beyond "appears okay"? These questions help avoid turning AI into a "confident black box."

AI Project Managers Cultivate System Maturity, Not Just Tasks

What I appreciate about the AI Project Manager role is its flexibility - there isn't a single way to manage AI projects. Business and technology environments are still shaping standards, best practices are clarifying, but each organization has its maturity level. This presents both challenges and opportunities for young professionals.

If you see AI PMs as just Jira updaters, deadline reminders, and meeting note-takers, you'll miss the best part. Skilled AI PMs operate tactically to manage specific projects but also contribute to overall AI strategy and technical discussions. You're not deciding architecture like an architect, but you can query: does this project enhance the company's AI maturity, does it allow data or pipeline reuse for future projects, are additional databases, frameworks, or governance tools needed?

The three operational groups - DevOps, MLOps, and GenAIOps/LLMOps - complement each other:

• DevOps ensures software and infrastructure reliability
• MLOps manages the ML model lifecycle
• GenAIOps/LLMOps address generative AI and LLM nuances like prompts, hallucinations, safety, latency, and cost

As an AI PM, observe your organization's maturity across these realms, enabling teams to adopt the best practices and tools suitable for scaling. Your role doesn't end after implementation. It extends into productization and daily usage. As backlogs wrap up, systems continue releasing new features, collecting performance metrics, assessing model efficacy, and determining whether experimentation can transition into real products. Here, automation, traceability, pipelines, monitoring, and governance demonstrate their value.

A common misconception is equating "managing AI projects" with "AI for project managers." Managing AI projects involves overseeing projects with AI components: data, models, software, operations, and risks. AI for project managers involves using AI to enhance PM tasks: writing user stories, summarizing meetings, analyzing backlogs, creating templates, planning assistance. Both are valuable, but distinct.

A practical example: use AI for backlog enrichment. From meeting transcripts or requirement documents, prompt AI to create technical stories following this structure: Title, priority, points estimate, story description, acceptance criteria. If information is lacking, AI notes "TO BE DEFINED." You can also request outputs as Jira-compatible data models, exporting as JSON or CSV for tool import. Remember: AI drafts only. PMs must verify logic, consult the team, confirm criteria, and keep the backlog accurate.

Final advice: Develop a "T-shaped skill" set. Broadly understand data, software, AI platforms, DevOps, MLOps, LLMOps, security, and governance. Deeply focus on project management, stakeholder communication, backlog management, risk management, and delivery. You needn't master everything immediately. However, each week, choose a concept and ask: "If I were a PM in this project, what should I ask to mitigate risks?"