How Should SaaS PMs Evaluate ML for Product Features?
For Product managers at SaaS companies · Based on Ng Machine Learning Orientation Framework
// TL;DR
The Ng Machine Learning Orientation Framework helps SaaS product managers systematically evaluate whether a product feature actually needs machine learning. Instead of chasing ML hype, PMs walk through six steps: state the feature problem in plain language, check if rules-based logic suffices, map it to a known ML pattern, verify data exists, classify it on the consumer-to-industrial spectrum, and write a clear ML justification. Use it during feature discovery, roadmap planning, or when engineering proposes an ML-based solution and you need to validate the approach before committing resources.
Why do SaaS product managers need an ML evaluation framework?
Machine learning is one of the most over-proposed and under-validated technologies in SaaS product development. Engineering teams suggest ML for features that could be solved with simple rules, or worse, product managers greenlight ML projects without verifying that training data exists. The Ng Machine Learning Orientation Framework, based on Andrew Ng's foundational DeepLearningAI teaching, gives PMs a structured six-step process to separate genuine ML opportunities from hype.
The core principle is simple: machine learning is the science of getting computers to learn without being explicitly programmed. If you can write every rule by hand and they will not change, you do not need ML. If the rules are too complex, too numerous, or evolve with user behavior, ML becomes the right tool.
How do you evaluate a proposed ML feature as a PM?
Start with step one: write one plain-language sentence describing what the feature should do. Not "we need a recommendation engine" but "we need to surface the three most relevant help articles when a user submits a support ticket." This forces clarity.
Step two: can this be solved with explicit rules? If your product has 50 help articles and clear keyword matching works, ML is overkill. If you have 5,000 articles and user queries vary wildly, rules break down.
Step three: map it to a known ML pattern. Your help article feature maps to ranking and retrieval — the same pattern powering web search. This tells your engineering team what class of models to explore.
Step four: check the learning signal. Do you have historical support tickets paired with the articles that actually resolved them? If yes, you have labeled data. If not, you need a data collection strategy before ML is viable.
Step five: classify it on the consumer-to-industrial spectrum. In-app article recommendations are consumer-facing ML — they tolerate some imprecision and improve with scale.
Step six: articulate the justification. "User queries are too varied and our article library too large for keyword matching to reliably surface the best content. The system needs to learn relevance from historical resolution data."
What mistakes do SaaS PMs make when greenlighting ML projects?
The most common mistake is skipping the learning signal check. A PM approves an ML feature assuming the data exists, only to discover months later that no labeled examples are available. The framework forces this check in step four.
Another mistake is conflating ML complexity levels. A churn prediction model (classification) and a real-time pricing optimizer (optimization) are very different ML problems with different data needs, timelines, and risks. The pattern mapping step prevents this conflation.
Finally, many PMs describe features in ML jargon they picked up from blog posts. The framework's first step requires plain language, which exposes whether the problem is well-defined or just buzzword-driven.
What should a PM do after completing the framework?
Once the six steps produce a validated ML opportunity, use the output as the foundation for a product requirements document. Your plain-language problem statement becomes the feature description. The pattern mapping guides engineering's technical approach. The learning signal assessment becomes the data requirements section. The ML justification becomes the executive summary. Share this document with engineering, data science, and leadership before sprint planning.
If the framework reveals that ML is not needed, you have just saved your team weeks or months of unnecessary work — and that is equally valuable.
Next step: Take your highest-priority proposed ML feature and run it through all six steps today. Document the output and review it with your engineering lead before your next roadmap meeting.
// FREQUENTLY ASKED QUESTIONS
How do I know if a SaaS feature needs ML or just better business rules?
Apply the framework's second step: ask whether every rule governing the feature can be written out exhaustively. If user behavior is predictable and the rule set is small, business rules work. If behavior varies widely, rules change frequently, or the pattern is buried in large datasets, ML is justified. The dividing line is whether explicit programming breaks down at the scale and variability you face.
What is the biggest risk of skipping ML evaluation as a product manager?
The biggest risk is committing engineering resources to an ML project that either does not need ML or lacks the data to support it. Both outcomes waste months of development time. The framework's explicit-programming check and learning signal verification prevent these two failure modes. Running six steps that take 30 minutes can save 3-6 months of misguided work.
Can non-technical PMs use this framework effectively?
Yes. The framework is designed to work without technical expertise. Step one requires plain language, not jargon. Pattern mapping uses everyday analogies like spam filtering and photo tagging. The learning signal check asks practical questions about data availability, not statistical concepts. The framework builds PM judgment about ML suitability without requiring them to understand algorithms.