LLM features should start with an existing customer problem, because the customer, user persona, and product mission do not change when a new model becomes available.
2
Non-user-facing uses such as alert summarization, data labeling, and converting unstructured logs into structured features can add value inside an existing product.
3
The LUMUS framework asks teams to examine existing solutions, user impact, measurement, customer delight, and long-term sustainability before building an LLM feature.
Summary
Harini Kannan argues that companies adding LLMs to existing products should keep ordinary product discipline. The customer problem remains the starting point, while the model is another tool for adapting the solution. She separates LLM products into user-facing chat interfaces, user-facing systems for specific structured tasks, and non-user-facing layers inside a larger product. Her examples from cybersecurity focus on alert fatigue, missing labels, and unstructured log data. LLMs can summarize alerts, find context across sources, label events, and turn logs into structured data for traditional machine learning systems. Kannan proposes the LUMUS framework: list existing solutions, assess user impact, measure results, over-deliver on delight, and check whether the feature is sustainable in cost, infrastructure, and scale. In the question period, she recommends safeguards around model outputs, including additional models, confidence scores, thresholds, testing, and feedback loops. She also argues for shipping useful features quickly enough to learn from users, while checking cost and product fit first.
Customer problems stay fixed while the solution changes
Kannan says teams should remain grounded in good product principles because the problems customers face have not changed. She warns against beginning with a list of existing projects and asking where an LLM can be inserted. The starting point should be the customer problem, the product mission, and the user persona. An LLM is another tool in the toolbox, alongside product judgment, data, and machine learning foundations. This framing keeps teams from treating the arrival of a new model as a reason to abandon the product strategy they already need to follow.
Kannan groups LLM applications into three broad categories. The first is a user-facing chat interface, such as a customer service assistant or an expert system that answers questions about a complex topic. The second is a user-facing system tuned for a specific task, such as translating natural language into a query language, domain-specific structured output, or shell commands in cybersecurity. The third is a non-user-facing layer inside a larger product. It can engineer features, improve labels, or turn unstructured data into tabular features for traditional machine learning models.
Non-user-facing layers can address expensive cybersecurity work
Kannan uses cybersecurity to show where an LLM can add value without becoming the product interface. Security teams can receive hundreds of thousands of alerts per day, creating alert fatigue that has remained a problem for years. An LLM can summarize alerts, add structure to logs, improve explainability, and find patterns and context across alerts from different vendors. It can also help label malicious or attack-related data when teams lack enough labels. A further use case is converting large amounts of unstructured log data into tabular data that can feed traditional machine learning systems.
The value-versus-complexity model changes when LLMs reduce manual work
Kannan applies the familiar value-versus-complexity framework to LLM features. Before LLMs, teams generally started with work that had high value and low effort, while placing higher-effort projects in the backlog. LLMs can make some previously expensive work more practical, especially tasks that require people to convert unstructured text into structured features or label large datasets. She does not suggest skipping product prioritization. The point is that the effort estimate for certain forms of data preparation can change when an LLM is available.
LLMs can act as delight features, but delight becomes expected
Using the Kano model, Kannan describes basic features that customers expect, performance features that increase satisfaction as they improve, and delighter features that customers did not expect but appreciate strongly. She suggests that LLM capabilities can create this kind of delight by making complex technology accessible or by adding an unexpected capability to an existing product. In the discussion, she adds that a feature that initially delights users can eventually become a basic expectation. Teams therefore need to judge both the immediate customer benefit and how the feature fits the product over time.
The LUMUS framework tests whether an LLM feature is worth building
Kannan proposes the LUMUS framework as a checklist before starting an LLM feature. 'List Solutions' means identifying existing solutions and deciding where an LLM fits. 'User Impact' asks how directly users will be affected. 'Measure' asks whether the team can track metrics, market feedback, and model or user experience changes. 'Over Deliver' focuses on using the feature to create customer delight. 'Sustainable' asks whether the cost, infrastructure, and scaling demands work for the full pipeline. The framework is meant to help teams decide whether to proceed rather than assuming every project needs an LLM.
Production use depends on the value of solving data and label shortages
In response to a question about whether LLM-based feature engineering is suitable for production, Kannan says teams should move from less complex approaches toward more complex ones. She still sees production value in cybersecurity because the volume of data is very large and missing labels can delay projects. Without enough labels, teams may also struggle to validate their models and obtain useful metrics. Her argument is conditional: the return should justify the cost and effort, and the LLM should be evaluated against other ways to solve the same problem.
Model outputs need checks before they influence production decisions
Kannan sees more room for controls when an LLM is a layer inside a machine learning pipeline than when it directly handles an open-ended chat interaction. For labeling, she would not initially send the LLM output straight into a production model. She suggests adding another model or checking layer, producing a confidence score, and accepting only outputs above a chosen threshold. Teams should review false positives and false negatives through feedback, then tune the system. She is clear that these controls do not provide a complete guarantee, but they create a way to compare and improve the final result.
