11 lessons learned from doing deployments

Sol Rashidi, ExecutiveAI LLC35:59 · Oct 2024 · 275 views
Thumbnail for 11 lessons learned from doing deployments Watch on YouTube
TL;DR
  1. 1

    Sol Rashidi says only 30% of her AI deployment hurdles were technical, while 70% came from resistance, weak alignment, talent gaps, unclear ownership, budgets, and other organizational problems.

  2. 2

    AI strategies need to match an organization's data, technical, and organizational maturity. Rashidi says teams should think big, start with a small use case, and scale only after they have built the necessary capabilities.

  3. 3

    Use cases should be ranked by criticality and deployment complexity rather than business value alone, because business-value estimates often contain subjective or unsupported assumptions.

Summary

Sol Rashidi draws on more than 200 POCs and dozens of production products to explain why AI deployments fail after the technology appears promising. Her central claim is that technical work accounts for about 30% of the difficulty. The larger share comes from organizational friction, including fear, weak executive alignment, unclear ownership, poor communication, missing skills, incomplete budgets, and resistance to change. She argues that strategy must fit the organization's actual maturity, especially its data readiness. Teams should clarify why they are deploying AI, then choose a small use case that matches that reason. Rashidi's preferred selection method scores criticality and deployment complexity. It considers competitive threats, regulation, data quality, available people, stakeholder commitment, dependencies, and infrastructure. She also explains why different project phases need different specialists and personalities. The talk is direct about the work behind production systems and the assumptions that make ambitious plans look easier than they are.

Key ideas
03:13

AI adoption is still far behind the amount of discussion around it

Rashidi says only about 40% of the ecosystem has even started doing something with AI, and that figure includes board discussions, use-case exploration, and bringing in a consulting firm. It does not mean that organizations have built POCs or moved systems into production. She contrasts exploration with deployment and says the gap is large, including in the United States despite the amount of hype and conference activity. Her point is that public enthusiasm should not be confused with operational adoption. Many industries are discussing AI, but discussion alone does not create a working product or a production process.

10:20

Most deployment problems come from people and organizations rather than technology

After more than 200 POCs and 39 enterprise products in production, Rashidi says only 30% of her hurdles were technical. Those technical issues included infrastructure, workflow pods, model training, data, and security. The remaining 70% involved fear of job loss, immature workforces, missing talent, poor judgment about consultants, and teams chasing new technology without knowing how to deploy it. She also points to the speed of change, repeated waves of new terminology, and organizational inertia. Companies may say they want innovation while withholding the resources needed to support it.

13:27

An AI strategy has to match the organization's actual maturity

Rashidi says executives often approve an ambitious AI strategy without checking whether practitioners have the data, infrastructure, or organizational support to execute it. She estimates that 75% of proposed use cases may not be doable because the organization lacks technical, data, or organizational maturity. Her first check is the data strategy: what is already in place, what is in flight, and what the organization hopes to build. The AI strategy needs to be mapped against those facts. A polished presentation does not make an aspirational use case deployable.

15:02

The reason for deploying AI should determine the size and type of use case

Rashidi asks whether a project exists because the board requested it, because it solves a real business problem, because of a competitive threat, because the company wants to appear innovative, or because of FOMO. A board-mandated project should use a simple case because the organization may not support a major investment. A real problem or competitive threat can justify more effort. Her advice is to think big, start very small, and decide whether to scale after learning from the first deployment. The use case has to match the motivation behind the project.

16:18

Reframing AI can make its value easier for executives to understand

Rashidi dislikes treating intelligence as something artificial and offers three interpretations of the A in AI. Automated intelligence handles mundane tasks. Augmented intelligence helps people do more, such as helping customer service staff search product information across a large catalog. Anticipatory intelligence uses surrounding data to find patterns that may support business decisions beyond descriptive analytics or linear regression. She uses these terms to set more practical expectations and connect the technology to the work people already understand.

17:54

Technical teams lose support when they speak in their own abbreviations

Rashidi reads a dense marketing sentence full of terms such as CRM, CMS, ABM, CLV, CAC, AB testing, CTA, UX, CRO, and ROI. She says this is how customers and executives hear technical or specialist language when they do not share the speaker's vocabulary. People who control budgets may not use the same terms as engineers or AI teams. Teams therefore need to explain benefits, capabilities, and business value in the listener's language. Communication is part of deployment work because a technically sound project can still lose support if decision-makers cannot understand it.

20:21

Use-case selection should score criticality and deployment complexity

Rashidi does not use projected business value as the primary way to select a use case. She says those estimates often contain assumptions that differ across manufacturing, supply chain, procurement, and brand teams, and can turn into emotionally charged comparisons. Her framework scores criticality, including competitive threats, market consolidation, regulation, fines, exposure, and press. It also scores complexity, including stakeholder commitment, available people, data access and integrity, dependencies on other functions, and infrastructure. Low-criticality, high-complexity cases are non-starters. High-criticality, lower-complexity cases are good candidates, while high-criticality, high-complexity work needs preparation.

25:48

Production budgets and data quality need to be considered before the POC

Rashidi says organizations often approve money for a POC without funding the work required to operate the product in production. She therefore makes teams estimate the cost of exploration, the POC, and production before starting. She also recommends choosing data that is at least 'goodish' rather than waiting for perfect data. Examples include raw materials, bills of materials, 10-Ks, contracts, event logs, opt-in and opt-out data, and e-commerce data, especially where poor quality would create fines or bad public relations. Better data selection raises the chance of a successful deployment.

30:23

Different phases need different people and different kinds of expertise

In the question period, Rashidi explains that generalists and specialists are useful at different points in a project. A skeptical subject-matter expert may be unhelpful during ideation but valuable when a team moves from scoping to solution design because that person can expose weak assumptions and blind spots. DevOps should not be pulled into ideation, and product staff should not be assigned to strategy without a reason. The executive stakeholder group can stay consistent, while the people doing the work change by phase. She also argues that projects need a clear lead, since committees often fail to maintain momentum.

"The stakeholder will stay consistent. It's the talent that you need to unlock and complete each phase in a healthy way so you can move forward in the next phase."32:41
Who should watch
  • You are responsible for moving AI projects from POC into production and keep discovering that the hardest problems are outside the model.
  • Your organization has an AI strategy, but its data readiness, ownership, staffing, or production budget is unclear.
  • You need a practical way to compare use cases when projected ROI numbers are based on competing assumptions.