Salman Avestimehr says developers and enterprises want ownership, control, scalability, and privacy when they build their own generative AI applications.
2
FedML Nexus AI covers model deployment, serving, monitoring, fine-tuning, agent creation, multi-cloud GPU access, and on-premise operation.
3
Salman Avestimehr expects smaller foundation models, on-device agents, and federated learning to matter more as companies seek lower costs and better privacy.
Summary
Salman Avestimehr describes FedML as a platform for teams that want to build and operate generative AI applications without giving up control to a hosted model API. He sees ownership, scalability, observability, privacy, and infrastructure cost as recurring problems for developers, startups, and enterprises. FedML Nexus AI provides dedicated model endpoints, autoscaling, prediction logs, safety controls, zero-code fine-tuning, distributed training support, agents, and access to GPU providers across clouds. Salman explains the operating loop as deployment, monitoring, evaluation, data improvement, fine-tuning, and redeployment. He is also working on smaller foundation models that can run on affordable infrastructure and eventually on devices. The conversation connects this direction to federated learning, which trains models across distributed data without collecting it centrally. Salman is clear that federated learning has less demand for current foundation-model applications because compute is the larger constraint, while private data may become more important later.
Ownership is driving teams away from simple hosted APIs
Salman Avestimehr says developers and enterprises increasingly want to own their models, applications, data, and intellectual property. Some teams may start with OpenAI or another hosted model, while others cannot use an external API because of privacy or control requirements. Ownership also includes observability: teams need to see how people use the system, when answers are wrong, and what feedback should shape the next iteration. FedML addresses this with fully on-premise deployments inside an enterprise VPN and with dedicated endpoints for startups and developers.
Production generative AI needs more than a model endpoint
Salman Avestimehr says a commercial application needs infrastructure around the model. FedML can create dedicated endpoints, autoscale as demand changes, replicate serving across GPUs, and expose query, SLA, prediction, question, and answer logs. Those logs can later support fine-tuning. The platform also provides a safety layer that can analyze responses, filter sensitive questions, and reduce data leakage. Salman contrasts this with a quick model API combined with a retrieval system, which may be enough for an experiment but does not cover the operational depth of a large production service.
FedML follows an operating loop from deployment to improvement
The platform is designed around repeated model improvement. After deployment, teams collect interactions and monitor how the model performs. They can use the resulting data for zero-code fine-tuning by changing hyperparameters, then move to more advanced distributed training and machine learning operations. Salman describes this as a loop of deploying, observing, evaluating, preparing better responses, fine-tuning, and putting the updated model back into production. Human feedback can help repair weak question-and-answer pairs before they become training data.
Evaluation becomes harder when models give open-ended answers
Salman Avestimehr distinguishes established benchmarks for general-purpose language models from the thinner benchmark coverage for vertical models such as healthcare systems. Teams also need to measure hallucination, correctness, credibility, and privacy leakage. Correctness is difficult because language-model answers are dynamic and open-ended, unlike classification tasks with known classes such as cat or dog. He discusses research on meaning-aware response scoring, where tokens receive different weights. In the example, 'Paris' matters more than the surrounding words when judging an answer about France's capital.
Endpoint testing can combine playgrounds, routing, and user feedback
For a quick developer check, FedML provides a playground inside endpoint creation. Teams can ask questions before connecting the endpoint to an application. For comparisons, an API Gateway can route the same request to two endpoints, generate responses from both, and collect data for an A/B test. Customers can also alternate models for users and collect thumbs-up or thumbs-down feedback. Salman presents these features as a way to assess a new model before fully switching production traffic to it.
Different customers arrive with different infrastructure problems
Salman Avestimehr describes individual developers, startups, and enterprises as FedML users. Startups may have a strong proprietary model but lack the infrastructure needed to deploy, autoscale, monitor, secure, and serve it at a manageable cost. FedML gives them access to multi-cloud, multi-provider GPU capacity and lets them add their own instances. Enterprises often need a fully on-premise platform inside a VPN, especially for sensitive use cases such as an internal coding copilot. Individual developers may come for access to models, agents, experiment tracking, or federated learning.
Federated learning trains on distributed private data
Salman explains that FedML began with federated learning before expanding into generative AI. Federated learning trains a model across distributed data without collecting the data in one place. He gives next-word prediction on smartphones as an example, where a model can learn from user behavior without directly gathering that data. Salman says current foundation models are already trained on massive datasets, so compute and cloud access are the larger constraints for many generative AI teams today. He expects private data to become a stronger bottleneck later, which could increase demand for federated learning.
Smaller models are intended to make ownership and edge deployment practical
Salman says FedML is preparing its own foundation model, focused on smaller language models rather than the largest parameter counts. His argument is that a high-performing small model is easier to train, maintain, deploy, and improve on reasonable GPUs. Smaller models also fit on-device use cases. He imagines agents on phones that can call APIs, search websites, summarize information, write notes, and send emails. He expects simpler content creation and image editing tasks to move onto phones sooner, while agents handling sensitive actions will still need verification.
"The first challenge we are seeing is how can we help people want to build, deploy their own models, their own applications in this complex software stack."Salman Avestimehr04:47
Who should watch
You are building a generative AI product and need dedicated serving, autoscaling, monitoring, and fine-tuning without building a large infrastructure team.
Your enterprise needs to keep model workloads and proprietary data inside its own environment or VPN.
You are evaluating smaller models, on-device agents, or federated learning for privacy-sensitive applications.
