SurveyMonkey built its own ML platform because no single product covered its data, serving, and model management requirements together.
2
The feature store provides both bulk, point-in-time training data and fast real-time feature reads, which reduces training-serving skew.
3
The platform combines a model service gateway, synchronous and asynchronous inference, feedback pipelines, monitoring, and reproducible retraining.
Summary
Shubhi Jain explains how SurveyMonkey moved from separate microservices for each ML use case to a shared ML platform. The team first defined its requirements across data, serving, and management, then chose to build because available products would not fit its architecture or support needs. Its feature store gives data scientists flat, entity-based tables with daily versions for training, while a streaming path provides matching real-time values for inference. A service gateway routes requests to model versions and retrieves features automatically, with asynchronous processing for heavier jobs. SurveyMonkey monitors service health, input data, predictions, and eventual true values. Jain also describes feedback pipelines, selective auto-retraining, human tests of the development flow, PII controls, and the close collaboration between data scientists and ML engineers. He is candid that scoping the platform and learning unfamiliar tools were major obstacles.
SurveyMonkey identified ML-specific problems that ordinary microservices did not solve
When Shubhi Jain joined SurveyMonkey, the company had a microservice for each ML use case. Teams developed a model and then wrapped it in a service for production. That process worked for basic deployment, but ML systems also needed model retraining, feedback pipelines, data from several sources, inference logging, A/B testing, and model version management. SurveyMonkey decided around April 2018 that these requirements called for a shared platform. The existing software engineering processes made microservices possible, but they did not address the full data and model lifecycle.
The build-versus-buy decision came from writing down the platform requirements
The team held a multi-day brainstorming session and described an ideal platform before choosing an implementation path. They grouped the requirements into data, serving, and management. Data scientists needed to find data easily, reduce training-serving skew, and receive live feature updates. Serving needed real-time and batch feedback pipelines, complete inference records, and support for experiments. Management needed versioning for models, projects, and incoming data. Shubhi says no single product covered all of this, and buying a large platform could lock SurveyMonkey into capabilities that did not fit its architecture. The company had enough internal software expertise to build the pieces itself.
The platform increased the pace of ML use-case delivery while keeping maintenance low
The initial effort involved roughly four or five data scientists and five or six ML engineers. Data scientists mainly developed models, while ML engineers mainly developed the production services, with overlap between the groups. The team later grew only by one or two people in each area, while its output increased. Shubhi describes a change from taking six to eight months to move a use case from idea to production to taking about a month. The platform team tries to automate routine upkeep. Maintenance takes about five to ten percent of engineers' and data scientists' time, while the remaining platform work adds capabilities through quarterly goals and two-week sprints.
The feature store separates bulk training reads from low-latency serving reads
Before the feature store, data scientists had to search hundreds of warehouse tables, understand their quality checks, build training sets, and map warehouse fields to existing serving services. This process could take months and often produced training-serving skew. SurveyMonkey created flat tables for each entity, with one entity ID per day and the features associated with that entity. The design also supports point-in-time views, such as knowing what a user's data looked like on a particular date. Bulk training tables use EMR for processing, AWS Athena for querying, and S3 for storage. A separate streaming architecture updates a matching real-time store from application logs, allowing single-line reads with latency of a few milliseconds.
A single service gateway hides feature retrieval and model routing from partner services
SurveyMonkey's ML services gateway is the common entry point for model calls. A partner service can provide an identifier such as a user ID or survey ID instead of collecting every raw feature itself. The gateway determines which features the requested model needs, reads them from the real-time feature store, and routes the request to model project pods. An NGINX routing layer selects the model and version based on the request parameters, including A/B testing choices. The same architecture supports synchronous inference for interactive features and asynchronous inference for heavy jobs. SurveyMonkey used the asynchronous path for analyses such as calculating sentiment across a large survey.
Monitoring covers service health, incoming data, and model outcomes
Jain describes three monitoring areas. Service monitoring checks the ML service like any other production service, using New Relic across the organization. Input monitoring records the data entering models through application logs and other streams. Prediction monitoring stores every inference in a queryable prediction table, which supports checks for feature drift, prediction distributions, and A/B test results. When true outcomes become available, feedback endpoints or feedback pipelines connect them to the original inputs and predictions. Teams can then review model performance in daily or weekly views, receive updates in Slack, and trigger retraining when performance falls below a chosen threshold.
Reproducibility matters even when automatic retraining is selective
SurveyMonkey does not automatically retrain every model. The team enables automatic retraining for use cases where data drift or changing performance makes it useful. Every model can still be retrained by clicking a button. Training sets are versioned from the data scientist's selected timestamp, and the exact data used for a model version is stored in S3. That lets the team recreate a model if an artifact is corrupted or lost. Jain also connects this process to staff changes, since a new engineer should be able to understand how an artifact was produced and obtain the same result.
Data scientists and ML engineers work together from the start
SurveyMonkey moved away from handing a finished model artifact from a data scientist to an ML engineer. The two roles now work together from the beginning. This lets data scientists learn more software development and lets ML engineers understand the model in detail. The team can identify constraints early, such as a feature that is updated only once a day or an infrastructure limit on model complexity. Data scientists still lead model development, while ML engineers lead productionization, but the boundary has more overlap than it did in the past. Jain says this reduces blockers that previously appeared late in the process.
"I think at the end of the day it was really the scoping, figuring out what can we take on, what can we do, is this possible, what do we really need."Shubhi Jain54:34
Who should watch
You are deciding whether to build an internal ML platform or adopt a large external product, and need a way to compare requirements with integration and support costs.
Your team is dealing with training-serving skew, slow feature discovery, or separate paths for batch training and real-time inference.
You want practical guidance on connecting model development, deployment, monitoring, feedback, versioning, and retraining across data science and ML engineering.
