Nubank was built as a technology company, so machine learning, cloud infrastructure, mobile technology, design, and data science were part of its foundation rather than later additions.
2
Nubank standardizes model development with Fklearn, an open-source library, and an internal tool called Ship that handles model configuration, artifacts, APIs, training, and serving.
3
The team can change models quickly, but monitoring still requires people to inspect data and model behavior, while a feature store is being developed to support reusable, low-latency features.
Summary
Caique Lima and Cristiano Breuel describe how Nubank runs machine learning inside a technology-led financial services company. Machine learning supports credit decisions, fraud detection, and customer service. The team moved from generalist data scientists toward a structure with machine learning engineers and a data science productivity team. Fklearn standardizes model construction across common Python libraries, while the internal Ship tool gives data scientists a model configuration, project template, training workflow, and batch or real-time serving path. Nubank uses Kubeflow for notebooks and training pipelines, but serves models through a Clojure sidecar that calls a local Python service. The speakers are candid about unfinished work. Monitoring still needs substantial human review, feature lineage is difficult to track, and real-time serving requires fresh features from APIs. They are building a feature store and considering further automation for distributed training, hyperparameter tuning, infrastructure operations, and service-level objectives.
Nubank treats machine learning as part of its technology foundation
Cristiano Breuel describes Nubank as a technology company in financial services, rather than a traditional bank that later adopted technology. The company started in the cloud and treated cloud, mobile technology, design, customer experience, and data science as founding pillars. It has no physical branches, and its systems were designed around having data available for analysis. Machine learning has supported important decisions from the beginning, including credit-card acceptance, credit limits, customer-service routing, and fraud detection. Cristiano contrasts this with traditional companies, where data is often split across silos and technology may not receive early priority.
The team introduced machine learning engineering to make model work repeatable
Caique Lima says Nubank had generalist data scientists for four or five years before creating a machine learning engineer role about two years before the meetup. The earlier setup worked while there were fewer models, but it did not provide a structured way to scale. The team began building on the existing infrastructure and setting standards for how models are developed. Caique describes Fklearn, an internal library later open sourced by Nubank, as the first major MLOps component. It brings functional programming ideas into model construction and provides a consistent interface across libraries such as scikit-learn and XGBoost.
Fklearn and Ship separate modeling work from infrastructure work
Fklearn wraps common machine learning components and lets data scientists describe models as pipelines with different learners. Nubank uses the same package in production that it released as open source. The internal Ship tool handles the surrounding work. Data scientists provide a model configuration with items such as training data and parameters, then write the training function using Fklearn. Ship manages tasks such as saving artifacts, setting up an API, and serving a model in batch or real time. A command-line template creates a GitHub repository with the files a new model needs, reducing repeated setup and copy-and-paste implementations.
Nubank uses Kubeflow for development and training, but a Clojure sidecar for serving
Nubank uses Kubeflow to provide Jupyter notebooks and to train models through standard pipelines or user-designed pipelines. It does not use Kubeflow to serve models. Most of Nubank's services and infrastructure are written in Clojure, so the team avoids recreating all of that infrastructure in Python. Its serving pattern runs a Python model service on localhost and places a Clojure sidecar beside it. The sidecar connects the model to the rest of Nubank's infrastructure. This lets machine learning models use the same deployment and testing pipelines as other microservices.
Monitoring remains dependent on people even when deployment is fast
Caique says Nubank can put models into production and change them quickly because of the systems built over the preceding years. The harder problem is monitoring. During the coronavirus period, the team could observe changes in user behavior, but business analysts and data scientists still had to inspect the data and decide what those changes meant for models. Nubank already has substantial monitoring, yet people remain responsible for interpreting it and judging whether a model change is safe. The team is working toward a more standardized monitoring approach so that several groups do not maintain separate manual solutions.
A feature store is intended to make real-time features reusable
Cristiano says real-time model serving is one of Nubank's largest current problems. Models are trained from batch data produced by ETL, but production serving needs fresh feature values obtained through APIs. The team is building a feature store around streaming transformations so that the same transformations can support batch training and low-latency online serving. A central store could also prevent data scientists from recreating the same feature for different models. Caique adds that Nubank is researching existing approaches and adapting ideas from available solutions to its own architecture rather than designing the system without outside reference.
Feature lineage and personal-data governance are still incomplete
Nubank records model information in a model configuration and sends metadata to a service whenever a model reaches production. The metadata includes details such as which model went to production, when it was trained, and who trained it. For data governance, engineers can tag datasets and columns with personally identifiable information, and that information is propagated through the ETL process. The team responsible for data protection handles requirements such as Brazil's LGPD and rules in other countries. Caique says feature lineage can still require manual investigation. If a feature is removed from one dataset, the team may not know whether another model uses it elsewhere.
Automation should follow repeated work instead of eliminating flexibility early
Cristiano argues for starting with the fastest workable process and automating after repeated patterns appear or a task becomes too time-consuming. This avoids forcing data scientists into a rigid workflow before the team understands what needs standardization. Possible future work includes distributed training when models no longer fit on one machine, more consistent hyperparameter tuning, automated algorithm and architecture searches, infrastructure automation, and better service-level objectives. The data science productivity team plans to speak with data scientists about their roadblocks, then look for common problems that can be improved for everyone.
"If you had a centralized feature store you could enable that, like people just say, 'Oh, there's this feature that's already created.'"Cristiano Breuel39:25
Who should watch
You are building model tooling for a financial services company and need examples of how a team connects Python models to an existing Clojure service architecture.
Your data scientists repeat setup work, use inconsistent model patterns, or need a simpler path from training to batch and real-time serving.
You are deciding whether to build a feature store internally and want to hear how Nubank weighed integration and customization against buying an existing product.
