Meetup

Deep Dive on Paperspace Tooling

Misha Kutsovsky, PaperspaceEpisode 21 · 1:07:15 · Jul 2020 · 263 viewsHosted by Demetrios Brinkmann
Thumbnail for Deep Dive on Paperspace Tooling Watch on YouTube
TL;DR
  1. 1

    Paperspace grew from GPU-backed virtual machines into Gradient, an MLOps platform for notebooks, experiments, training jobs, and model deployments.

  2. 2

    Gradient tracks code, containers, metadata, artifacts, metrics, and model lineage when work moves from local or notebook development into managed jobs and endpoints.

  3. 3

    Misha recommends defining the path to production early, establishing a baseline first, and avoiding over-optimizing infrastructure or tool choices before the business problem is understood.

Summary

Misha Kutsovsky describes how Paperspace expanded from GPU-backed virtual machines into Gradient, a platform for developing, scaling, and deploying machine learning systems. The product combines notebook workspaces, persistent storage, container snapshots, managed jobs, experiment tracking, model repositories, and inference endpoints. Users can run workloads on Paperspace or in their own infrastructure, with Kubernetes underneath and support for different machine types, GPUs, autoscaling, and multiple processing sites. Gradient also exposes a Python SDK and command-line interface, allowing teams to launch jobs, stream logs, inspect metrics, and connect training to source-control workflows. Misha explains how pull requests can trigger training and validation, with model metrics deciding whether a change should proceed. He argues that teams should make deployment possible early, track the full lineage of production models, and avoid forcing machine learning engineers to manage low-level infrastructure. He expects more of this infrastructure work to become specialized and abstracted away.

Key ideas
01:26

Paperspace expanded from GPU virtual machines into an MLOps platform

Misha explains that Paperspace began with GPU-backed virtual machines for visual effects, gaming, and other workloads. Machine learning users needed more than a VM because their work involved experiments, models, programs, deployments, and scaling. Paperspace added a job runner and a community product built around easy notebook and JupyterLab environments. Its enterprise work then moved toward running training at scale, deploying machine learning applications, and supporting infrastructure where the data already lives. The goal is to help teams move from research code to a working business application without making them assemble every operational layer themselves.

07:43

Gradient combines simple workflows with access to the underlying infrastructure

Misha says the team wants users to do whatever they need while making the recommended path easy. Gradient runs on Kubernetes and Docker, and advanced users can open an SSH server or work directly with containers. The product team watches how customers use the platform, then turns repeated patterns into first-class components. Examples include distributed training, TensorFlow Serving endpoints, and automatic capture and versioning of models. Machine learning engineers can focus on their code and application, while administrators configure clusters, machine types, autoscaling limits, and policies for resource use.

11:52

The platform separates centralized services from workload execution

Gradient has a central hub for services, APIs, project navigation, and the user interface. Workloads can execute at different processing sites, including Paperspace infrastructure, a customer's own environment, or public clouds such as AWS. Misha says this supports portability across regions and infrastructure. A team may need data to remain in Germany for compliance, or may want to move work between an internal environment and a public cloud. The same project experience can manage those workloads while execution happens where the required compute and data are located.

14:44

Managed jobs move training out of the notebook and into tracked execution

Misha warns against doing production training directly inside a notebook. Gradient lets a user submit a local directory, source-control commit, machine type, and resource count as a managed job or experiment. The platform orchestrates the workload and provides live logs, metrics, and metadata. It also tracks machine-learning-specific information such as hyperparameters, artifacts, and models. A model can then move into an inference service, where it can be updated without downtime. The workflow covers experimentation, larger-scale training, and serving while allowing users to bring their own containers and source repositories.

17:03

Source control and model metrics can become part of the development workflow

Gradient can connect model development to pull requests. A change entering source control can trigger one or more experiments, and the results can be used to decide whether the new model is better than the baseline. Misha describes comparing metrics such as accuracy or F1 score, then approving or rejecting the change through the CI workflow. For checkpoint-based models, the system can aggregate metric values such as minimum, median, maximum, and standard deviation. Teams can set numerical thresholds, or have the training code produce the values used for validation. This gives model quality a place in the same workflow as code changes.

36:27

Notebook exploration remains flexible, while launched work becomes reproducible

Misha draws a boundary between experimentation in a notebook and work launched as a real experiment or deployment. Users can install packages and try things interactively, then snapshot or fork a workspace. When they launch a managed experiment, Gradient requires enough environment information to reproduce it, such as a requirements file or Dockerfile. Jobs receive unique identifiers and version histories, and their inputs and outputs can be queried later. The system builds on existing pieces such as Git protocols, object storage APIs, and Kubernetes instead of asking users to learn a separate mechanism for every part of the workflow.

48:07

Inference endpoints support custom models, containers, scaling, and lineage

Gradient connects deployed endpoints to the model, experiment, and code that produced them. Misha describes one-click paths for model formats such as ONNX and TensorFlow SavedModel, along with support for standard serving containers and custom applications. Endpoints can expose REST APIs, use CPU or GPU machines, share GPU resources when appropriate, and scale based on runtime signals. Users can update models without downtime and configure replica counts. The platform also supports observing logs, metrics, CPU, GPU, and memory usage while a deployment runs. Teams can load-test an endpoint or send shadow traffic before making a production decision.

55:54

Misha expects machine learning infrastructure to become more abstracted

Misha expects data scientists and machine learning engineers to spend less time managing Kubernetes, Docker, and fragmented infrastructure tools. He says teams currently need to know a little about many layers, while future platforms should provide better abstractions and allow workloads to run across different clouds based on requirements. He also expects more automation for hyperparameter search, early stopping, and feature engineering. His advice is to decide how a project will reach production before investing heavily in the notebook, establish a healthy baseline, and avoid spending weeks over-optimizing a platform before solving the actual business problem.

"The second you basically make a request that says, okay, I want to run this as a machine learning experiment, everything is starting to get tracked and versioned."Misha Kutsovsky37:19
Who should watch
  • Your team is deciding whether to build its own machine learning platform or adopt managed tooling for notebooks, training, and deployment.
  • You need a workflow that connects source control, model validation, experiment tracking, and production endpoints.
  • You are supporting both machine learning users and infrastructure administrators, and want users to choose resources without exposing every cluster detail.