The group chose a lightweight, vendor-run compliance test as the first implementation for the SGT model deployment standard.
2
A fully automated CI system remains a longer-term option because it would require infrastructure, maintenance, deployment conventions, and more resources.
3
The group planned a public documentation site covering model deployment and possible standards for monitoring, metadata, framework logging, and feature store serving.
Summary
The Social Good Tech model deployment working group compared two ways to test compliance with NVIDIA's v2 inference specification. Proposal one would provide protobufs, test models, training code, and a Python test script. Each vendor would deploy the model, run the tests, upload the results to a repository, and receive a GitHub badge. Proposal two would automate deployment and testing through a CI system maintained by SGT and vendors. The group considered it too expensive and complicated for the first step, partly because model servers use different loading mechanisms. The discussion then moved to a documentation site containing the rationale for each standard, the specification, compliance tests, and ways to contribute. Possible future areas included model monitoring, model metadata, framework logging, and feature store serving. The meeting ended with plans to define initial models and tests, start a repository, collect feedback asynchronously, and move meetings to a biweekly schedule.
The first proposal keeps compliance testing outside a central CI system
The group proposed that SGT create the test specification, test models, protobufs, and a Python script. Vendors or open-source projects would load the supplied models onto their own model server endpoint and run the assertions. They would record their setup instructions and upload the results to a certification repository. SGT could then use those results to create a GitHub badge for each specification. The approach leaves deployment with the vendor and gives the community a common test process without requiring SGT to operate every environment.
The initial certification process depends on vendor self-assessment
The first design assumes an honor system. Vendors would report their own test results, with the expectation that community members might independently verify them while the number of participating organizations remains small. The proposal also avoids prescribing one model artifact format. The repository would include training code so vendors could produce another supported format when the supplied binary format does not work with their serving layer. This gives the test a fixed starting point while leaving room for different frameworks and artifact formats.
A fully automated CI system is a longer-term option with many operational costs
The second proposal would have SGT maintain a CI system that pulled a release, deployed it, loaded models, ran health checks and test requests, and executed the full test suite. Vendors would need to provide a repeatable deployment environment, such as a Docker container or Helm chart, along with authentication and other conventions. Participants said this would require infrastructure, engineers, and ongoing maintenance. They also pointed out that model servers do not share a standard loading mechanism, so even loading a model from an image could be difficult.
The group selected the lightweight proposal as the practical starting point
Participants described proposal two as a superset of proposal one and agreed that the group should begin with proposal one. The automated system could remain a north-star idea for later. The first proposal already looked stricter than the conventions used by some widely adopted standards because it would include a shared test suite. The group wanted to see how vendors and practitioners responded before committing to the infrastructure required by automated testing.
The documentation site would connect deployment with other MLOps standards
The proposed site would let an ML engineer read the rationale and history of a standard, view its core specification, find compliance testing information, and learn how to participate. The broader outline included model monitoring, model metadata, framework logging, and feature store serving. The site would be open source, hosted from GitHub, and deployed through a lightweight continuous delivery setup to static storage. The goal was to move ideas out of scattered Google Docs and meetings into a place where people could read and contribute asynchronously.
SGT planned to use open outlines to test community interest
The proposed next step for the other standards was to publish blank or partial requests for proposals. Contributors could define what belongs in scope without pretending that a complete specification already exists. The organizer described conversations with Allegro AI and Grid AI about possible work, including framework logging. The plan was to publish the outlines, discuss them through podcasts and conference settings, contact interested enterprises, and see which areas attract enough participation to move forward.
The first model tests should avoid unnecessary dependence on one framework
The group struggled to identify a model format that worked across the target model deployment servers. PyTorch and XGBoost were discussed, while scikit-learn was considered less suitable for one server. Another suggestion was to begin with health endpoint checks that did not depend on a model. Participants also considered a very small mathematical operation or a one-layer neural network so the same behavior could be reproduced across frameworks. They agreed that the initial choice could change later, but they still needed a concrete starting point.
The immediate work was to define tests, models, and documentation asynchronously
After the meeting moved to a biweekly cadence, participants were asked to comment on the testing tool documents. The next technical step was to agree on supported models and the overall test design, then specify individual tests and implement an initial repository. The group also wanted to add candidate models and model types to the shared notes. Documentation had no firm deadline, although the organizer hoped to publish an initial site within about a month if someone had the bandwidth to build it.
"Proposal one aims to be more lightweight and it's essentially centered around testing the specification and having manual work on the vendor side to deploy the model."11:34
Who should watch
You are helping define a model serving standard and need to choose between vendor-run tests and centrally managed CI.
Your serving platform has different model loading or deployment conventions from other vendors, and you need to understand how the group handled that problem.
You want to contribute to an early-stage MLOps specification and need a concrete view of the proposed tests, documentation, and community process.
