# Tour of Upcoming Features on the Hugging Face Model Hub Julien Chaumond, Hugging Face | MLOps Coffee Sessions | Episode 48 | 52:05 Source: https://www.youtube.com/watch?v=03uallDwq6o Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks. Page: https://mlopstalks.com/talks/tour-of-upcoming-features-on-the-hugging-face-model-hub Published: 2021-07-27 Tags: deployment, model-serving, open-source, platform-teams ## TL;DR - Hugging Face grew from open-source NLP experiments into a platform for sharing, discovering, training, and deploying models and data sets. - The Model Hub uses Git and Git LFS so users can version model weights, keep revision history, and move their repositories without platform lock-in. - Hugging Face runs inference widgets and customer endpoints on Kubernetes, with support for cloud, GPU, CPU, and on-premises deployments. ## Summary Julien Chaumond describes how Hugging Face developed from a small NLP company into an open-source community and model platform. The company began by releasing practical projects, including a co-reference resolution system, then gained strong adoption for its PyTorch implementation of BERT. Users soon wanted to share model checkpoints, which led to the Model Hub. Julien explains why the Hub treats models and data sets as Git repositories, with Git LFS storing large files such as model weights and S3 providing the storage backend. The platform also uses metadata in each repository to deploy inference endpoints and power free model widgets. Customers can receive optimized hosted endpoints or run containers on their own Kubernetes infrastructure. Julien is direct about the company's product decisions being incremental and community-driven. He also discusses the Hugging Face course, TensorBoard integration, hardware choices, inference optimization, and his belief that team autonomy and community involvement will determine how well the project scales. ## Key ideas ### Hugging Face began by making advanced NLP models easier to use [04:34](https://www.youtube.com/watch?v=03uallDwq6o&t=274s) Julien says Hugging Face started with open-source software that put a simple interface around models built by leading companies and universities. The first major adoption came after the company released a PyTorch implementation of the original BERT code. The community response encouraged the team to keep building accessible tools. Transformers later grew beyond NLP toward other machine learning tasks, based on what users were building and contributing. Julien describes the broader aim as bringing people in AI together to build machine learning tools as a community. ### The company grew through experiments rather than a fixed product plan [07:41](https://www.youtube.com/watch?v=03uallDwq6o&t=461s) Hugging Face began in 2016 with Julien and his co-founders, Clément and Thomas. They wanted to explore modern NLP and machine learning, after having worked with older approaches earlier in their careers. One early project was a co-reference resolution system that linked pronouns to the subjects of sentences. The team built it on spaCy and made it faster than the open-source systems available at the time. Julien says releasing that project was not the result of a formal open-source strategy. Sharing useful work was already normal for the founders, so they continued doing it at Hugging Face. ### Users turned the model library into a platform for sharing checkpoints [12:15](https://www.youtube.com/watch?v=03uallDwq6o&t=735s) After the first version of Transformers, users began proposing new models and asking to share model checkpoints, including the weights rather than only the code. The team quickly hacked together an upload system using pre-signed S3 URLs. Julien admits that the early setup was messy, with conflicting files in storage, but it proved that users and organizations wanted to publish models. He connects this demand to the growing use of transfer learning, where teams start with a model trained on large data sets and fine-tune it for a specific task. The Hub developed around that workflow. ### Git gives models and data sets version history without locking users in [17:38](https://www.youtube.com/watch?v=03uallDwq6o&t=1058s) Hugging Face chose Git as the backbone for model and data set versioning. Julien says every item is exposed as a Git repository, so users can clone a model, retain its snapshots and revision history, and move it elsewhere if needed. Git LFS handles large binary files such as model weights by storing their hashes in Git and uploading the contents separately. The storage backend is S3, with CloudFront caching files at edge locations. This approach reuses tools that software engineers already understand instead of creating a proprietary versioning system. ### Repository metadata lets the Hub deploy model inference automatically [28:42](https://www.youtube.com/watch?v=03uallDwq6o&t=1722s) The inference API behind the model widgets uses metadata in each model repository to determine how to run the model. Transformers models include configuration such as the model type and number of layers. Other open-source libraries can provide their own configuration or metadata files. Hugging Face uses that information to deploy inference endpoints on demand. Popular models can already be loaded, while less-used models may take several seconds to load after a request. Julien says the system uses custom containers and images, selected and optimized for the model, library, and target hardware, with Kubernetes handling deployment. ### Hugging Face supports hosted inference and customer-controlled deployments [33:01](https://www.youtube.com/watch?v=03uallDwq6o&t=1981s) The public inference widgets are free to use on the Model Hub. Companies can ask Hugging Face to optimize endpoints for latency or high request volume, and the company can also ship a container for on-premises deployment. Julien names network latency, privacy, and security as reasons a customer may need to run the API on its own hardware. He says many companies already have an on-premises Kubernetes cluster, so deployment can involve specifying the required hardware and running the provided endpoint. The same model-serving approach can therefore work in Hugging Face's environment or inside a customer's infrastructure. ### Inference performance depends heavily on hardware and optimization [35:08](https://www.youtube.com/watch?v=03uallDwq6o&t=2108s) Julien distinguishes the variety of hardware used for training from the more common CPU and GPU choices for inference. He mentions TPUs, CPUs, GPUs, and Cerebras wafer-scale chips. For inference, CPU servers can be much cheaper and more energy efficient when the software is optimized well. Julien gives BERT as an example: an ordinary web server might need roughly 50 to 100 milliseconds per inference, while extensive optimization can produce several million requests per hour with sub-one-millisecond response times for some customers. The talk does not reduce deployment to a single preferred hardware type. ### Community feedback also shapes training and observability features [48:48](https://www.youtube.com/watch?v=03uallDwq6o&t=2928s) Julien says Hugging Face noticed users were uploading TensorBoard traces alongside their models. The team responded by integrating a TensorBoard server into the Hugging Face website. Users can now upload training traces and view a dashboard with the model, rather than storing only the final artifact. Julien presents this as an example of the company's incremental product process, where features emerge from observing how the community uses the platform. He also points people to the recently released Hugging Face course, which teaches NLP, machine learning, the Hub, Git, and Git LFS. ## Notable quotes - "We have this vision of building a community where the greatest people in AI can come together and basically invent the future of machine learning together." (04:30) - "The goal was to explore at the beginning." (11:51) - "Everything is exposed as a Git repo." (21:45) - "The team that you assemble is everything." (40:50) ## Tools & references mentioned - Hugging Face - Transformers - BERT - PyTorch - spaCy - Git - Git LFS - Amazon S3 - Amazon CloudFront - Kubernetes - TensorBoard - Cerebras ## Who should watch - You are building a model platform and need a practical approach to storing model weights, data sets, and revision history. - Your team wants to expose model inference through hosted widgets or customer-controlled Kubernetes deployments. - You are deciding whether transfer learning, model metadata, and training artifacts should be part of your machine learning workflow. ## Related talks - [Hugging Face Cofounder on AI Agents, LLMs and Open Source](https://mlopstalks.com/talks/hugging-face-cofounder-on-ai-agents-llms-and-open-source) (Thomas Wolf, Hugging Face & Euro Beinat, Prosus Group, 27:31) - [Building an ML Platform from Scratch: Live Coding Session](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session) (Alon Gubkin, Aporia, 1:57:24) - [How to Leverage ML Tooling Ecosystem](https://mlopstalks.com/talks/how-to-leverage-ml-tooling-ecosystem) (Mariya Davydova, Neu.ro, 55:57) - [AI Innovations: The Power of Feature Platforms](https://mlopstalks.com/talks/ai-innovations-the-power-of-feature-platforms) (Mahesh Murag, Tecton & Jose Navarro, Cleo & Nikhil Garg, Fennel, 1:05:28) - [The Future of Feature Stores and Platforms](https://mlopstalks.com/talks/the-future-of-feature-stores-and-platforms) (Mike Del Balso, Tecton & Josh Wills, Angel Investor, 1:11:15)