DuckDB can ingest and preprocess data from sources such as cloud warehouses, object storage, Parquet, CSV, Delta Lake, and Iceberg without requiring a separate server.
2
DuckDB can store embeddings and combine vector search with full-text search for hybrid retrieval, using SQL.
3
DuckDB can run on a client or in the cloud through MotherDuck, allowing teams to move computation closer to data or users and scale when needed.
Summary
Mehdi Ouazza presents DuckDB as a practical component in AI and LLM systems, especially before and after model execution. He starts with DuckDB's embedded design: it runs inside a Python process, reads and writes many data sources, and can also run in a browser. MotherDuck adds cloud storage and compute while keeping the client and server connected. For AI workloads, Ouazza focuses on data preparation, since model projects often begin by pulling data from object storage or a cloud warehouse. DuckDB can also store embeddings, run vector search, and combine it with full-text search for hybrid retrieval. He describes SQL as a useful common language across data and ML work, including simple regression functions through extensions. The talk makes a measured case for using fewer separate tools when DuckDB already fits the workload.
DuckDB is an embedded analytical database that runs inside an application
Ouazza introduces DuckDB as a process analytical database that Python users can install with pip and run inside their existing process. It has interfaces for Java, Rust, and R, plus a command-line interface. There is no server to install for the embedded use case. DuckDB is packaged as a binary with support for reading and writing many sources, including Parquet, CSV, Delta Lake, and Iceberg. Its own file format stores metadata with the tables and supports ACID transactions. The same database can run in a browser through WebAssembly, including offline queries on shell.duckdb.org.
MotherDuck extends the same client-side model into the cloud
Ouazza describes MotherDuck as a serverless cloud layer for DuckDB. A MotherDuck extension connects a local DuckDB process to cloud storage and compute, and users can create and share datasets through the service. The important difference from a standard cloud data warehouse is where computation happens. A local DuckDB client can compute locally, while the cloud can handle work when scale is needed. In his demonstration, he runs a query over a three-gigabyte dataset locally, then attaches to a cloud database and runs the same query there. He presents the two locations as working together rather than as mutually exclusive choices.
DuckDB fits the data preparation stage of an ML or LLM project
Ouazza says AI projects often begin by fetching and preprocessing data from a cloud warehouse or object storage. DuckDB can pull those sources into one tool, write the result to its own file format, and run analytics and preparation operations there. He connects this to the quality of the final AI or ML system, saying that data quality matters throughout the project. The same approach can also be used later in the pipeline, close to inference. Because DuckDB can run on a client, teams can perform some analytics or ML work where the user or application is running instead of sending every operation to a remote service.
DuckDB can keep cleaned data and embeddings in the same place
Ouazza describes storage and search as parts of an LLM stack for retrieval-augmented generation and fine-tuning. After cleaning data during preprocessing, a team can use DuckDB to store embeddings in that same environment. He presents this as a way to avoid creating a separate system for every stage of the pipeline. DuckDB can act as a vector store, and its SQL-first design lets users work with the data through SQL even though the database also runs within Python. The argument is practical: if the data is already being transformed and stored in DuckDB, embeddings and related retrieval operations can live there too.
SQL enables vector and full-text hybrid search in DuckDB
Ouazza points to a DuckDB approach that combines embedding-based vector search with full-text search. DuckDB supports full-text search, and the two methods can be integrated into a hybrid search flow written in SQL. He describes SQL as a common language for data engineers and ML engineers, which can reduce the number of separate pieces in a pipeline. He also mentions an example from Hugging Face, which uses DuckDB's full-text search functionality behind its asset search. This gives his argument a concrete production example rather than leaving hybrid retrieval as a theoretical database feature.
DuckDB extensions can run simple ML operations as SQL functions
DuckDB supports extensions, and Ouazza says users can build custom ones in addition to the built-in extensions for formats such as Parquet and CSV. He gives the example of putting standard linear or logistic regression behind a SQL function. That lets a user run a simple ML workload in the same analytical database where data transformations and embeddings are stored. His broader point is that a pipeline with SQL in one place and Python in another can be harder to manage and debug. When an operation fits SQL, keeping it there can make the workflow easier to follow.
Client and cloud computation can change cost and latency choices
Ouazza presents DuckDB and MotherDuck as a database model that can run on the client and the server. A local machine can do computation instead of sending only SQL text across the network, while MotherDuck provides cloud compute when the workload needs to scale. He says this can reduce cloud costs and improve latency for users, especially when the local machine can handle the work. His demonstration is meant to show that moving from local DuckDB to cloud execution does not require a large operational change. The same interface can begin with local data and then attach to a cloud database.
"You can use it as a vector store, you can store your embeddings, and it's great because it's the same place where you did your preprocessing data in your data engineering to have your clean data set."08:03
Who should watch
You are building an ML or LLM pipeline and your source data is spread across object storage, cloud warehouses, and file formats.
You want to compare an embedded analytical database with separate vector-search and preprocessing tools.
Your team is considering whether some retrieval, embedding, or simple ML work can run in SQL, locally, or through a cloud extension.
