# Let's Talk About Raw Documents

Crag Wolfe, Unstructured.io | MLOps Meetup | Episode 120 | 49:56
Hosted by Ben Epstein

Source: https://www.youtube.com/watch?v=MBKFdy_URpg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/lets-talk-about-raw-documents
Published: 2023-03-03
Tags: data-engineering, multimodal, open-source

## TL;DR
- Unstructured.io reduces raw-document preprocessing to partitioning, cleaning, and staging steps that can feed downstream ML applications.
- Pipeline notebooks define domain-specific preprocessing APIs that Unstructured tooling can turn into local web APIs.
- The library can process documents directly, while related examples use the extracted content for labeling, sentiment analysis, and summarization.

## Summary
Crag Wolfe presents Unstructured.io as an open-source library for turning raw documents into structured data for downstream machine learning work. The initial focus is on PDFs, Word documents, PowerPoint files, emails, HTML, XML, text, and images. He divides preprocessing into partitioning, cleaning, and staging. Partitioning identifies elements such as titles, narrative text, columns, and bounding boxes. Cleaning removes items such as headers, footers, page numbers, and unwanted whitespace. Staging converts the result into formats needed by tools such as Label Studio or downstream models. A notebook for SEC filings demonstrates extracting sections such as risk factors and business discussion, then turning the notebook into a web API. Wolfe also shows direct library use on PDFs and describes examples for sentiment analysis and summarization. He says the project plans to support media beyond text and expects to work further on document ingestion from sources such as S3.

## Key ideas
### Unstructured.io targets the repetitive work before a model sees a document
[01:48](https://www.youtube.com/watch?v=MBKFdy_URpg&t=108s)
Crag Wolfe defines raw documents broadly. The current scope includes PDFs, Word documents, PowerPoint files, email, HTML, XML, text, PNGs, and JPEGs. The library is focused on extracting clean text, with broader media such as audio, video, and drone footage as possible future areas. Wolfe says data engineers often receive many PDFs, then spend time searching for parsing rules before reaching the data needed by their actual application. Unstructured.io is intended to reduce that setup work so teams can move from raw files to usable content.

### The preprocessing pipeline has partitioning, cleaning, and staging phases
[05:46](https://www.youtube.com/watch?v=MBKFdy_URpg&t=346s)
Wolfe breaks preprocessing into three stages, with reusable pieces of logic called bricks. Partitioning identifies the basic document elements. For a scanned newspaper, this can mean placing bounding boxes around columns so they can be OCR'd separately, then identifying text as a title or another text element. Cleaning handles work such as removing headers, footers, page numbers, and extra whitespace. Staging applies the final transformation needed by the next system, such as tokenizing content for a model or formatting it for Label Studio.

### SEC filing pipelines can extract named sections from inconsistent reports
[15:39](https://www.youtube.com/watch?v=MBKFdy_URpg&t=939s)
The SEC filings example fetches 10-K, 10-Q, annual, quarterly, and S-1 filings, then processes their HTML content. Wolfe uses Royal Gold's filing to show that reports contain sections such as risk factors and business discussion, while formatting can vary between companies and reports. The pipeline detects item headings and uses the library's title information and a regular expression to find them. Extra whitespace initially prevents some headings from matching. Applying the clean-extra-whitespace function produces the individual items, after which the pipeline extracts the risk factor section.

### Pipeline notebooks define APIs that can be generated into web services
[20:49](https://www.youtube.com/watch?v=MBKFdy_URpg&t=1249s)
In the repository convention Wolfe demonstrates, files under a pipeline notebooks directory define the source of truth for a domain-specific preprocessing API. A function marked with the pipeline API comment specifies inputs, response formats, schemas, and optional parameters such as the SEC section to extract. The tooling scans the notebooks, generates FastAPI code, and exposes routes that accept filings and return structured data. Users can request JSON, CSV, initial structured data, or a Label Studio schema. Wolfe says the resulting API can be containerized and deployed as needed.

### The library works without the pipeline API conventions
[28:51](https://www.youtube.com/watch?v=MBKFdy_URpg&t=1731s)
Wolfe separately demonstrates the core Unstructured library on a sample PDF. The auto-partition function detects the file type and extracts elements, including their text, type, and metadata. A staging function converts those elements into JSON-serializable data. He also discusses installation requirements, including operating-system libraries and the local inference extra for image processing. The pipeline repositories are optional. Teams can clone the main repository, install the dependencies, and use the partitioning and staging functions directly for their own document workflows.

### Image and mixed-document processing adds layout information before cleaning
[34:17](https://www.youtube.com/watch?v=MBKFdy_URpg&t=2057s)
The second pipeline example processes army evaluation reports represented as images or mixed image-and-text PDFs. A layout model identifies individual text elements by placing boxes around content. Wolfe focuses on a section called significant duties and responsibilities, where the initial extraction includes more material than the desired content. Cleaning bricks then narrow the result to the relevant text. The notebook repeats this pattern for other sections, applies staging where needed, and defines a pipeline API that can be converted into a web service.

### Extracted documents can feed labeling and language-model workflows
[39:04](https://www.youtube.com/watch?v=MBKFdy_URpg&t=2344s)
Wolfe points to example notebooks that use Unstructured output beyond basic extraction. One sentiment-analysis notebook takes sections from the SEC filings work, formats them for Label Studio, labels examples, and trains a sentiment model. Another summarization notebook uses Unstructured to pull content from the Institute for the Study of War website before training a summarization model with Argilla and Hugging Face. In both cases, document extraction is the first step that supplies usable text to labeling or model-building tools.

### The project plans to expand media support and simplify bulk ingestion
[44:44](https://www.youtube.com/watch?v=MBKFdy_URpg&t=2684s)
In response to Ben Epstein's questions, Wolfe says Unstructured is currently focused on NLP because that matches the founding team's experience and current demand. He expects the project to move into other media types and other kinds of data. He also says the team is more concerned about the volume of documents than the size of one document, especially for hosted APIs. Near the end, he mentions a newly released ingestion tool that can read documents from an S3 bucket and produce JSON outputs for a large collection of PDFs.

## Notable quotes
- Crag Wolfe: "Our mission that we're really focused on here is to reduce the amount of time it takes a data engineer to preprocess raw documents for downstream applications by 90%." (04:43)
- Crag Wolfe: "We break it into three stages, so there's a partitioning stage, a cleaning stage, and a staging stage." (05:46)
- Crag Wolfe: "Every pipeline notebook by convention will define this pipeline API." (21:26)
- Crag Wolfe: "You can just use the Unstructured library as is for your own use cases." (28:51)
- Crag Wolfe: "We also just dropped in a way to ingest documents, so say you have an S3 bucket with a thousand PDFs and you just want to get a bunch of JSONs for them." (49:11)

## Tools & references mentioned
- Unstructured.io
- Unstructured-IO/unstructured
- Label Studio
- FastAPI
- SEC filings
- 10-K
- 10-Q
- S-1 filings
- Royal Gold
- Argilla
- Hugging Face
- Institute for the Study of War
- LangChain
- S3

## Who should watch
- You are building an ML or NLP application and are spending time writing parsers for PDFs, HTML, office files, or scanned documents.
- Your team needs a repeatable preprocessing API for a document domain such as SEC filings.
- You want examples of turning extracted document content into labeling, sentiment-analysis, or summarization workflows.

## Related talks

- [Bringing Structure to Unstructured Data with an AI-First System Design](https://mlopstalks.com/talks/bringing-structure-to-unstructured-data-with-an-ai-first-system-design) (Will Gaviria Rojas, Coactive AI, 12:16)
- [Monitoring Unstructured Data](https://mlopstalks.com/talks/monitoring-unstructured-data) (Aparna Dhinakaran & Jason Lopatecki, Arize AI, 13:12)
- [Visualize - Bringing Structure to Unstructured Data](https://mlopstalks.com/talks/visualize-bringing-structure-to-unstructured-data) (Markus Stoll, Renumics, 50:39)
- [So Fresh and So Data Clean](https://mlopstalks.com/talks/so-fresh-and-so-data-clean) (Tommy Dang, Mage, 49:58)
- [The Future of AI and ML in Process Automation](https://mlopstalks.com/talks/the-future-of-ai-and-ml-in-process-automation) (Slater Victoroff, Indico Data, 57:50)