PodcastFounding, Funding, and the Future of MLOpsClaimMihail Eric says data quality needs to be addressed end to end, from labeling through training and serving, with model predictions fed back into the data and labeling process.9:56
Threads / Data quality / 2024
Data quality in 2024
37 sessions
PodcastFounding, Funding, and the Future of MLOps
PodcastHow Data Platforms Affect ML & AIClaimThe balance between modeling, data quality, and speed should depend on user requirements, business criticality, and the need to scale.24:32
PodcastLightweight Feature PlatformClaimTecton is a feature platform that decouples feature engineering code from models and helps manage feature pipelines.2:57
PodcastAds Ranking Evolution at PinterestClaimConversion optimization required third-party integrations to collect, transform, identify, and pass conversion data into model-training pipelines.16:46
PodcastInformation Retrieval & RelevanceClaimBuilding a vector system involves combining data engineering and machine learning engineering, including deciding where models run and how inference services are accessed from data pipelines.24:44
PodcastManaging Data for Effective GenAI ApplicationClaimData quality for generative AI must be checked at the input, prompt, and output stages.18:59
PodcastA Decade of AI Safety and TrustClaimData quality, model validation, and model improvement are tightly connected and should be addressed across the AI life cycle.27:09
PodcastThe Art and Science of Training LLMsClaimDavis Blalock says tokenization can create invalid bytes, unexpected whitespace behavior, and evaluation failures.1:03:15
Building a Python-Centric Feature Platform to Power Production AI ApplicationsClaimTecton initially used Spark across batch pipelines, streaming pipelines, and training-data generation jobs, while managing orchestration, backfills, and retries for users.13:03
From MVP to ProductionClaimAndrew Hoh says keeping knowledge current is fundamentally a version-control problem involving retrieval infrastructure, data sources, processing pipelines, and the underlying language model.25:56
Introducing DBRX: The Future of Language ModelsClaimThe team used PyTorch FSDP because it is flexible across model architectures and can distribute training across large clusters without requiring three-dimensional or pipeline parallelism.22:54
Innovative Gen AI Applications: Beyond TextClaimThe call-center pipeline uses diarization, transcription and translation, personally identifiable information recognition, and large-language-model analysis.29:29
PodcastThe Rise of Modern Data ManagementPushed backChad Sanderson says the main problem with data lineage is upstream of analytical systems, rather than downstream inside tools such as Snowflake or Databricks.29:07
PodcastML and AI as Distinct Control Systems in Heavy Industrial SettingsPushed backRichard Howes rejects the idea that subject-matter professionals should be replaced in large-scale ML analysis and says they are needed to validate the outputs.38:35
Data Quality = Quality AIClaimMaria Zhang says data quality means monitoring metrics such as completeness, accuracy, validity, and timeliness.0:50
PodcastBigQuery Feature StoreClaimMalt manages feature-table updates through SQL files in GitLab, with validation through a pull request and scheduled execution by Airflow.20:10
PodcastRAG Quality Starts with Data QualityClaimTonic Textual is designed to build data pipelines for people creating retrieval-augmented generation systems.4:41
PodcastGlobal Feature Store: Optimizing Locally and Scaling Globally at Delivery HeroClaimCole Bailey said Logistics had built a real-time feature store using streaming pipelines, while Pandora had built a batch feature store using warehouse queries and caches.10:30
11 lessons learned from doing deploymentsClaimData does not need to be perfect, but teams should prioritize use cases whose data is at least reasonably clean and accessible.27:19
Chronon: Airbnb's Open-Source Data PlatformClaimBuilding production machine-learning or prompt systems requires multiple pipelines and infrastructure components, including batch processing, streaming, key-value storage, serving, orchestration, and monitoring.5:19
Data Contracts: The Missing Piece of the Data PuzzleClaimData contracts and data observability are both needed because they address different parts of data quality.1:14
Data Engineering: The Missing Piece of Your Data Science PuzzleClaimThe infrastructure needs an online feature store for low-latency model serving and offline computation for batch models and training pipelines.3:24
Data Quality Management Techniques - The Complete GuideClaimData quality problems can create legal liability, especially when biased or inaccurate data causes harm or violates regulations.4:35
How Data Capture Transforms ML ObservabilityClaimPushkar says the serving endpoint is the last place in the pipeline where data issues that were missed earlier can still be identified.8:02
How Feature Stores WorkPushed backSimba Khadder argues that data scientists should not have to become expert data engineers to build production-grade feature pipelines.8:27
How to Make Your Data Science Reproducible (and Why You Should Care)ClaimA broken pipeline can be caused by changes in the code, data, environment, or cloud architecture.2:18
Real-Time Event Processing for AI/ML with NumaflowClaimA Numaflow pipeline consists of vertices connected by edges, with each vertex running as a container.8:54
Scaling Data Reliably: A Journey in Growing Through Data Pain PointsClaimData reliability engineering treats data quality as an engineering problem and applies engineering practices to improve data and the systems built on it.5:07
The Daft distributed Python data engine: multimodal data curation at any scaleClaimDaft provides an interface for selecting and filtering rows and then streaming the selected data directly into training pipelines.15:50
The Only Constant is (Data) ChangeClaimData teams need to control the transition from one-off data pipelines to production assets so that unmanaged dependencies do not cause later failures.20:47
Turn Data Chaos into AI Strategy with Programmatic AI Data DevelopmentClaimData labeling is needed throughout a generative AI pipeline, including retrieval-augmented generation, fine-tuning, alignment, prompt engineering, and evaluation.5:10
Building an ML Platform from scratchPushed backBen says feature transformations should ideally be centralized in SQL mesh or a similar modeling system, while Eric favors monolithic pipelines early and sees decoupling as a later maturity step.1:26:27