Anatomy of a Software 3.0 CompanyPushed backSarah Guo disputed the idea that all AI companies must immediately be gross-margin positive, saying a company can accept negative gross margins if inference costs are expected to decline.27:41
Threads / Paying for it / 2024
Paying for it in 2024
14 sessions
Anatomy of a Software 3.0 Company
Reliable Hallucination Detection in Large Language ModelsClaimThree to five samples can provide competitive performance while reducing sampling cost compared with larger sample sizes.17:35
LLMOps and GenAI at Enterprise Scale - Challenges and OpportunitiesClaimFor enterprise foundation-model selection, cost per query, cost per user, latency, throughput and whether the model solves the business problem matter more than leaderboard position.2:30
Streamlining Model DeploymentClaimDaniel Lenton says the number of AI-as-a-service endpoints has grown rapidly, with different endpoints offering different cost, performance, and specialization profiles.1:04
Productionizing AI: How to Think From the EndClaimDeploying LLM-based features requires considering API rate limits, cost, processing time, tokenization, embeddings, compute, evaluation, and the quality of user-provided data.6:09
No GPU Before PMFClaimFor B2C AI products, the current priority order is cost, speed, and performance.2:40
Navigating the Emerging LLMOps StackClaimHien Luu says that organizations supporting high-scale QPS use cases need to consider cost and latency.2:30
PodcastAWS Trainium and InferentiaClaimAWS developed Trainium and Inferentia as purpose-built AI accelerators to offer customers more choice, higher performance, lower cost, and easier access to compute.2:48
How To Cut Your Data Infrastructure Costs in HalfClaimJose Navaro says that tracking data infrastructure costs is part of the culture at Cleo because the company helps users manage their finances.1:27
How to Actually Use Cost Effective AI in Your BusinessPushed backThe speakers distinguish AWS Trainium and Inferentia from GPUs rather than treating them as AWS versions of GPUs.22:03
LLMs to agents: The Beauty & Perils of Investing in GenAIPushed backMeera Clark argues that many AI businesses are not currently viable because of cost, while George Robson and Sandeep Bakshi say proprietary data and relative competition can still support viable businesses.20:31
PodcastAI-Driven Code: Navigating Due Diligence & Transparency in MLOpsClaimSema’s cloud-cost assessment can estimate spending trends, spending maturity and potential savings after about an hour of setup.11:21
