Nikolaos Vasiloglou distilled NeurIPS by starting with keynotes, oral presentations, and workshops, then following references and repeatedly cutting the material down into a structured set of slides.
2
Agent systems are useful for generating scenarios and supporting decisions, but Nikolaos does not trust them with important decisions or execution because probabilistic systems are difficult to control and debug.
3
Verification, knowledge graphs, tabular foundation models, data attribution, and composable smaller language models are directions Nikolaos expects to develop further.
Summary
Nikolaos Vasiloglou explains how he distilled NeurIPS into a six-hour presentation and a smaller NotebookLM podcast. He began with keynotes, oral presentations, and workshops, then followed references and refined roughly 2,000 screenshots into about 600 slides. He describes open language models, reasoning systems, agents, forecasting, and emerging models for tabular data. His view on agents is cautious. They can generate scenarios for research or finance, but important decisions and execution need stronger verification because probabilistic systems are hard to control and debug. He connects this problem to knowledge graphs, which can encode facts, relationships, and rules and let companies verify model outputs against structured enterprise information. The conversation then turns to the difficulty of building these graphs from databases, documents, code, Slack, and email. Nikolaos closes with two possible directions: attributing generated text to training sources and paying contributors, and composing specialized smaller models instead of modifying one large monolithic model.
Nikolaos treats conference distillation as a way to make research usable
Nikolaos says research produces more material than people can read, while useful papers can remain unnoticed for 10, 15, or 20 years. He wants to ingest that material and communicate it to research managers, CEOs, and venture capitalists. NeurIPS was especially difficult because it had close to 8,500 publications and hundreds of hours of video. He reviewed presentations, workshops, and selected papers rather than every submission. Afterward, he made about 600 slides, divided them into 12 lectures, and created a podcast version. He says this process also taught him how enterprise teams might use generative AI to prepare and organize large bodies of material.
The conference review process uses human and program committee filtering before deeper reading
Nikolaos starts with the NeurIPS keynotes and oral presentations, then examines workshops and talks that appear interesting. He uses the judgments already made by reviewers and workshop organizers as an initial ranking signal. Presentations also point toward recent papers and work likely to appear in the next publication cycle, so one conference gives him a view of what happened in the previous six months and what may appear in the next six. He follows references from the talks, including papers that were not widely noticed when published. He took about 2,000 screenshots and repeatedly cut and consolidated them until the material formed a smaller set of presentations.
Open language models are becoming more open while complaints about them point toward future research
Nikolaos says language models were everywhere at NeurIPS, although they were not yet for everyone. He saw many papers describing models that provide checkpoints, reproducible training options, and datasets that users can modify or include in their own deployments. He also points to a presentation by the inventor of long short-term memory models, which framed widespread complaints about language models as evidence that people use them heavily. In his view, complaints about what models cannot do can guide the next advances. He also connected the conference to anniversaries of the sequence-to-sequence model and generative adversarial networks, which helped him reflect on how quickly the field has changed.
Agents are better suited to decision support than to making and executing important decisions
Nikolaos says agents dominated the conference, but he is cautious about claims that smaller models had already displaced frontier models. Six months after NeurIPS, he still sees frontier models ahead, although reinforcement learning and chain-of-thought methods are producing results. He is more enthusiastic about groups of agents that simulate a biology, materials, or antibiotics lab, because occasional imaginative errors can help researchers explore possibilities. For a finance department, agents can also generate scenarios for a what-if analysis. He draws a firm boundary at decision support. Giving an agent control over a bank account or an important operational decision means trusting a probabilistic system with work that can be difficult to control, debug, and correct.
Tabular and time-series foundation models are approaching practical use, but at a cost
Nikolaos says deep learning has overtaken traditional methods in many areas, while tree-based models such as XGBoost have remained strong for tabular prediction. He saw the first group of models that could beat XGBoost on tabular tasks, though they required more computation. He expects language models for tables and databases to take structured data as input, support prediction, and answer questions directly. Demetrios Brinkmann connects this to forecasting foundation models, where the research appears ahead of common industry practice. Nikolaos agrees that the models are still being tested and that the cost of GPUs matters. He also says enterprises want one platform that can handle prediction, retrieval, sentiment analysis, knowledge extraction, and optimization rather than many separate systems.
Verification is easier when a model's output can be checked by a symbolic system
Nikolaos says verification is becoming central to making AI more reliable. A language model can propose a theorem proof, while a verifier checks whether the proof is valid. The same pattern can apply to generated software, although checking that code compiles is much easier than proving that it does what the user requested. The verifier may need to create abstract tests, such as proving that a variable never becomes negative or exceeds a bound. Nikolaos says these verifiers are generally symbolic systems built by people. For a proposed route, a map graph can check whether each connection exists and whether the stated distance is correct. Enterprise verifiers often involve graphs and rules, which leads directly to knowledge graphs.
Knowledge graphs connect enterprise facts and rules to model outputs
Nikolaos describes a knowledge graph as an intermediate representation that humans and machines can understand. It contains facts, qualifiers, relationships, entities, and rules. In a company, it can provide a shared representation of how the business thinks about its data, rather than leaving definitions inside long SQL queries or application code. He distinguishes an ontology, which contains more stable types such as locations, from facts that change and carry timestamps or expiration dates. RelationalAI's relational knowledge graph approach keeps the graph close to a relational database, using existing database support for retention, access roles, and scaling. The hard work is gathering information from databases, documents, code, Slack, email, and other sources while respecting security and data policies.
Data attribution could connect generated text to the sources that influenced it
Nikolaos says he saw research on data attribution even though data monetization was not a main workshop topic. A system could identify the page or training example that influenced a generated sentence, potentially allowing the source creator to receive a payment. He describes this as a response to copyright disputes around training data. Attribution would need to work at a fine level, such as a sentence, word, or token, because the model may draw on a source without repeating it directly. He gives an example of a model connecting a shutdown scene to a science-fiction novel written in another language. His idea is that traceable influence could create a way to pay contributors when their work helps produce generated content.
Composing specialized smaller models could reduce dependence on monolithic models
Nikolaos sees model composition as another direction that has not fully taken off. Large models are expensive and monolithic, similar to writing an entire operating system in one file. Smaller models could instead be trained for separate tasks and combined. He gives a translation example: one model handles Greek to English and another handles English to Italian, so composing them could support Greek to Italian without training a new model for that pair. This approach differs from adapting one large Llama model with a low-rank update. Nikolaos says composition has advantages and disadvantages, but it could let groups train separate capabilities and combine them later.
"The problem is as I said, building the perfect knowledge graph, it's not a task, it's a journey and it can take time."Nikolaos Vasiloglou42:02
Who should watch
You need a practical method for filtering a large research conference or technical literature set before asking AI tools to summarize it.
Your team is considering agents for research, analytics, or operations and needs a clear boundary between scenario generation and autonomous execution.
You are building enterprise AI around structured data and want to understand why verification, knowledge graphs, and data lineage matter.
