Neptune's mission is to give AI teams control and confidence while they build, test, deploy, and monitor models.
2
Experiment tracking covers model debugging, comparison, versioning, reproducibility, and handing models over to operations, so the work extends beyond research experiments.
3
Prompt engineering will need testing, validation, lineage, and connections to classical models before foundation-model systems can be used with the same confidence as conventional ML systems.
Summary
Piotr Niedźwiedź explains how his path from software engineering and competitive programming led to Codilime, DeepSense.ai, and Neptune. Neptune began as an internal experiment tracker for a consulting business, created because teams were managing models with spreadsheets and homegrown tools. Piotr says the hardest product question is not which problem can be solved, but whether it will remain valuable after cloud providers or larger platforms address it. He connects this discipline to Neptune's mission: giving AI teams control and confidence over model development and testing. Large language models change the building process because prompts, agents, and foundation models become part of the system. They do not remove the need for experiment tracking. Teams still need to understand behavior, compare versions, validate changes, reproduce results, and trace which component uses which version. Piotr expects numerical validation methods, combinations of foundation and classical models, and a more integrated ecosystem of focused tools rather than one end-to-end platform.
Competitive programming gave Piotr confidence to start companies
Piotr describes competitive programming as an important part of his education and early career. He won a silver medal at the national level in the Polish Olympiad in Informatics and later competed in Google Code Jam and Facebook Hacker Cup. These experiences gave him access to strong universities and a broad international network. They also gave him confidence that he could attempt difficult work and build a company. After receiving offers from large technology companies in the Valley, he chose entrepreneurship instead. His first company, Codilime, began as a way to generate money for a future product company and later grew to more than 400 people, with his brother as CEO.
Neptune started after Piotr saw teams managing models in spreadsheets
Piotr co-founded DeepSense.ai after seeing the potential of deep learning and wanting to build expertise with other people. The company began by entering Kaggle competitions and later provided machine learning services. Neptune came from an internal tool created by one of the data scientists, who combined experiment tracking with infrastructure management. At the time, training infrastructure required a lot of engineering work. Piotr then asked data scientists and leaders at other companies how they managed model development. He repeatedly heard that they used spreadsheets or homegrown systems. That repetition convinced him there was a real problem, even though it did not yet prove that a lasting business could be built around it.
The hard product decision is separating temporary problems from lasting needs
Piotr says that spotting problems in MLOps or large language model operations is relatively easy. A capable engineer can often build a quick solution for a specific issue faster than a cloud provider. The harder questions are whether the provider will solve the issue within six months or a year, and whether a company can defend its position over time. He frames product work around the jobs that must be done, why they matter, and why another company should not solve them. Neptune once passed on enterprise deals that would have doubled its revenue because the requested work did not fit its core jobs to be done. Piotr sees this market as a marathon rather than a winner-takes-all contest.
Neptune's mission stays stable while the definition of a model changes
Piotr distinguishes between a company mission, which should change very rarely, and a product vision, which can adjust more often. Neptune's mission is to help machine learning or AI teams gain the control and confidence they need while building and testing models. Large language models complicate the meaning of model building because a system may involve a prompt, an agent, several models, or a combination of these parts. Piotr still sees the same underlying need as in software development: teams should be able to develop, test, deploy, and monitor systems in a controlled way. He expects the mission to remain stable even as Neptune's product changes to support new methods.
Large language model systems still lack dependable validation
Piotr says many production uses of large language models still include a human in the loop, especially where the model can be wrong. Conventional deep learning systems have validation metrics and established ways to build confidence, while prompt-based systems often depend on a person deciding which prompt looks better. He expects new methods for testing and validating prompts and other foundation-model systems. These may include a foundation model judging prompt sets, an agent calling a classical model for a prediction, or a combination of fine-tuning and additional models. The specific solution is not yet clear to him. The need is clear: production systems require evidence for why one version is better than another.
Experiment tracking covers production lineage, not only research
Piotr says Neptune is seeing more demand for experiment tracking than expected, even with the attention around large language models. He thinks the name is too narrow because the work is not limited to experimentation. Teams need to understand what happens while building a model, debug the process, compare it with other runs, version it, reproduce results, share it for feedback, and hand it to an operations team. Prompt engineering adds another form of building, with prompt chains and agents replacing some conventional model-development steps. Neptune plans to support prompt and chain visualizations, including integration with LangChain. Piotr still treats this as part of a broader system for tracking how components are built and used.
A mature ecosystem will connect focused tools instead of replacing them with one platform
Piotr predicts that systems built around foundation models will gain numerical methods for testing and validation. Classical models will remain useful for narrower tasks such as predictive maintenance and recommender systems, and teams will need ways to combine them with foundation models. He also expects the MLOps ecosystem to become more mature through clearer categories and stronger integrations. Market pressure may reduce the number of vendors, making it easier for a focused product such as Neptune to decide which partners to integrate with. Piotr is skeptical that one end-to-end platform will dominate, since software has many focused tools that work together. He is much less certain about the ultimate capability ceiling of current foundation-model methods.
"If you are building a solution based on a foundational model that is doing something specific with your data, then maybe fine-tuning, maybe it would be done by an agent that would ask your smaller classical model for some prediction."Piotr Niedźwiedź35:46
Who should watch
You are deciding whether an experiment-tracking product should expand into prompt, agent, or foundation-model workflows.
Your team uses spreadsheets or homegrown tools and needs reproducibility, comparison, versioning, and evidence for model decisions.
You are building an MLOps product and need a way to separate short-lived customer requests from problems that can support a lasting product.
