Duncan Curtis argues that human intelligence matters most when people select meaningful data, recognise context, and correct model mistakes.
2
Sama helps companies choose, curate, annotate, validate, and sometimes build the data and models needed for AI projects.
3
Curtis recommends proving business value with internal users before exposing an AI system to customers.
Summary
Duncan Curtis describes Sama as a partner for companies that need better data for AI. He focuses on the human intelligence involved in annotation, where people recognise objects, track them across frames, and understand unusual situations that automated tools miss. For autonomous vehicles, the work includes finding rare events, balancing classes and conditions, and accounting for regional differences in people, roads, and driving behaviour. Curtis also discusses how synthetic data, model-assisted annotation, interpolation, and targeted validation can reduce the amount of manual work without removing people from the loop. He expects some specialised annotation tasks to shrink as general models improve, while agentic systems will create new demand for human feedback because they struggle to detect and recover from mistakes across long sequences of actions. On AI project selection, he recommends small proofs of concept tied to business measures, with internal deployment before customer-facing release.
Annotation captures human understanding of a scene
Curtis says annotation is more than drawing boxes around objects. In an autonomous vehicle scene, a person may recognise that a car remains the same car after passing behind a truck because of a distinctive unicorn sticker on its bumper. An automated first pass can draw most of the scene, while a human corrects mistakes such as assigning a new identity after the car disappears briefly. This approach keeps human judgement in the process while reducing repetitive work. Curtis calls the human contribution intelligence because the valuable information is the person's recognition of what is happening, not only their ability to use annotation tools.
The best data to label is often a short unusual event
Curtis says teams should find relevant moments before sending data to annotation. A long commute contains little useful information when the vehicle stays in its lane and nothing changes. A car cutting someone off or another unexpected event is much more valuable. Curation can identify those sections, then teams can label a short window around them instead of processing the entire recording. Curtis also describes checking whether a dataset contains enough examples of different objects and events, including buses, bikes, skateboarders, weather conditions, glare, snow, rain, and night scenes. This helps teams find gaps before the model encounters them in production.
Bias and regional behaviour have to be represented in the data
Curtis recalls a motion-sensing game project that performed poorly for children and people of color because the team had collected training examples mainly from themselves. He uses this as an older example of a problem that still affects AI systems. For autonomous vehicles, coverage involves more than object classes. Teams need to consider attributes such as height and skin tone, along with weather and lighting. Driving behaviour and laws also vary between regions. Curtis mentions differences between US states, Europe, and Asia, as well as aggressive driving in New York and faster traffic in California. These differences need to be reflected in model training and validation.
Rare situations require synthetic data and new classes
Curtis says real-world data is only part of the autonomous driving process. Simulation and synthetic data help teams explore the large number of edge cases that vehicles may encounter. Demetrios Brinkmann mentions a horse and carriage that a model could not classify because the training data had not represented that combination. Curtis adds that even a class such as skateboarders can change when new devices appear. A model trained on upright people moving at around skateboard speed may not recognise a similar-looking device travelling much faster. Teams need to identify the new class and reinforce the model with suitable examples.
Technology changes can invalidate careful workforce plans
Curtis describes planning around a workforce focused mainly on English and local languages in East Africa, while many natural-language-processing customers appeared to want coverage across roughly 120 languages. The arrival of generative AI changed the immediate fit because most foundation models were focused on English, making the existing workforce and training pipelines more relevant than expected. He uses this as an example of preparing for a future problem that technology may remove or change. Curtis says Sama continues to consider where human work will be needed as models absorb more specialised knowledge, rather than assuming today's annotation demand will remain at the same scale.
Agentic systems create a wider need for human feedback
Curtis says agentic AI systems can complete complex, multi-step tasks such as planning a holiday, checking preferences, searching websites, and creating a calendar. Their weakness is that they often perform steps serially and may fail to notice when they have gone off track or recover from the error. He compares this with autonomous driving, where unusual cases expose gaps in a model. People also have to adapt when a software interface changes, first recognising that the change occurred and then working out how to proceed. Curtis expects this changing tool environment to create sustained demand for human feedback on model plans and actions.
Data has become a larger bottleneck because its volume and history are difficult
Curtis agrees that data is a major bottleneck, partly because the volume used to train large language models has grown enormously. He also points to the economic value of data, mentioning Reddit's decision to license its dataset after it had previously been scraped for model training. Private companies hold valuable interaction data that was often collected for older product needs, not machine learning. Teams then face the work of understanding what exists, making it machine-readable, and deciding which parts people or models should inspect. Curtis says failures in old systems and missing records can make this problem worse, especially in regulated or slow-moving industries.
Internal proofs of concept can show value before public release
Curtis recommends starting AI projects with a minimum viable product tied to business measures. He describes a company that wanted an external language model system for handling insurance claims but was concerned about customer-facing failures and reputational damage. Sama suggested deploying it first with internal staff, who could review and approve the generated responses. The company could compare users of the tool with those who did not use it, measuring speed, ticket volume, and customer experience. The early version exposed problems, but employees could catch them while the company still gained value. Curtis says the project had produced business value for more than six months even though the external release had not happened.
