Zum Inhalt springen

Ilya Sutskever and the GPT Series

Zusammenfassung

Ilya Sutskever co-authored AlexNet, co-founded OpenAI, served as its Chief Scientist through the entire GPT series from GPT-1 to GPT-4, voted to fire his own CEO in November 2023, expressed public regret within 24 hours, and left OpenAI in May 2024 to found Safe Superintelligence Inc — a company whose sole stated purpose is building superintelligence safely. He is one of the most technically accomplished AI researchers of his generation and one of the figures most responsible for the capabilities that now define the frontier of AI. He appears to believe, with unusual intensity, that what he is building could be the last thing humanity ever builds — and that this is a reason to continue, carefully, rather than to stop.

Nizhny Novgorod, Jerusalem, Toronto

Ilya Sutskever was born in 1986 in Nizhny Novgorod, Russia. His family emigrated when he was young, first to Jerusalem, where he spent his formative years and — while still a teenager — began taking computer science courses at the Open University of Israel, and then to Toronto, where he completed his undergraduate degree at the University of Toronto and enrolled in the PhD program under Geoffrey Hinton.

The Hinton lab at Toronto in the mid-2000s was one of the few places in academic computer science where neural network research was prosecuted seriously and without apology. The surrounding field had moved largely toward statistical learning theory, kernel methods, and support vector machines — approaches with cleaner theoretical properties and more reliable benchmark performance. Hinton’s group kept working on deep networks because Hinton was convinced they were right, and Sutskever absorbed both the technical content and the conviction.

He was widely regarded, by colleagues in the Toronto group, as possessing an unusual combination of theoretical clarity and engineering judgment — the ability to identify which ideas would work before the experimental evidence was conclusive, and to implement them with sufficient care that the results were clean enough to learn from. These are not separable skills in deep learning research; the gap between a correct idea and a working implementation is often where research programs die.

His relationship with Hinton was formative in the sense that matters most: he learned not just what Hinton knew but how Hinton thought about problems — which aspects of a system mattered, which benchmarks were worth caring about, which simplifications preserved the essential structure and which destroyed it. The intellectual inheritance was as important as the technical training.

AlexNet: The Proof

In 2012, Sutskever was one of three authors — with Alex Krizhevsky and Hinton — of the AlexNet paper, “ImageNet Classification with Deep Convolutional Neural Networks,” submitted to NeurIPS 2012. The paper described a deep convolutional neural network that won the ImageNet Large Scale Visual Recognition Challenge by a margin that redefined the field’s sense of what was possible.

The technical contributions of the three authors overlapped, but Sutskever’s roles were primarily in training infrastructure and regularization. He was centrally involved in making the CUDA implementation that trained the 60-million-parameter network on two consumer NVIDIA GPUs work efficiently — a non-trivial engineering achievement given the GPU programming tools available in 2011. He also contributed to the development of dropout regularization, which Hinton and Srivastava had first described but which was critically important to AlexNet’s performance on the validation set.

The training process for AlexNet required a week of computation on those two GPUs. Every hyperparameter decision — learning rate schedule, weight decay, data augmentation strategy, the specific architecture of the pooling layers — affected the final validation error. Getting the training right was as important as getting the architecture right, and Sutskever’s engineering judgment was central to both.

The result — a gap of more than ten percentage points over the second-place system on the world’s most competitive computer vision benchmark — was, in retrospect, the starting pistol of the deep learning era. (See Geoffrey Hinton and Deep Learning for the full account of AlexNet’s impact.)

OpenAI Co-Founder and Chief Scientist

After AlexNet, Sutskever spent a year at Google Brain — part of the DNNresearch acquisition that brought Hinton, Krizhevsky, and Sutskever to Google — before leaving to co-found OpenAI in December 2015. He was 29 years old.

The OpenAI founding team was assembled around a shared conviction: that artificial general intelligence was coming, that its development was the most consequential thing happening in science and technology, and that the default trajectory — developed primarily by for-profit companies with commercial incentives — would not produce outcomes good for humanity. Sutskever’s title was Chief Scientist, reflecting his role as the primary driver of OpenAI’s research direction and technical quality.

This role, held for nearly a decade, covered the entire arc of what became the most consequential research program in AI history.

The GPT Series: Scaling Toward Capability

GPT-1 (2018): The first Generative Pre-trained Transformer, described in “Improving Language Understanding by Generative Pre-Training.” The paper demonstrated that a language model pre-trained on a large text corpus could be fine-tuned on specific downstream tasks to outperform models trained on those tasks from scratch. Transfer learning worked for language, not just vision. The specific architectural choice — a Transformer decoder trained autoregressively to predict the next token — would prove extraordinarily durable.

GPT-2 (2019): A 1.5-billion-parameter language model that generated coherent, contextually appropriate long-form text at a quality that surprised researchers inside and outside OpenAI. The decision to withhold the full model at launch — citing concerns about misuse for disinformation — attracted both serious discussion about responsible disclosure and significant skepticism about whether the capability was actually as dangerous as claimed. OpenAI released the full model six months later when misuse had not materialized at the predicted scale. The episode was a rehearsal, however imperfect, for the governance questions that would define the following years.

GPT-3 (2020): 175 billion parameters, trained on roughly 300 billion tokens sampled from a ~499-billion-token dataset (higher-quality sources were weighted up, lower-quality Common Crawl down). The capability jump from GPT-2 was qualitative, not merely quantitative. GPT-3 demonstrated few-shot learning: given a few examples of a task in the prompt — “translate the following English to French: … / here are three examples” — the model could generalize to new instances without any fine-tuning. This was behavior that had not been explicitly trained for and that emerged from scale. The model could write code, compose poetry, answer factual questions, and engage in structured reasoning with a fluency and flexibility that surprised the researchers who built it.

Info

The scaling hypothesis — the belief that simply training larger models on more data with more compute would produce qualitatively new capabilities — was the central theoretical bet driving the GPT series. Sutskever was among its earliest and most consistent advocates inside OpenAI. Most AI researchers in 2018 and 2019 believed that algorithmic innovations, not just scale, would be necessary for significant capability gains. The GPT series, and especially GPT-3, provided strong empirical evidence that this was wrong — that scale alone could produce emergent capabilities that no one had explicitly designed for. This finding reshaped the field’s research priorities and drove the multi-hundred-billion-dollar investment in AI compute infrastructure that followed.

GPT-4 (2023): A multimodal model (accepting both text and image inputs) with capabilities substantially beyond GPT-3, including near-human performance on professional and graduate-level standardized tests, complex multi-step reasoning, and detailed instruction following. GPT-4 was the model underlying the version of ChatGPT that reached 100 million users in two months.

Alignment and RLHF: Making Capability Useful and Safe

Alongside the capability scaling work, Sutskever was a driving force behind OpenAI’s alignment research — the effort to make AI systems behave in ways aligned with human intentions and values.

The central technical contribution was Reinforcement Learning from Human Feedback (RLHF), developed by a team that included Paul Christiano, John Schulman, and others. The approach trained a reward model on human preference data — pairs of model outputs where human raters indicated which was better — and then used reinforcement learning to fine-tune the language model to maximize this reward. The result was InstructGPT (2022), a model that was demonstrably better at following instructions, more helpful, and less harmful than the base GPT-3 model, without loss of capability.

RLHF is the technique that made ChatGPT possible. The base GPT-3 model, asked to help a user with a task, might produce technically impressive but unhelpful or unsafe output. InstructGPT’s RLHF fine-tuning produced a model that understood what “help me with this” actually meant and tried to do it. The technique is now standard across the industry — every major language model uses some form of human-preference fine-tuning — and it originated from the alignment research program Sutskever helped build.

The Board Vote and Its Aftermath

On November 17, 2023, Sutskever voted with the other members of the OpenAI board to fire Sam Altman as CEO. He was the only co-founder on the board. The board’s stated reason was concerns about Altman’s candor in communications with the board.

Within 24 hours, Sutskever’s position had changed. He was among the signatories of the open letter, signed by nearly all OpenAI employees, calling for Altman’s reinstatement. He posted on social media: “I deeply regret my participation in the board’s actions. I never wanted to harm OpenAI. I love everything we’ve built together and I will do everything I can to reunite the company.”

Altman was reinstated on November 21. Sutskever was not among the board members replaced immediately, but his position within the organization — and his relationship with Altman — was irreversibly changed. He remained at OpenAI until May 2024.

Warnung

The reversal within 24 hours raises questions about what Sutskever believed he was doing. The most charitable reading is that he voted to fire Altman out of genuine safety concerns, discovered that the action had effects he had not fully anticipated — the near-total employee revolt, the Microsoft offer, the threat to the organization’s existence — and concluded that the cure was worse than the disease. The less charitable reading is that he was persuaded or pressured. He has not explained his reasoning in any detail. What is clear is that the safety concerns that motivated his original vote — whatever they were — were not resolved by Altman’s reinstatement; they simply became secondary to the organization’s survival.

Safe Superintelligence Inc

In June 2024, Sutskever co-founded Safe Superintelligence Inc (SSI) with former OpenAI colleagues Daniel Gross and Daniel Levy. The company’s structure was deliberately simple and deliberately unusual: no products, no revenue, no short-term milestones. A single long-term goal: build safe superintelligence.

The company raised $1 billion in its first funding round at a $5 billion valuation — a significant endorsement of Sutskever’s technical credibility. The structure was a direct response to what he had observed at OpenAI: that commercial pressures, product launches, and revenue requirements structurally subordinated safety work to capability deployment, regardless of the organization’s stated intentions. At SSI, there would be nothing to subordinate safety to because there would be no products, no revenue, no launch dates.

Whether this structure can actually be maintained — whether investors will remain patient indefinitely, whether the pressure to demonstrate capability will inevitably produce pressure to deploy — is not yet known. SSI is, as of 2026, in its early stages.

Warnung

The deepest tension in Sutskever’s position is this: he believes, apparently sincerely, that building superintelligence is one of the most dangerous things humans can attempt. He also believes that it is going to happen regardless — that the commercial and competitive pressures driving AI development are not subject to voluntary restraint. Given that, his argument is that it is better to have safety-focused researchers at the frontier than to cede the frontier to those less focused on safety. This is a coherent position. It is also a position that provides no logical terminus — no point at which the capability level is high enough that the argument for continuing becomes untenable. Critics argue that this is precisely its flaw.

His trajectory — building the most capable AI systems ever created, then leaving to build them more carefully — is the story of the AI era in miniature: extraordinary technical capability meeting extraordinary uncertainty about consequences, and the people most responsible for creating both unable to do anything other than continue.

📚 Sources