As AI Devours it All, Reset, Don’t Retreat

Data is everything in our modern world. Companies invest millions to collect, analyze, and protect it. Users are constantly asked to consent to — or decline — the processing and use of their personal information. Yet, data remains one of the most debated topics in technology today. Just think of the controversies surrounding data privacy and regulation, from major breaches to Mark Zuckerberg’s 2025 testimony during Meta’s data privacy trials. 

At the heart of these debates is a fundamental question: who has the right to control information in a digital world? Data privacy refers to the right of individuals to govern how their personal information is collected, used, and shared. But in the context of AI and large-scale data scraping, that line becomes increasingly blurry.

As large language models (LLMs) like ChatGPT and Gemini have become increasingly powerful, creators and publishers have begun to draw clear boundaries. The New York Times, for instance, and several other major publishers have blocked OpenAI’s web crawlers from accessing their sites, citing concerns over copyright and compensation.

But what makes data so important, and why do AI companies want it so much?

Why Data Matters More Than Ever

Several things make data extremely valuable, but for this article, we’ll be zeroing in on a very specific use of that data: how large language models (LLMs) use data to function, learn, and evolve. However, keep in mind that data can also be used for other purposes, such as marketing or consumer analytics. 

LLMs rely on data not just to make predictions, but to form the foundation of their “intelligence.” These models—like GPT, Claude, Gemini, and others—are trained on massive amounts of text data gathered from books, websites, forums, and other public or semi-public sources. They don’t merely reference this data—they use it to predict the next element in a sequence.

This dependence on data is both their strength and their vulnerability. LLMs require enormous, diverse, and high-quality datasets to generate coherent, contextually relevant outputs. And as the technology grows more sophisticated, the demand for such data continues to escalate.

This raises crucial questions: Where does all this data come from? Who owns it? Should it be used this way?

However, researchers from EPOCH AI posed another crucial question: Will they run out of high-quality data?

Without a continual influx of new, high-quality human-generated content—such as writing, art, dissertations, and insights—LLMs risk becoming echo chambers. If the majority of future data consists of AI-generated text, often referred to as “LLM slop,” models will begin learning from watered-down versions of their own output. In effect, they will be training on reflections of reflections, degrading the originality, nuance, and richness of the content over time.

For this reason, we have now entered the era of “synthetic data”: the most capable AI models are generating high-quality data at a large scale. Will this marvel be the sole culprit in their downfall?

The Tension: Between Hype, Collapse, and Potential

“Our 80% confidence interval is that the data stock will be fully utilized at some point between 2026 and 2032,” assure the experts Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn in their analysis of LLM scaling trends at Epoch AI. 

Figure 2: Projection of effective stock of human-generated public text and dataset sizes used to train notable LLMs. Individual dots represent dataset sizes of specific notable models. The dataset size projection is a mixture of an extrapolation of historical trends and a compute-based projection that assumes models are trained compute-optimally. Source: Epoch AI

That projection may sound abstract, but its implications are significant. If human-generated data stops growing—or worse, starts being drowned out by AI-generated content—we may reach a saturation point where models can no longer improve meaningfully. 

If these models do not continue to grow, it’s very bad news for everyone who is investing millions in them, including the capital influx to data centers, microchip manufacturing, and more. Can AI systems continue to evolve if they’re starved of novel human input? Or are we heading toward a feedback loop where models simply remix recycled fragments of the past?

Nowhere is this clearer than in software development. Code, at its core, is an act of creativity. It’s not just syntax and structure—it’s problem-solving, architecture, abstraction, and design. Ask two developers to solve a software development problem, and they will probably write two very different solutions. When developers use AI to generate code, they’re speeding up routine tasks, yes, but they’re also outsourcing some of the very creative decisions that drive innovation.

If AI is trained primarily on existing open-source repositories, tutorials, and outdated codebases, and if humans begin contributing less original code because AI handles the heavy lifting, the pipeline of new logic, design patterns, and conceptual breakthroughs begins to dry up. 

Eventually, models will be generating code based on past models’ output, not fresh engineering insight. This recursive cycle may accelerate stagnation rather than progress.

These concerns feed into a broader fear: the potential collapse of AI systems; not in the apocalyptic sense, but as a form of structural breakdown. Collapse here doesn't necessarily mean failure. Instead, it suggests a reset, a disruption of existing assumptions that forces organizations, institutions, and industries to re-evaluate how they design, deploy, and rely on AI.

What Collapse Could Mean — And Why It’s Not the End

The word collapse tends to conjure images of failure, chaos, or irreversible decline. But in the context of AI and data systems, collapse doesn't necessarily signal the end. It can also mean a turning point—a structural breakdown that clears the way for new models, new thinking, and new priorities.. 

In this sense, collapse is less about destruction and more about transformation. When existing systems reach their limits—be it in data availability, governance, or public trust—we are given a chance to pause, reassess, and reimagine. And this is where the opportunity lies, good news for us humans.

As many companies explore how to operate with fewer people and more AI, it becomes crucial to understand the limitations, risks, and potential failure points of this technology. Even if a full-scale collapse never occurs, we are still navigating a volatile transition—one in which over-reliance on automation without critical oversight could lead to stagnation, systemic bias, or loss of public trust.

And even in a best-case scenario—one where systems remain stable and productive—human creativity will continue to be the driving force behind meaningful progress. AI may replace tasks, but it won’t replace imagination. It can generate text, write code, or suggest designs, but it cannot invent context, purpose, or insight. That capacity remains uniquely human (at least for now).

As repetitive work becomes increasingly automated, fields rooted in creativity, ethics, and critical thought will become more important, not less. We are entering not the end of human value, but a new era for the humanities. Writers, historians, philosophers, and yes—even software developers—hold the keys to what powers AI: data. 

Let us remember that data is no more than human experience captured in an archive, to be preserved for posterity, to share knowledge with others, and to help our society function effectively. 

Every sentence in a forum post, every line of code, every published essay or story represents someone's thinking, feeling, building, or learning. AI learns from us—not just our facts, but our creativity, our conflicts, and our imperfections.

Preparing for the Future

In a landscape dominated by black-box models and opaque data practices, government policies, legal protections, and enforceable boundaries are not optional: they are non-negotiable. If we're serious about preparing for a future shaped by AI, we need to engage in a broad societal conversation about intellectual property, authorship, and data access. 

We already know that data is a valuable resource—perhaps the most valuable. And AI companies know it too. They’ve proven willing to pay for high-quality, well-organized data. Consider the case of Wiley, one of the world’s most prominent academic publishers. In 2024, Wiley reported a $23 million AI licensing deal and is projecting an additional $21 million in revenue in Q1 of fiscal 2025 thanks to partnerships with Anthropic, Perplexity, and AWS. This shows that clean, trustworthy, human-produced data now has real market value. And this is just one of the many news stories that have flooded the internet in recent weeks about AI companies spending significant money on fresh, high-quality content. 

While most are trying to leverage AI to create everything for them, AI companies are hiring creators to produce original content. 

This marks a shift in the economic model of AI. Where data was once scraped indiscriminately, we are now seeing a pivot toward licensed, curated, and protected content ecosystems. That shift opens up space for new roles and industries built around responsible data governance. 

As AI increasingly contributes to software development, a legal and ethical question emerges: who owns AI-generated code, especially when it’s built on a foundation of scraped public repositories? Licensing concerns—especially with GPL or copyleft licenses—are now real risks for teams adopting AI-generated outputs without scrutiny. We’re entering a phase where developers will need to act as both creators and compliance officers, ensuring that innovation doesn’t outpace legal or ethical clarity.

This opens the door for new roles within software organizations, from AI-aware documentation specialists to rights managers, governance advisors, and data stewards. It’s a new layer of responsibility in tech culture: preserving the integrity of codebases, protecting developer contributions, and shaping how their work is used and reused by machines.

There is real room for optimism here. Jobs such as digital rights management, data curation, knowledge conservation, licensing strategy, and ethical compliance will be extremely necessary and also deeply human. These are all applicable to our industry and require judgment, negotiation, historical context, and cultural sensitivity.

If data is the fuel of AI, humans will increasingly be the stewards of that fuel, the ones who create it, clean it, structure it, and decide how and when it should be used. Building forward means recognizing that value, protecting it through law and policy, and ensuring the people doing that work are respected and compensated.

Human Creativity Is the Future, And We're Investing In It

As AI continues to evolve and integrate into every layer of business and technology, one thing becomes increasingly clear: human creativity, context, and judgment are irreplaceable. Whether it’s writing code, managing data rights, or shaping policy, the future will be built by people who can think critically, adapt, and design systems that serve human needs, not just technical possibilities.

At Jalasoft, we understand this deeply. That’s why we don’t see AI as a replacement for human talent: we see it as a tool to enhance it: "Critical thinking, creativity and adaptability remain human strengths that AI can replicate", explains Pablo Gini. That's why, through our partnership with Jala University, we train our engineers to become more than coders. We equip them to think strategically, understand the ethical implications of their work, and collaborate across disciplines. 

That’s the future we’re building at Jalasoft, one where human insight stays at the center of technological progress.