60% of managers admit they need better KPIs for AI. Only 34% are doing anything about it. Meanwhile, the data that actually matters already exists, but nobody’s looking at it.

TL;DR: Companies measure activity (hours saved, tasks automated) instead of impact. A Stanford paper analyzing 25 million workers shows what to do instead: segment by role and seniority, distinguish substitutive from augmentative use, use control groups, monitor in real time. Those who adopt these principles will have an information advantage over those still tracking vanity metrics.

The 2025 AI adoption reports tell a strange story. On one hand, companies claim to measure everything: completed deployments, hours saved, tickets handled, costs reduced. On the other, 42% are abandoning most of their AI projects, more than double the previous year. According to MIT NANDA, 95% of pilot projects generate no measurable impact on the bottom line.

If we measure so much, why do we fail so often?

The problem is we’re measuring the wrong things. Typical enterprise AI metrics (time saved per task, volume of automated interactions, cost per query) capture activity, not impact. They tell you whether the system works technically, not whether it’s creating or destroying value.

A paper published in August 2025 by Stanford’s Digital Economy Lab offers a different approach to what it means to truly measure AI. And the implications for those managing technology investments are concrete.

The vanity metrics problem

Most corporate AI dashboards track variants of the same metrics: how many requests processed, how much time saved per interaction, what percentage of tasks automated. These are numbers that grow easily and look good in slides. Their flaw is fundamental: they say nothing about real business impact.

A chatbot handling 10,000 tickets per month looks like a success. But if those tickets still require human escalation 40% of the time, if customer satisfaction has dropped, if your most profitable customers are migrating to competitors, the number of tickets handled captures none of this.

The S&P Global 2025 report documents exactly this pattern: companies that accumulated “deployments” and “completed experiments” only to discover, months later, that ROI wasn’t materializing. Costs were real and immediate; benefits were vague and perpetually deferred to next quarter.

According to an MIT Sloan analysis, 60% of managers recognize they need better KPIs for AI. But only 34% are actually using AI to create new performance indicators. The majority continues using the same metrics they used for traditional IT projects, metrics designed for deterministic software, not for probabilistic systems interacting with complex human processes.

What serious measurement looks like

“Canaries in the Coal Mine”, the paper by Erik Brynjolfsson, Bharat Chandar, and Ruyu Chen published by Stanford’s Digital Economy Lab, isn’t about how companies should measure AI. It’s about how AI is changing the labor market. But the method it uses is exactly what’s missing from most enterprise evaluations.

The authors obtained access to ADP payroll data, the largest payroll processor in the United States, with monthly records of over 25 million workers. Not surveys, not self-reports, not estimates: granular administrative data on who gets hired, who leaves, how much they earn, in which role, at which company.

They then cross-referenced this data with two AI exposure metrics: one based on theoretical task analysis (which jobs are technically automatable) and one based on actual usage data (how people actually use Claude, Anthropic’s model, in daily work).

The result is an X-ray of AI’s impact with unprecedented granularity. Not the generic “AI is changing work” but precise numbers: employment for software developers aged 22-25 dropped 20% from the late 2022 peak, while for those over 35 in the same roles it grew 8%. In professions where AI use is predominantly substitutive, young workers lose employment; where it’s predominantly augmentative, there’s no decline.

This type of measurement should inform corporate AI decisions. Not because companies need to replicate this exact study, but because it illustrates three principles that most enterprise metrics ignore entirely.

Measure differential effects, not averages

Aggregate data hides more than it reveals. If you only measure “hours saved by AI,” you don’t see who’s saving those hours and who’s losing their job. If you only measure “tickets automated,” you don’t see which customers are receiving worse service.

The Stanford paper shows that AI’s impact differs radically by age group. Workers aged 22-25 in exposed professions saw a 13% employment decline relative to colleagues in less exposed roles. Workers over 30 in the same professions saw growth. The average effect is nearly zero, but the real effect is massive redistribution.

For a CFO, aggregate productivity metrics can mask hidden costs. If AI is increasing output from the senior team while making it impossible to hire and train juniors, the short-term gain could transform into a talent pipeline problem in the medium term. The paper calls it the “apprenticeship paradox”: companies stop hiring entry-level workers because AI handles those tasks better, but without entry-level today there won’t be seniors tomorrow.

The operational consequence is that every AI dashboard should segment impact by role, seniority, team, and customer type. A single “productivity” number is almost always misleading.

Distinguish substitutive from augmentative use

One of the paper’s most relevant findings concerns the difference between substitutive and augmentative AI use. The authors used Anthropic’s data to classify how people actually use language models: to generate final outputs (substitution) or to iterate, learn, and validate (augmentation).

In professions where use is predominantly substitutive, youth employment has collapsed. Where use is predominantly augmentative, there’s no decline; in fact, some of these categories show above-average growth.

Not all “deployments” are equal. A system that automatically generates financial reports substitutes human labor differently from one that helps analysts explore scenarios. Metrics should capture this distinction: classify each AI application as predominantly substitutive or augmentative, separately track impact on headcount, skill mix, and internal training capacity. Augmentative systems might have less immediate ROI but more sustainable effects.

Control for external shocks

One of the Stanford paper’s most sophisticated methodological aspects is its use of firm-time fixed effects. In practice, the authors compare workers within the same company in the same month, thus isolating the AI exposure effect from any other factor affecting the company: budget cuts, sector slowdown, strategy changes.

The result: even controlling for all these factors, young workers in AI-exposed roles show a relative decline of 16% compared to colleagues in non-exposed roles at the same company.

This kind of rigor is rare in corporate evaluations. When an AI project launches and costs drop, it’s easy to credit the AI. But maybe costs would have dropped anyway due to seasonal factors. Maybe the team was already optimizing before the launch. Maybe the comparison is with an anomalous period.

The solution is to define baselines and control groups before launch. Don’t compare “before vs after” but “treated vs untreated” in the same period. Use A/B tests where possible, or at least comparisons with teams, regions, or segments that haven’t adopted AI.

Toward high-frequency economic dashboards

In his predictions for 2026, Brynjolfsson proposed the idea of “AI economic dashboards”, tools that track AI’s economic impact in near real-time, updated monthly instead of with the typical delays of official statistics.

It’s an ambitious proposal at the macro level. But the underlying logic is applicable at the company level: stop waiting for quarterly reports to understand if AI is working and instead build continuous monitoring systems that capture effects as they emerge.

Most AI projects are evaluated like traditional investments: ex-ante business case, periodic reviews, final post-mortem. But AI doesn’t behave like a traditional asset. Its effects are distributed, emergent, often unexpected. A continuous monitoring system can catch drift before it becomes a problem.

In practice, this means working with real-time data instead of retrospective data. If the payroll system can tell you today how many people were hired yesterday in each role, you can track AI’s effect on headcount with a lag of days, not months. The same applies to tickets handled, sales closed, errors detected.

Another key principle: favor leading metrics over lagging ones. The actual utilization rate (how many employees actually use the AI tool every day) is a leading indicator. If it drops, there are problems before they show up in productivity numbers.

As the Stanford paper segments by age, corporate dashboards should segment by role, tenure, and prior performance. AI might help top performers while harming others, or vice versa.

Internal comparisons are also essential: teams that adopted AI vs teams that didn’t, periods with the feature active vs periods with it deactivated. These comparisons are more informative than pure time trends.

The cost of not measuring

There’s a direct economic argument for investing in better measurement. The 42% of companies that abandoned AI projects in 2025 spent budget, time, and management attention only to get nothing. With better metrics, some of those projects would have been stopped earlier. Others would have been corrected mid-course. Others still would never have started.

The MIT NANDA report estimates that companies are spending $30-40 billion per year on generative AI. If 95% generates no measurable ROI, we’re talking about tens of billions burned. Not because the technology doesn’t work, but because it’s applied poorly, measured worse, and therefore never corrected.

The Brynjolfsson paper offers a model of what AI measurement could be. Administrative data instead of surveys. Demographic granularity instead of aggregate averages. Rigorous controls instead of naive comparisons. Continuous monitoring instead of point-in-time evaluations.

No company has Stanford’s resources or access to ADP’s data. But the principles are transferable: segment, distinguish substitutive from augmentative use, control for confounding factors, monitor in real time. Those who adopt these principles will have an information advantage over those who continue tracking deployments and hours saved.

Sources

Brynjolfsson, E., Chandar, B., & Chen, R. (2025). Canaries in the Coal Mine: Six Facts about the Recent Employment Effects of AI. Stanford Digital Economy Lab.

Deloitte AI Institute. (2025). State of Generative AI in the Enterprise. Deloitte.

MIT Project NANDA. (2025). The GenAI Divide 2025. Massachusetts Institute of Technology.

MIT Sloan Management Review. (2024). The Future of Strategic Measurement: Enhancing KPIs With AI. MIT Sloan.

S&P Global Market Intelligence. (2025, October). Generative AI Shows Rapid Growth but Yields Mixed Results. S&P Global.

Three months later, Marco is gone. Luca replaced him. The team still isn’t working.

The scene keeps repeating because the diagnosis is wrong. Not wrong in an obvious way — wrong in a plausible way, which is worse. “Marco isn’t motivated” sounds reasonable. Everyone nods. Too bad the problem isn’t Marco.

Social psychology has a name for this: fundamental attribution error. Ross described it in 1977: when we observe behavior, we tend to attribute it to personal traits (laziness, poor collaboration) while underestimating the weight of the situation the person is in.

J. Richard Hackman spent forty years studying teams: flight crews, orchestras, intelligence squads. The bottom line of his research: structural conditions explain up to 80% of variance in team effectiveness (Wageman, Hackman & Lehman, 2005). When a team isn’t working, the most likely problem isn’t who’s in it. It’s how it was designed.

Before replacing people, check the conditions.

The five conditions: a checklist, not a theory

Hackman didn’t produce an abstract model. He produced a checklist. Five conditions that, when present, make it likely a team will work. When absent, make it likely it won’t. He validated them empirically across hundreds of teams in different contexts. They work as a diagnostic tool long before they work as theory.

Here’s a quick translation into software context. For the full picture, start with the previous piece on Hackman’s team vs. working group distinction.

1. Being a real team. Clear boundaries, stable composition, real interdependence. If even one of these three is missing, you don’t have a team. You have a group of people with a shared manager.

2. A compelling direction. A clear, challenging, meaningful objective. Not “close the sprint stories” — that’s a unit of measurement, not a direction. A compelling direction answers the question: why does this work matter?

3. An enabling structure. Roles, norms, skills. The minimum infrastructure so people don’t have to renegotiate everything from scratch every week.

4. A supportive organizational context. Does the team have the information it needs? The resources? Does the reward system incentivize group outcomes or individual performance? This is the condition the team can’t give itself. It depends on the organization around it.

5. Competent process coaching. Not technical mentoring: someone who helps the team work better together. How they make decisions, handle disagreements, distribute work. It’s the condition that matters least of the five (the 10% in Hackman’s 60/30/10), but it’s the one everyone focuses on.

The instinct when managing teams is to start at the bottom: coaching, facilitation, interpersonal dynamics. Hackman says: start at the top.

The table that flips the diagnosis

This is the core of the argument. Six phrases I hear repeated constantly. For each one: the usual diagnosis, the most likely structural cause, and where to look instead.

Some of these mappings are directly anchored to Hackman’s research. Others are my translation into the software context, and I flag them as such.

“Marco isn’t motivated”

It’s Marco’s problem, people say. He lacks drive, ownership. Maybe he’s not right for the role.

More likely, the team lacks a compelling direction (condition 2). If the team’s objective is vague, purely administrative, or changes every two weeks, motivation isn’t a trait of Marco’s. It’s a rational response to a context that gives no reason to invest energy. I’ve seen brilliant developers seem disengaged because their team’s mandate was “support the product” — which in practice meant answering tickets endlessly with no vision of where things were heading.

Don’t look at Marco. Look at the team’s mandate. If you can’t explain in one sentence why the work matters, that’s your problem.

“Sara doesn’t communicate”

Where to look here is at work design. If everyone works on a separate feature and interactions are limited to the occasional comment on a pull request, there’s no structural reason to communicate more. Sara isn’t uncollaborative. She’s rational: why would she update others on work that doesn’t affect them?

The cause is a lack of real interdependence (condition 1). You can run all the communication workshops you want: if the work structure doesn’t create mutual dependencies, communication will remain an empty ritual.

“The team doesn’t trust each other”

The classic response: team building activities, vulnerability exercises, an offsite to get to know each other better.

Trust in a team doesn’t come from an afternoon workshop. It comes from working together long enough to predict how the other person will behave. Hackman saw this with flight crews: crews that had flown together for a while made fewer errors than newly formed ones, even with less experienced pilots. The most likely problem is unstable composition (condition 1). If you rotate people between teams every quarter, you restart from zero each time.

How many times has the team’s composition changed in the past year? If the answer is “often,” the trust deficit isn’t a relationship problem. It’s structural turnover.

“Retros don’t produce anything”

Here the question comes before the answer: is there a shared process to improve? Retrospectives assume a team that collaborates on a common outcome and wants to improve how they do it. If everyone has their own workflow, priorities, and blockers, the retro has no object. The format is irrelevant. The cause is that the group isn’t a real team (condition 1).

“The lead can’t guide the team”

The usual diagnosis: they lack soft skills. Send them to a leadership course.

The lead might be excellent. But if the team doesn’t have access to the information it needs, if resources get cut without notice, if the evaluation system rewards individual performance and ignores group outcomes, the lead is in an impossible position. It’s like asking a pilot to land well with a poorly designed airplane — you can send them to every flight course there is, but if the flaps don’t work, the problem isn’t their technique.

The cause is the organizational context (condition 4). Does the lead have the authority to make decisions? Are priorities stable enough to allow a plan? If not, the leverage is in the organization, not the person.

“Too much conflict”

Here the answer is less straightforward. Conflict in a team can be a good sign: if the team is real and negotiating norms for working together (condition 3), some friction is physiological. Hackman says it clearly: the best teams aren’t conflict-free. They’re teams that have learned to manage conflict.

But conflict can also signal unclear boundaries (condition 1): who decides what? Who has authority over which part of the system? If two people both think they’re responsible for the same area, the conflict isn’t relational. It’s structural.

The useful distinction: if the conflict is about how to do things, it’s probably healthy. If it’s about who should do what, it’s a boundary problem. And if it’s chronic despite everyone’s best intentions, it’s almost certainly not a people problem.

Why we keep getting the diagnosis wrong

If structural conditions matter this much, why does the default remain “someone’s fault”?

The first reason is that structure is invisible. You see people. You see behaviors. You don’t see the conditions in which those behaviors emerge. Hackman pressed this point in the last paper of his career, “From causes to conditions” (Hackman, 2012): research on teams has focused too much on internal causes (who does what, how they behave) and too little on external conditions (how the team is designed, what context it operates in). The same applies to people managing teams in practice.

The second is that replacing people feels easier. Giving Marco feedback, sending Sara to a workshop, swapping the lead: these are all actions a manager can take tomorrow morning. Redesigning the team’s mandate, stabilizing composition, changing the incentive system — those require time, authority, and often negotiation with someone above. The interpersonal diagnosis is attractive because it has solutions at hand. Too bad they’re the wrong solutions.

The third is more insidious, and I’ve seen it up close: the organization incentivizes individual diagnosis. Performance reviews evaluate people, not conditions. PIPs apply to people. “Cultural fit” is an attribute of people. The entire management apparatus is built around the idea that performance is an individual trait. Admitting the problem is structural means admitting the system is poorly designed — and the person who designed the system is often the one making the diagnosis.

Next time someone says “the problem is Marco,” pause for a moment. Not because Marco is necessarily innocent: sometimes the problem really is the person. But it’s less common than we think, and the interpersonal diagnosis is so intuitive we always get there first.

Try reframing. Not “who isn’t working?” but “what in the structure isn’t working?” Move from person to condition. Then ask yourself: is this condition under my control? If yes, that’s where the energy goes. If not, at least you know that no team building workshop will fix it.

It’s not a small difference. It’s the difference between debugging the code and debugging the compiler.

Sources

• Hackman, J.R. (2002). Leading Teams: Setting the Stage for Great Performances. Harvard Business School Press.
• Hackman, J.R. (2011). Collaborative Intelligence: Using Teams to Solve Hard Problems. Berrett-Koehler.
• Hackman, J.R. (2012). From causes to conditions in group research. Journal of Organizational Behavior, 33, 428-444.
• Wageman, R., Hackman, J.R. & Lehman, E.V. (2005). Team Diagnostic Survey: Development of an Instrument. Journal of Applied Behavioral Science, 41(4), 373-398.
• Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology (Vol. 10, pp. 173-220). Academic Press.

If this sounds familiar, you’re not alone. The 17th State of Agile Report by Digital.ai (2024) found that only 11% of practitioners report being “very satisfied” with Agile practices in their organization. The frustration runs deep enough that two original signatories of the Agile Manifesto have publicly turned against what Agile has become: Ron Jeffries, co-creator of Extreme Programming, wrote that developers should abandon Agile, or at least the version organizations have made of it. Dave Thomas, another signatory, declared that Agile is dead, hollowed out by marketing and mass certification.

The most common diagnosis is that Agile has become bureaucracy: too many rituals, too much process, not enough code. It makes sense. Who hasn’t thought that at least once, walking out of yet another endless planning session?

But there’s another possibility. One that has nothing to do with Agile itself, but with something more fundamental: the structure you’re applying it to.

J. Richard Hackman spent 40 years studying real teams: flight crews, orchestras, intelligence teams, surgical teams. His research, condensed in Leading Teams (2002) and Collaborative Intelligence (2011), arrives at a distinction that almost nobody in software makes: the distinction between a team and a working group. They are different things. They work differently. And they require different tools.

Agile practices are designed for teams. Apply them to a working group and you get exactly the frustration you’re feeling. The problem isn’t the method. It’s a structural mismatch.

The most overused word in software

In software, “team” is the default word for any group of people working on the same project. Five developers sharing a Jira board? Team. Two backend engineers, a frontend dev, and a designer reporting to the same manager? Team. Eight people across three time zones who meet at the 9 AM standup? Team.

Hackman would disagree. In Leading Teams he defines a “real team” through three minimum properties, all required.

The first is clear boundaries: everyone knows who is on the team and who is not. Sounds obvious, but in software practice it isn’t. The designer “shared” across three teams — in or out? The developer “on loan” for two sprints — a member? If you can’t make the list without hesitating, the boundaries aren’t clear.

The second is stable composition. People stay the same long enough to develop shared ways of working. Hackman studied flight crews: NASA data, which he reports in Leading Teams, showed that newly formed crews made more errors than those who had been flying together for a while, even when the newer crews had more experienced pilots. In software, quarterly rotation of people between teams destroys this effect. Every time, you start from zero.

The third, and most underrated, is real interdependence. The team’s output depends on collaboration between members — it’s not the sum of individual contributions. This is where most software “teams” fall apart. Five developers working on five independent features, with five cross-reviews done as a formality, are not interdependent. One person’s work doesn’t change another’s. If you removed all the meetings and put them in separate rooms, the result would be the same.

The test is brutal in its simplicity: if you eliminated standups, retros, and planning tomorrow, and everyone worked on their own, would the final product suffer? If the answer is no — if the result would be identical, maybe even faster without the interruptions — then what you have is not a team. It’s a group of individuals with a shared manager.

That’s not an insult. It’s a diagnosis. And the diagnosis is the first step toward stopping the use of the wrong tools.

Structural conditions: the 60/30/10

Hackman didn’t stop at the definition. He studied what makes a team effective, and the answer is less intuitive than you’d think.

Research by Hackman and his collaborator Ruth Wageman identifies five conditions that enable team performance: being a real team, having a compelling direction, an enabling structure, a supportive organizational context, and competent coaching. I won’t go deep on all of them here — each deserves its own article. The relevant point is different: how much do these conditions matter compared to what a leader does day to day?

Hackman’s answer, laid out in Collaborative Intelligence (2011), is the 60/30/10: 60% of a team’s effectiveness depends on design — the structural conditions put in place before the team starts working. 30% depends on launch — how the team is kicked off, the first days, the initial norms. The remaining 10% on ongoing coaching.

A clarification: Hackman presents this split as a “best estimate,” not as the result of a single study yielding those exact percentages. It’s a heuristic that synthesizes decades of research. Not a precise data point.

But the order of magnitude has solid empirical grounding. The Team Diagnostic Survey (TDS), developed by Wageman, Hackman, and Lehman and published in 2005, was administered to 2,474 people across 321 teams. The finding: structural conditions explain up to 80% of the variance in team effectiveness (Wageman, Hackman & Lehman, 2005). Not 20%. Not 50%. Eighty percent.

An earlier study by Wageman (2001) on 43 self-managing teams at Xerox had already shown the same pattern: a leader’s design activities influenced team performance. Day-to-day coaching activities did not.

The implication for software is direct. When a team isn’t working, the instinctive reaction is to work on dynamics: facilitate retros better, improve communication, do team building. Hackman’s framework suggests the most powerful lever is upstream. Who is on the team? What is the mandate? How is the work designed? Does the organizational context support it? If these conditions aren’t there, no amount of facilitation will compensate.

But the first of those five conditions — “being a real team” — opens a question that almost nobody in software asks.

Team or working group: the distinction that changes everything

A point Hackman makes often, and that gets misunderstood just as often: the distinction between team and working group is not a value judgment. It’s not “team = good, working group = bad.” They are two different organizational modes, each with its own strengths.

A working group is a set of people reporting to the same manager who may coordinate, but whose output is primarily individual. Everyone has their own goals, responsibilities, deliverables. The manager coordinates, assigns, removes obstacles. Deep interdependence is not required.

A team produces collective output. The result cannot be decomposed into the sum of individual contributions: it requires continuous collaboration, shared decisions, mutual adjustment. The cost is higher — you need stable boundaries, shared norms, time to develop ways of working together. But for certain kinds of problems, it’s the only configuration that works.

The damage comes from confusing the two.

A working group managed as a team generates overhead without benefit. Coordination meetings exist, but there’s nothing substantial to coordinate. Retrospectives don’t produce actions because there’s no shared process to improve: everyone has their own workflow, priorities, blockers. The Agile ceremony becomes a fixed cost on work that doesn’t require it.

The damage goes both ways, though. A real team managed as a working group is equally dysfunctional. If you have people who need to collaborate on a complex problem and treat them as individual contributors — assigning separate tasks, evaluating them individually, without protecting time for joint work — you’re undermining the one thing that makes that team effective: interdependence.

I’ve seen both scenarios. The first is more common in software, because the default organization tends to be the working group (developers assigned to individual features), while the process infrastructure is almost always that of a team (Scrum, Kanban with standups, retros, planning).

The operational question isn’t “how do I improve my team?” It’s more fundamental: is what I’m leading a team or a working group? The answer changes everything that follows.

The Agile mismatch: right tools, wrong structure

Back to the frustration we started with. Agile practices — Scrum in particular — didn’t emerge in a vacuum. They’re designed around a specific assumption: that a small group of people works interdependently toward a shared goal, iterating together. The sprint assumes a common goal. The standup assumes that knowing what others are doing changes your work. The retro assumes a shared process to inspect. Planning assumes collective prioritization decisions.

These are all tools that assume interdependence. Without it, they lose their point.

Now look at the typical “team” setup in many software organizations. People are assigned — often rotated — to a “team” that is really an organizational container. Everyone works on their own user story, with interactions limited to code review and the occasional Slack question. The “sprint goal” is the sum of individual stories. Interdependence is minimal or absent.

Apply team tools to this structure and the result is predictable. The standup becomes a round of updates nobody listens to. The retro produces generic complaints. Planning becomes bureaucracy. Not because Scrum is bureaucracy — but because you’re using it on a structure it wasn’t designed for.

The Manifesto signatories say as much, in different words. Jeffries talks about “Dark Scrum” — organizations using Agile rituals as control mechanisms, draining them of collaboration. Thomas says “Agile” was turned into a commercial noun when it was meant to be an adjective describing a way of working. Their critique is legitimate. But the diagnosis they offer — “organizations have corrupted Agile” — is incomplete. Hackman’s framework provides a more structural one: many organizations haven’t corrupted Agile. They’ve applied it to structures that aren’t teams.

I bring this up because I’ve seen it happen more times than I’d like to admit, including in contexts with competent people and good intentions. The standard response to the malaise was always the same: change the facilitator, try a different retro format, add a ceremony. Hackman’s framework gave me the vocabulary for what I felt but couldn’t articulate: the problem wasn’t the execution. It was the premise.

This changes the diagnosis and the solutions. If the problem is “Agile has become bureaucracy,” the answer is less process, fewer rituals, more autonomy. Sometimes that’s right. But if the problem is a structural mismatch, the answer is different: either transform the structure into a real team — with the costs that entails — or accept that you have a working group and adopt tools consistent with that reality. Both options are legitimate. What doesn’t work is staying in the middle.

Back to the twelve-minute standup we started with. Five people, five updates, no interaction. The obvious diagnosis is that the standup is poorly facilitated, or that Scrum doesn’t work, or that the team needs to “work on communication.” These are all responses that address the symptom.

Hackman’s question is different, and it comes first: do those five people need to talk to each other every morning to do their work? If each person’s work doesn’t depend on the others’, the standup isn’t poorly facilitated. It’s useless by design. The retro doesn’t produce actions because there’s no shared process to improve. Planning is bureaucracy because there are no collective decisions to make.

It’s not a question of execution. It’s a question of structure.

Next time you think “Agile doesn’t work,” try reframing. Don’t ask how to improve the process. Ask whether what you’re leading is a team or a working group. The answer isn’t a judgment. It’s a diagnosis. And from the diagnosis, everything else follows.

Sources

• Hackman, J.R. (2002). Leading Teams: Setting the Stage for Great Performances. Harvard Business School Press.
• Hackman, J.R. (2011). Collaborative Intelligence: Using Teams to Solve Hard Problems. Berrett-Koehler.
• Wageman, R. (2001). How Leaders Foster Self-Managing Team Effectiveness: Design Choices Versus Hands-on Coaching. Organization Science, 12(5), 559-577.
• Wageman, R., Hackman, J.R. & Lehman, E.V. (2005). Team Diagnostic Survey: Development of an Instrument. Journal of Applied Behavioral Science, 41(4), 373-398.
• Digital.ai (2024). 17th State of Agile Report.
• Jeffries, R. (2018). Developers Should Abandon Agile.
• Thomas, D. (2014). Agile is Dead (Long Live Agility).

Companies don’t hire juniors because AI does those tasks better. But without junior developers today, who will lead teams in ten years?

There’s a question that rarely appears in quarterly reports: if we stop hiring people who are learning, who will know how to do this job a decade from now?

The numbers tell a story that should concern anyone managing technical teams. Employment for software developers between ages 22 and 25 has dropped 20% from the peak in late 2022, according to a paper from Stanford’s Digital Economy Lab based on payroll data from 25 million workers. It’s not a uniform decline: in the same period, employment for those over 35 in the same roles grew 8%.

The mechanism is what we might call the apprenticeship paradox: companies stop hiring entry-level because AI does those tasks better than a recent graduate. But without entry-level workers today, they won’t have senior engineers tomorrow.

The numbers of collapse

The contraction is not an impression. It’s documented by multiple independent sources.

Entry-level hiring at the top 15 tech companies dropped 25% between 2023 and 2024, according to SignalFire. Since 2021, the average age of technical hires has increased by three years. Companies aren’t just hiring less: they’re hiring differently, preferring senior profiles who can be productive from day one.

Tech internships have collapsed 30% since 2023, according to Handshake. Meanwhile, applications have increased 7%. More people competing for fewer positions, and the remaining positions require increasingly prior experience.

A Harvard study of 285,000 American companies found that when firms adopt generative AI, junior employment drops 9-10% within six quarters. Senior employment remains stable. These aren’t mass layoffs: it’s a silent hiring freeze. Companies simply stop opening entry-level positions.

The pattern repeats in Europe. Junior tech positions have dropped 35% in major EU countries during 2024, based on aggregated data from LinkedIn, Indeed, and Eures. In the UK, the Big Four consulting firms cut graduate hiring between 6% and 29% in two years. In India, IT companies have reduced entry-level roles by 20-25%, according to an EY report.

The World Economic Forum, in its Future of Jobs Report 2025, warns that 40% of employers expect to reduce staffing where AI can automate tasks. And automatable tasks are, almost by definition, the ones junior developers used to do.

The logic of the short term

The rationale behind these choices is understandable. A senior engineer with AI tools can do what previously required two or three juniors, at least for certain tasks. GitHub Copilot, Cursor, and similar tools promise productivity gains of 20-50% according to their vendors. For a CFO looking at the next quarter, hiring a junior who will need six months of training before being productive seems like a difficult investment to justify.

James O’Brien, a computer science professor at Berkeley who works with startups, describes the shift: “Previously, startups would hire one senior person and two or three early-career coders to assist. Now they ask: why hire a recent graduate when AI is cheaper and faster?”

It’s a reasonable question in the short term. Code generated by AI isn’t top quality, but neither is code written by a recent graduate. The difference, O’Brien notes, is that the iterative process to improve AI code takes minutes. A junior might take days for the same task.

Heather Doshay, head of talent at SignalFire, sums it up: “Nobody has the patience or time for hand-holding in this new environment, where much of the work can be done autonomously by AI.”

The problem nobody calculates

There’s a flaw in this logic, and it’s called the talent pipeline.

Matt Garman, CEO of AWS, said it explicitly: “If you don’t have a talent pipeline you’re building, if you don’t have junior people you’re mentoring and growing in the company, we often find that’s where the best ideas come from. If a company stops hiring juniors and developing them, eventually the whole system falls apart.”

It’s not rhetoric. It’s demographic mathematics applied to organizations. Every senior engineer, every tech lead, every CTO was once a junior. The path from recent graduate to technical leader requires years of experience on real projects, mistakes made and corrected, feedback received, patterns internalized. There is no shortcut.

If the industry stops hiring juniors in 2023, by 2033 it will have a structural shortage of mid-level talent. By 2038, there will be a shortage of senior engineers. By 2043, there will be no one to promote to technical leadership roles.

The problem is that this cost doesn’t appear in any quarterly balance sheet. It’s an invisible debt that accumulates silently, and when it becomes obvious, it will be too late to remedy quickly.

AI that teaches and AI that atrophies

There’s a further irony in this situation. The same AI tools that are eliminating junior roles could, in theory, accelerate learning. An AI tutor available 24/7, patient, answering every question: it sounds like every student’s dream.

The reality is more complicated.

An experiment conducted by Wharton and Penn researchers on nearly a thousand high school math students tested two versions of a GPT-4-based tutor. The group with access to a ChatGPT-like interface (GPT Base) achieved 48% better results during assisted practice sessions. The group with a tutor designed to guide without giving direct answers (GPT Tutor) achieved 127% better results.

But here’s the point: when AI was removed and students took the exam on their own, the GPT Base group achieved 17% worse results than the control group who never used AI. The GPT Tutor group, by contrast, achieved results similar to control.

Students were using AI as a crutch. They performed better with assistance but learned less. When the assistance was removed, they found themselves worse off than those who never had it.

A study from MIT Media Lab documented what researchers call “cognitive debt”: using LLMs for writing seems to reduce mental effort during the task, but at the cost of more superficial learning. Researcher Nataliya Kosmyna expressed concern about developing brains: “Developing brains are the ones at highest risk.”

It doesn’t mean AI can’t help learning. The Wharton study shows it can, if designed with the right safeguards. But “wild” AI, the kind that gives answers instead of guiding toward answers, can do damage.

The new profile of the junior

If fewer juniors will be hired, what characteristics must they have to be hired?

Market signals are clear. It’s no longer enough to know how to code. Employers expect recent graduates to be able to manage projects, communicate with clients, understand the software development lifecycle. The “grunt work” that once served as a training ground is being automated. Those entering must be operational at a higher level almost from day one.

Jamie Grant, who manages career services for engineering at the University of Pennsylvania, describes the change: “They’re not necessarily just programming. There’s much more high-level thinking and understanding of the software development lifecycle.”

David Malan of Harvard, who teaches the world’s most-followed introduction to programming course, notes that the biggest impact of AI has been on programmers, not on roles that were expected (like call centers). The reason: programming work is relatively solitary and highly structured, perfect for automation.

But Malan also notes something interesting: in the United States, employment for “programmers” dropped 27.5% between 2023 and 2025, but employment for “software developers,” a more design-oriented position, dropped only 0.3%. The difference is in the level of abstraction. Those who write code are vulnerable. Those who design systems less so.

Three scenarios for the future

Scenario 1: The collapse of the pipeline

Companies continue not to hire juniors. In five to ten years, the shortage of mid-level talent becomes acute. The remaining senior engineers command astronomical salaries. Companies that can’t afford them lose competitiveness. The industry polarizes between a few giants who can attract talent and everyone else struggling.

Scenario 2: Apprenticeship reinvented

Some companies realize the problem is coming and invest against the trend. They create intensive training programs, perhaps assisted by AI designed to teach instead of replace. They become the preferred employers for top talent, who know they can grow there. In the long term, they have a competitive advantage.

Scenario 3: Uneven democratization

AI lowers the barrier to entry for some skills (writing working code) but raises it for others (designing systems, debugging complex problems, managing AI itself). Those with access to quality training and mentorship can skip some steps. Those without remain stuck. Inequality of opportunity increases.

None of these scenarios is inevitable. They are possibilities that depend on choices companies, educational institutions, and policymakers will make in the coming years.

What those who hire can do

If you manage a team or influence hiring decisions, some questions deserve reflection.

Are you optimizing for the next quarter or the next ten years? A junior costs more in the short term. But the alternative is to depend entirely on the external market for talent, competing with everyone else who made the same choice.

Is your team still teaching? If senior people spend all their time producing and no one teaching, you’re consuming human capital without regenerating it.

How do you use AI in training? If your juniors use Copilot to get answers instead of learning to find them, you’re accelerating their short-term productivity while compromising their long-term growth.

Are you hiring for today’s skills or tomorrow’s adaptability? Specific technical skills have an increasingly short half-life. The ability to learn, to reason about new problems, to work with people—those last.

What those starting out can do

If you’re early in your career in a market that seems to close doors on you, some principles can help.

AI isn’t eliminating all junior work. It’s eliminating repetitive, isolated junior work. The roles that survive require human interaction, judgment about ambiguous problems, creativity applied to specific contexts. Look for those.

Learn to use AI as a tool, not a crutch. The difference between using ChatGPT to get answers and using it to explore problems is the difference between atrophying and growing.

Networking matters more than ever. If junior positions are scarce, competition is fierce, and often the person with a connection wins, not the person with the best CV. It’s not fair, but it’s real.

Cross-functional skills are not optional. Communication, project management, understanding the business: these are things AI can’t do and employers seek even in technical profiles.

The unanswered question

I return to the initial question: who will the senior engineers of tomorrow be?

I don’t have a certain answer. No one does. We’re conducting a real-time experiment, without a control group, on a global scale.

What I know is that every senior person I know was once a junior who someone decided to hire and train. Every tech lead made beginner mistakes that someone had the patience to correct. Every systems architect wrote embarrassing code before writing elegant code.

If we eliminate that phase, if we treat it as a cost to cut rather than an investment to protect, we’re not optimizing. We’re consuming capital that we don’t know how to regenerate.

The question isn’t whether AI can replace juniors. It can, for many tasks. The question is whether we want an industry that only knows how to consume skills or one that also knows how to produce them.

For now, the numbers suggest we’ve chosen the first option. The bill will come. Not next quarter. But it will come.

Sources

Brynjolfsson, E., Chandar, B., & Chen, R. (2025). Canaries in the Coal Mine: Six Facts about the Recent Employment Effects of AI. Stanford Digital Economy Lab.

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2024). Generative AI Can Harm Learning. The Wharton School Research Paper.

Stack Overflow. (2025, December). AI vs Gen Z: How AI has changed the career pathway for junior developers. Stack Overflow Blog.

IEEE Spectrum. (2025, December). AI Shifts Expectations for Entry Level Jobs.

Rest of World. (2025, December). AI is wiping out entry-level tech jobs, leaving graduates stranded.

Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv.

World Economic Forum. (2025). Future of Jobs Report 2025.

FinalRound AI. (2025). AWS CEO Shares 3 Solid Reasons Why Companies Shouldn’t Replace Juniors with AI Agents.