The Skill Disruption Index

Note: The research referenced in this report is based on data from 2016-2021, but new research covering 2021-2024 is now available. Read "The Speed of Skill Change"

Jobs in today’s dynamic labor market keep evolving and adding new requirements—the Skill Disruption Index is how you can measure that rate of change. 

Shifting Skills, Moving Targets, and Remaking the Workforce, a new report from Lightcast, The Burning Glass Institute and Boston Consulting Group (BCG) introduces the index and uses it to show the dramatic disruption that has occurred since 2016: 37% of the top 20 skills requested for the average job have changed over the past five years, and one in five skills is entirely new.

Understanding  how and how fast jobs change is vital for employers and employees to stay relevant in the short term and get ahead moving forward. The Skill Disruption Index makes this possible by creating an accessible and data-driven method of comparing how rapid and significant changes are in specific roles. Here’s how it works.

Lightcast data consists of millions of online job postings that are scanned every day and analyzed using artificial intelligence technologies. The SDI is sourced from 15 million job postings that were listed from 2016 to 2021, sorted into 680 specific jobs according to the proprietary Lightcast Occupation Taxonomy. 

By comparing the skills requested by postings in each occupation over time, researchers can see which jobs stayed the same and which had changed. The Skill Disruption Index was built by calculating the absolute value of the difference in demand between 2016 and 2021. The resulting differences for each occupation were then normalized on a scale of 100 to 0, and a score for each of the 680 occupations was calculated within that range.

The occupation that changed the most was Data Engineer. New skills requested in this role since 2016 include Microsoft Power BI and artificial intelligence, and the skills that increased most in importance for this role are technical in nature. Skills that declined in importance were also specific technology capabilities, such as Ruby and the PERL scripting language.

The high level of skill disruption for the Data Engineer occupation is part of a larger trend that showed dramatic change among several jobs in IT-related fields.

The Skill Disruption Index also takes into account the relative importance of different skills to a given occupation. This includes how broadly a skill is in demand, as well as whether employers are willing to pay a premium to workers who have it. Skills in Microsoft Office, for example, are needed across many fields, but unlikely to result in a salary premium in the way that a coding language like Python would. Social media skills are also being requested across a wide range of occupations– one of the report’s major takeaways.

Why It Matters

While it feels like the world of work has been in a constant state of flux over the past several years, particularly in the two since the Covid-19 pandemic hit, the Skill Disruption Index provides a way to quantify that change and provide clear, actionable insights moving forward for employers and employees alike. Big data and analytics—the tools underpinning the Shifting Skills report—can create new understanding of which skills are emerging, declining, and changing in importance over time.

In a tight and complex job market, understanding trends in skill change is vital for staying competitive and getting ahead. To learn about those patterns—and see the full Skill Disruption Index for all 680 occupations—download the free Shifting Skills report from Lightcast, The Burning Glass Institute, and BCG here.