Trends in Developer Jobs: A Meta Analysis of Stack Overflow Surveys

I'm really interested in the trends we see in the software engineering job market. Sometimes it's really hard to tell a cohesive and accurate narrative about what's happening because it just happens so dang fast, and very few people are collecting data on the matter.

For example, here are some questions that I'd like to know the answer to:

• Does "DevOps" and increased cloud usage mean fewer traditional ops jobs?
• Do things like Firebase and Supabase mean fewer back-end jobs?
• Do things like tailwind mean fewer designers?
• Are we moving into more specialist or more generalist roles?
• Are people still specializing as DBAs?

I aggregated all the answers about developer employment from the last 9 years of Stack Overflow surveys in order to start being able to answer some of these questions. If you're curious about the job outlook for developers, hopefully this is helpful data.

First, a note on the data

1. All the numbers in the tables represent the percentage of survey takers who identified themselves as a given job type.
2. The questions asked by SO have changed over the years, so we should take that as a giant grain of salt.
3. I've normalized the answers. For example, "developer, back-end" and "back-end web developer" I've grouped together as "backend".
4. Stack overflow allowed people to select 0-many job titles after 2015, before that, it appears they were limited to one.
5. In some years developers had more job options to pick from.
6. I've thrown away a lot of off-topic job types like "enterprise services developer" and "elected official".
7. This is only Stack Overflow data, so it's biased by the size of their user base each year.

That said, let's get into the data. I'll drop the script I wrote to create the aggregate data as well as a link to the raw data at the bottom of this article.

Web Trends: Full-Stack, Front-end and Back-end

In 2013 the ratio of "front-end" developers to "full stack" developers was 15/85, while the ratio of "back-end" developers to "full stack" developers was 24/76.

The trend of "more full stack, less specialization" continues through the entire ten years of data. It's important to note that in 2017/2018 the survey changed drastically, which is why we see a big change in the numbers.

Interpretation #1

It seems reasonable to conclude that the trend for the last decade has been that there are a shrinking percentage of developers who do only front-end work or only back-end work. More and more are doubling as full-stack engineers. Keep in mind, this also might be that there have been more small to mid-sized companies over the years. Smaller companies typically require more generalists, but now I'm just guessing.

Interpretation #2

The ratio of front-end/back-end engineers has stayed fairly consistent, with there being slightly less than twice as many back-end engineers than front-end engineers.

This actually surprised me, I expected the ratio of front-end to back-end engineers to be closer throughout the years.

IT Operations Trends: Back-end, DevOps and Traditional Ops

In my script I tried to split more "traditional ops" roles into the "ops" category, and the "devops" stuff into the "devops" role. For example, "SRE" I've considered as "devops", while "systems administrator" is "ops".

Caveat on the data for 2017

What the hell happened in 2017? The data honestly just seems broken. I dug through the data manually because according to the website, they claim 24% of web developers said they were back end, and 75% of respondents claimed to be web developers. At the moment it's looking to me like they must have had some qualifiers on those numbers, because it's just not adding up on my end. I'm going to exclude 2017 from my analysis in the interpretations.

Interpretation 1

DevOps appears to be gaining ground on traditional ops. In 2013 no one was identifying themselves as a "devops" person, but by 2020 and 2021 the numbers are looking very similar. It's worth pointing out that in 2016 the devops numbers actually eclipsed the "ops" numbers for a single year. My best guess is that 2016 was about when many companies started simply rebranding their "ops" teams as "devops" teams in order to look cool.

It's hard to care too much about these numbers, because devops is mostly being done wrong in my opinion. I don't trust that the "ops" titles and the "devops" titles are all that different at most companies.

Interpretation 2

"Devops" seems to be trending down the least in recent years, in fact it had a nice jump in 2022. However, if you look at "devops" and "ops" together, then the category is still trending down a bit. Interestingly "ops" has been trending down from the start, while "back-end" was trending up until 2016 when the trend reversed and it's been down since.

At first, I assumed we're simply seeing the same trend that we saw in web development: more generalists, less specialists. However, I grew skeptical because when I looked across all job categories, I noticed nearly all of them were trending down... which clearly can not be the case when we're looking at percentages - it's a zero sum game. I decided to add a new section to my script to dig in further. I calculated how many jobs on average each survey taker was laying claim to, and got this data:

It appears that from 2013-2015, developers were restricted to only submitting a single answer, which helps to account for the super low numbers. However, from 2019->2021 the average number of jobs per user went down, which is antithetical to the "more generalists" theory. It's also worth pointing out, that as the years went on, Stack OVerflow actually added more specialized categories, which I then took the liberty of grouping into these broader groups. So there's actually good evidence that developers are specializing more, or at the very least that there are more possible ways in which one can specialize.

That said, even after looking at this data I think there is still a good case to be made that back-end developers will be doing more and more "devops" work, especially at smaller companies.

Data trends: Data science, data engineering, and back-end

Interpretation

It's super interesting to me that data engineering really only started to appear in the survey data in 2019. Until then, I'm guessing that role was swallowed up by back-end engineers and data scientists. That new specialization is certainly interesting. Machine learning has absolutely grown over the last decade, but it looks like there may have been a bit of a "hype bubble" in 2017?

The Rest of the Data

I've talked about my personal interpretations regarding the data that I found most the interesting, but here's all the data I aggregated so you can inspect it yourself:

Raw CSV Data

Here's a link to the raw CSV data on Stack Overflow.

My Script

Here's the full Python script I used to crunch the numbers. Sorry for the code sloppiness, I didn't sink a ton of time into the code. The most interesting part of the script is probably the get_mapped_job function near the bottom, that's where I aggregate all of the many job types reported by stack overflow users into the few I included in the chart.

import csv

outpath = "csv/out.csv"

type_devops = "devops"
type_ops = "ops"
type_backend = "backend"
type_frontend = "frontend"
type_mobile = "mobile"
type_fullstack = "fullstack"
type_desktop = "desktop"
type_embedded = "embedded"
type_data_science = "data_science"
type_ignore = "ignore"
type_management = "management"
type_education = "education"
type_design = "design"
type_marketer = "marketer"
type_data_engineer = "data_engineer"
type_game = "game"
type_analyst = "analyst"
type_qa = "qa"


def main():
    files = [
        (2013, [6]),
        (2014, [6]),
        (2015, [5]),
        (2016, [8, 9, 10]),
        (2017, [14, 15, 16, 17]),
        (2018, [9]),
        (2019, [12]),
        (2020, [13]),
        (2021, [11]),
        (2022, [11]),
    ]

    out_dict = {}
    jobs_per_user_dict = {}

    for f_tup in files:
        counts = {}
        path = f"csv/{f_tup[0]}.csv"
        print(f"generating report for {path}")
        out_dict[f_tup[0]]: {}
        with open(path, "r") as csvfile:
            rows = csv.reader(csvfile, delimiter=",")
            count = 0
            rows_cpy = []
            jobs_per_user = []
            for row in rows:
                count += 1
                rows_cpy.append(row)
            for row in rows_cpy:
                try:
                    jobs = get_jobtext_from_cells(f_tup[1], row)

                    mapped_jobs = set()
                    for job in jobs:
                        mapped_jobs.add(get_mapped_job(job))
                    jobs_per_user.append(mapped_jobs)
                    for mapped_job in mapped_jobs:
                        if mapped_job not in counts:
                            counts[mapped_job] = 0
                        counts[mapped_job] += 1
                except Exception as e:
                    print(e)

            avg_jobs_per_user = 0
            for user_jobs in jobs_per_user:
                avg_jobs_per_user += len(user_jobs)
            jobs_per_user_dict[f_tup[0]] = round(
                avg_jobs_per_user / len(jobs_per_user), 2
            )

            for job in counts:
                counts[job] /= count
                counts[job] *= 100
                counts[job] = round(counts[job], 2)
            out_dict[f_tup[0]] = counts

    write_out(out_dict, jobs_per_user_dict)


def get_jobtext_from_cells(indexes, row):
    if len(indexes) == 0:
        return []
    job_texts = []
    for i in indexes:
        cell = row[i]
        cell_job_texts = cell.split(";")
        job_texts += cell_job_texts
    return job_texts


def write_out(out_dict, jobs_per_user_dict):
    types = [
        type_fullstack,
        type_frontend,
        type_backend,
        type_devops,
        type_ops,
        type_mobile,
        type_desktop,
        type_embedded,
        type_data_science,
        type_data_engineer,
        type_game,
        type_management,
        type_qa,
        type_education,
        type_design,
        type_analyst,
        type_marketer,
        type_ignore,
    ]

    with open(outpath, "w") as csvfile:
        w = csv.writer(csvfile)
        w.writerow(["year", "avg_jobs_per_user"] + types)
        for year in out_dict:
            row = [year, jobs_per_user_dict[year]]
            for t in types:
                row.append(out_dict[year][t] if t in out_dict[year] else 0)
            w.writerow(row)


def get_mapped_job(job):
    job = job.lower().strip()
    if job == "":
        return type_ignore
    if job == "devops specialist":
        return type_devops
    if job == "designer":
        return type_design
    if job == "c-suite executive":
        return type_management
    if job == "analyst or consultant":
        return type_analyst
    if job == "back-end developer":
        return type_backend
    if job == "windows phone":
        return type_mobile
    if job == "i don't work in tech":
        return type_ignore
    if job == "growth hacker":
        return type_marketer
    if job == "desktop developer":
        return type_desktop
    if job == "analyst":
        return type_analyst
    if job == "executive (vp of eng., cto, cio, etc.)":
        return type_management
    if job == "mobiledevelopertype":
        return type_mobile
    if job == "engineer, data":
        return type_data_engineer
    if job == "graphics programmer":
        return type_game
    if job == "systems administrator":
        return type_ops
    if job == "developer, game or graphics":
        return type_game
    if job == "desktop software developer":
        return type_desktop
    if job == "nondevelopertype":
        return type_ignore
    if job == "elected official":
        return type_ignore
    if job == "engineering manager":
        return type_management
    if job == "web developer":
        return type_fullstack
    if job == "machine learning specialist":
        return type_data_science
    if job == "data or business analyst":
        return type_analyst
    if job == "devtype":
        return type_fullstack
    if job == "response":
        return type_ignore
    if job == "developer, qa or test":
        return type_qa
    if job == "machine learning developer":
        return type_data_science
    if job == "developer, front-end":
        return type_frontend
    if job == "database administrator":
        return type_ops
    if job == "android":
        return type_mobile
    if job == "webdevelopertype":
        return type_fullstack
    if job == "blackberry":
        return type_mobile
    if job == "system administrator":
        return type_ops
    if job == "mobile developer - android":
        return type_mobile
    if job == "developertype":
        return type_fullstack
    if job == "ios":
        return type_mobile
    if job == "developer with a statistics or mathematics background":
        return type_ignore
    if job == "qa or test developer":
        return type_qa
    if job == "educator or academic researcher":
        return type_education
    if job == "engineer, site reliability":
        return type_devops
    if job == "marketing or sales professional":
        return type_marketer
    if job == "student":
        return type_ignore
    if job == "back-end web developer":
        return type_backend
    if job == "educator":
        return type_education
    if job == "front-end developer":
        return type_frontend
    if job == "developer, desktop or enterprise applications":
        return type_desktop
    if job == "senior executive/vp":
        return type_management
    if job == "occupation":
        return type_ignore
    if job == "scientist":
        return type_ignore
    if job == "developer, full-stack":
        return type_fullstack
    if job == "graphic designer":
        return type_design
    if job == "developer, embedded applications or devices":
        return type_embedded
    if job == "embedded application developer":
        return type_embedded
    if job == "quality assurance":
        return type_qa
    if job == "graphics programming":
        return type_game
    if job == "senior executive (c-suite, vp, etc.)":
        return type_management
    if job == "it staff / system administrator":
        return type_ops
    if job == "business intelligence or data warehousing expert":
        return type_data_engineer
    if job == "full stack web developer":
        return type_fullstack
    if job == "developer, mobile":
        return type_mobile
    if job == "front-end web developer":
        return type_frontend
    if job == "desktop applications developer":
        return type_desktop
    if job == "other (please specify):":
        return type_ignore
    if job == "mobile developer":
        return type_mobile
    if job == "devops":
        return type_devops
    if job == "enterprise level services developer":
        return type_ignore
    if job == "data scientist":
        return type_data_science
    if job == "executive (vp of eng, cto, cio, etc.)":
        return type_management
    if job == "mobile developer - ios":
        return type_mobile
    if job == "game or graphics developer":
        return type_game
    if job == "which of the following best describes your occupation?":
        return type_ignore
    if job == "other":
        return type_ignore
    if job == "desktop or enterprise applications developer":
        return type_desktop
    if job == "c-suite executive (ceo, cto, etc.)":
        return type_management
    if job == "embedded applications/devices developer":
        return type_embedded
    if job == "product manager":
        return type_ignore
    if job == "mobile application developer":
        return type_mobile
    if job == "mobile developer - windows phone":
        return type_mobile
    if job == "data scientist or machine learning specialist":
        return type_data_science
    if job == "educator or academic":
        return type_education
    if job == "embedded applications or devices developer":
        return type_embedded
    if job == "quality assurance engineer":
        return type_qa
    if job == "enterprise level services":
        return type_ignore
    if job == "full-stack developer":
        return type_fullstack
    if job == "na":
        return type_ignore
    if job == "academic researcher":
        return type_education
    if job == "manager of developers or team leader":
        return type_management
    if job == "marketing or sales manager":
        return type_marketer
    if job == "developer, back-end":
        return type_backend
    if job == "full-stack web developer":
        return type_fullstack
    if job == "designer or illustrator":
        return type_design
    if job == "programmer":
        return type_ignore
    if job == "developer":
        return type_ignore
    if job == "manager":
        return type_management
    if job == "engineer":
        return type_ignore
    if job == "sr. developer":
        return type_ignore
    if job == "full stack overflow developer":
        return type_fullstack
    if job == "ninja":
        return type_ignore
    if job == "mobile dev (android, ios, wp & multi-platform)":
        return type_mobile
    if job == "expert":
        return type_ignore
    if job == "rockstar":
        return type_ignore
    if job == "hacker":
        return type_ignore
    if job == "guru":
        return type_ignore
    if job == "self_identification":
        return type_ignore
    if job == "occupation_group":
        return type_ignore
    if job == "cloud infrastructure engineer":
        return type_devops
    if job == "project manager":
        return type_management
    if job == "security professional":
        return type_ops
    if job == "blockchain":
        return type_backend
    if (
        job
        == "mathematics developers (data scientists, machine learning devs & devs with stats & math backgrounds)"
    ):
        return type_data_science
    raise Exception(f"job not mapped: {job}")


main()