Most claims about how to build software are opinions wearing the costume of fact. "Pair programming catches more bugs." "Microservices scale better." "This linter improves quality." Each sounds authoritative, and each is testable. Empirical software engineering is the discipline that does the testing: it studies how software is built, maintained, and used by collecting real data and analyzing it, instead of trusting whoever argues most confidently.

Think of it like the shift medicine made a century ago. Doctors once prescribed treatments because a respected mentor swore by them. Then medicine started running trials, tracking outcomes, and pooling results across studies. Software engineering is partway through the same shift, and understanding how it works lets you tell a real finding from a well-dressed guess.

## Prerequisites

Before reading, be comfortable with the software development lifecycle (writing code, reviewing it, shipping it, maintaining it) and with basic descriptive statistics such as averages, distributions, and the idea that correlation is not causation.

You do not need to be a researcher or a statistician. You need enough curiosity to ask "how would anyone actually know that?" when someone states a strong claim about tools, teams, or practices. If you want the statistics groundwork first, read [Fundamentals of Statistics][stats] before continuing.

By the end of this article, you will understand how empirical software engineering turns questions about software into evidence, and why the same result can be trustworthy in one study and worthless in another.

> **At a glance:** By the end, you will be able to judge whether a claim about software practice rests on evidence, and spot the flaws that make weak evidence look strong.

**Learning Outcomes**
By the end of this article, you will be able to:

* Explain **why** empirical methods beat expert opinion for many software questions, and when they do not.
* Describe the main **research designs** (experiments, case studies, repository mining, surveys) and what each can and cannot show.
* Explain **why measurement is the hard part**, and how construct validity decides whether a metric means anything.
* Identify the four kinds of **validity threats** and read a study for its weakest link.
* Recognize the **laws, biases, and fallacies** that make software evidence misleading.

![Cover: questions flowing into data collection, analysis, and validated evidence](fundamentals-of-empirical-engineering.png)

> Type: **Explanation** (understanding-oriented).  
> Primary audience: **beginner to intermediate** developers, leads, and managers who want to reason about evidence, not run studies.

**Jump to:** [What It Is](#section-1-what-empirical-software-engineering-is--evidence-over-authority) • [Research Designs](#section-2-research-designs--how-questions-become-studies) • [Measurement](#section-3-measurement--the-hard-part) • [Validity](#section-4-threats-to-validity--why-a-result-might-be-wrong) • [From Evidence to Practice](#section-5-from-evidence-to-practice--ebse) • [Common Mistakes](#section-6-common-mistakes) • [Misconceptions](#section-7-common-misconceptions) • [Laws, Bias, and Fallacies](#laws-bias-and-fallacies) • [Glossary](#glossary)

If you only want to become a sharper reader of claims, focus on Section 3 and Section 4. If you are trying to run a small study on your own team, read all sections in order.

### TL;DR – Empirical Software Engineering in One Pass

If you remember one idea, make it this: a claim about software is only as good as the way it was measured and the way confounders were ruled out.

* **Start from a question, not a metric,** so you collect data that maps to a decision.
* **Match the design to the claim,** because only some designs can support "X caused Y."
* **Interrogate the measure,** since most software metrics are proxies that drift from what they claim to capture.
* **Name the threats to validity,** because honest studies say why they might be wrong.

The empirical workflow looks like this:

```mermaid
graph TB
    A[Question<br/>what do you want to know?] --> B[Design<br/>experiment, case study, mining, survey]
    B --> C[Measure<br/>operationalize the concept]
    C --> D[Collect Data<br/>repos, telemetry, subjects]
    D --> E[Analyze<br/>quantitative and qualitative]
    E --> F[Assess Validity<br/>what could make this wrong?]
    F --> G[Evidence<br/>hold belief in proportion]
    G -.replicate.-> A

    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style C fill:#e8f5e8
    style D fill:#e8f5e8
    style E fill:#fff3e0
    style F fill:#ffebee
    style G fill:#e1f5fe
```

## Section 1: What Empirical Software Engineering Is – Evidence Over Authority

Empirical software engineering is the branch of the field that studies software development as it actually happens, using systematic data collection and analysis rather than intuition, tradition, or the loudest voice in the room. The word *empirical* means "based on observation," and that is the whole stance: look at what real teams and systems do, then reason from the observations.

Its subject is broad. It covers how code is written and reviewed, how defects arise and get fixed, how teams estimate and plan, how tools change behavior, and how users interact with what ships. Anything in the making, maintaining, and using of software is fair game if you can observe it.

### Why authority is not enough

Software has strong opinions and weak feedback loops. A senior engineer who has shipped for twenty years has seen a lot, but they have also seen a biased sample: the projects they happened to work on, remembered through the projects that went well or badly enough to stick. Expertise is real, and it is also a small, non-random dataset with a faulty recorder attached.

Empirical methods do not discard that expertise. They test it. When a strong intuition survives a careful study, you get to keep the intuition and the evidence. When it does not survive, you learned something that no amount of confidence would have told you.

### What this is and isn't

This article explains the principles and reasoning of empirical software engineering: why it exists, how studies are designed, and how to judge the evidence they produce. It focuses on making you a better consumer and a cautious producer of software evidence.

It does not teach you to run rigorous statistical analyses, design a publishable experiment, or use a specific tool. Those are deep skills; treat this as the map that tells you which territory to explore next.

### Quick Check: The Core Stance

Before moving on, test your understanding:

* Why is a veteran engineer's memory a biased dataset rather than a neutral record?
* What does "empirical" commit you to that "best practice" does not?
* Name one claim you currently believe about software that you have never seen tested.

**Answer guidance:** **Ideal result:** you can explain that memory is a small, non-random, selectively recalled sample, that "empirical" commits you to observation and to being willing to be wrong, and you can name at least one untested belief without flinching. If the last one is hard, that is the point of the discipline.

## Section 2: Research Designs – How Questions Become Studies

A research design is the plan that connects a question to data. The design you choose decides what you are allowed to conclude, so picking it is not a formality. The central divide is between designs that can support a causal claim ("X caused Y") and designs that can only support an associational one ("X and Y appear together").

### Controlled experiments

A controlled experiment manipulates one factor, holds the rest fixed, and randomly assigns subjects to conditions. Randomization is the magic: it makes the groups equivalent on average for every factor you did not think of, so a difference in the outcome can be pinned on the treatment. This is the only design that cleanly supports causal claims.

The catch is realism. To control everything, experiments often shrink the task to something small and the subjects to whoever is available, frequently students. A result that holds for students writing a fifty-line program may not hold for staff engineers changing a million-line system.

### Online controlled experiments (A/B tests)

Large software organizations run controlled experiments on live users constantly. An A/B test splits real traffic between variants and compares an outcome, such as conversion or latency. It keeps the causal power of randomization while running at full production scale, which is why it has become the dominant evidence engine inside product companies.

### Case studies

A case study investigates one project, team, or system in depth, in its real context. It cannot prove that a practice causes an outcome, because no controlled comparison exists, but it can explain *how* and *why* something happened with a richness that no experiment matches. Case studies are how the field understands messy, situated phenomena that do not fit a lab.

### Mining software repositories

Every commit, pull request, issue, and build log is data already recorded. Mining software repositories (MSR) analyzes this exhaust at scale to study real development across thousands of projects. The data is abundant and genuinely real, which is its strength. Its weakness is that the data was recorded for operations, not research, so what looks like a clean signal is often an artifact of how a tool logs events.

### Surveys and qualitative methods

Surveys and interviews capture what practitioners believe, feel, and report doing, which is the only way to reach attitudes and motivations. Self-report limits them: people misremember, rationalize, and answer the question they wish you had asked. Qualitative methods such as grounded theory build explanations from this kind of data when no existing theory fits.

### Choosing a design

**Controlled experiment**
Best for causal claims on a narrow question. Strong internal validity, often weak external validity.
--card--
**A/B test**
Causal claims at production scale on live users. Limited to what you can safely vary in production.
--card--
**Case study**
Deep "how and why" in real context. No causal proof, high situated insight.
--card--
**Repository mining**
Large-scale patterns across many real projects. Association only, and data is an operational artifact.
--card--
**Survey / interview**
Attitudes, beliefs, and self-reported practice. Limited by memory and honesty.
### Quick Check: Matching Design to Claim

Test your understanding:

* Someone says "teams that use trunk-based development ship faster" based on repository mining. What can they actually conclude?
* Why can an A/B test claim causation when repository mining cannot?
* When would a single case study beat a large-scale mining study?

**Answer guidance:** **Ideal result:** you note that mining shows association, not causation (faster teams might adopt the practice for other reasons), that the A/B test randomizes and mining does not, and that a case study wins when you need to understand mechanism rather than measure prevalence.

## Section 3: Measurement – The Hard Part

The hardest problem in empirical software engineering is not statistics. It is deciding what to measure. Almost everything worth studying, such as productivity, quality, complexity, or maintainability, is abstract, and you cannot collect an abstraction. You collect a proxy and hope it tracks the real thing.

### Operationalization and its gap

Turning a concept into a concrete measurement is called operationalization. "Developer productivity" becomes "commits per week." "Code quality" becomes "defects per thousand lines." Each substitution opens a gap between the concept and the number, and that gap is where nearly every measurement failure lives.

Commits per week is a case in point. It rewards many tiny commits over one careful change, ignores review and design work entirely, and drops to zero during the week someone untangles the gnarliest bug in the system. The number is easy to collect and easy to game, and it barely means what it claims.

### Construct validity

Construct validity asks the blunt question: does this measure actually capture the concept it names? A metric with poor construct validity is worse than no metric, because it looks like evidence while pointing the wrong way. Before trusting any software number, ask what concept it stands for and how far the proxy could drift from it.

### Goal-Question-Metric

Good measurement runs top-down, not bottom-up. The Goal-Question-Metric (GQM) framework starts from a goal, refines it into questions you could actually answer, and only then chooses metrics that answer those questions. This ordering is the antidote to the common failure of collecting whatever is easy to log and reverse-engineering a story from it.

```mermaid
graph TB
    G[Goal<br/>reduce escaped defects] --> Q1[Question<br/>where do defects escape?]
    G --> Q2[Question<br/>which changes are riskiest?]
    Q1 --> M1[Metric<br/>defects by discovery phase]
    Q2 --> M2[Metric<br/>change size and coupling]

    style G fill:#e1f5fe
    style Q1 fill:#f3e5f5
    style Q2 fill:#f3e5f5
    style M1 fill:#e8f5e8
    style M2 fill:#e8f5e8
```

For a deeper treatment of what makes a metric useful or corrosive, read [Fundamentals of Metrics][metrics], and for the analysis side, [Fundamentals of Data Analysis][data-analysis].

### Quick Check: Reading a Metric

Test your understanding:

* A dashboard measures "quality" as the number of open bugs. Why might that number fall while quality also falls?
* What does GQM force you to do before you pick a metric?
* Give one reason "lines of code changed" is a poor proxy for effort.

**Answer guidance:** **Ideal result:** you can explain that open bugs fall when people stop filing them or close them without fixing, that GQM forces you to state a goal and question first, and that lines changed rewards verbose edits and ignores the hard, small changes.

## Section 4: Threats to Validity – Why a Result Might Be Wrong

Every empirical study is wrong somehow; the honest ones tell you how. "Threats to validity" is the standard vocabulary for the reasons a study's conclusions might not hold. Reading a study for its threats, rather than its headline, is the single most useful habit in this discipline. Threats come in four kinds.

### Construct validity

Covered above: does the measurement capture the intended concept? If the study measured productivity by commit count, its findings about "productivity" inherit every flaw of that proxy.

### Internal validity

Internal validity asks whether the studied factor actually caused the observed effect, or whether a confounder did. Suppose a study finds teams using a new code review tool have fewer defects. If those teams are also the most experienced, experience is a confounding variable, and the tool may deserve none of the credit. Randomization defends internal validity; its absence demands that every confounder be argued away.

### External validity

External validity asks whether the result generalizes beyond the exact subjects and setting. A finding from computer science undergraduates on a two-hour task in a lab may say little about senior engineers on a legacy system under deadline. Neither setting is wrong; the error is assuming one transfers to the other without argument.

### Conclusion validity

Conclusion validity asks whether the statistical reasoning holds: was the sample large enough, were the tests appropriate, is the "significant" difference actually meaningful. A p-value below 0.05 on a tiny sample, or a statistically significant effect too small to matter, both fail here.

### Why replication matters

Because any single study carries all four threats, one study should move your belief only a little. Confidence comes from replication: independent studies, in different settings, that keep finding the same thing. Software engineering has a thin replication record compared with older sciences, which is exactly why you should treat a lone dramatic result with friendly suspicion.

### Quick Check: Spotting the Weak Link

Test your understanding:

* A study claims a language feature reduces bugs, based on student volunteers. Which validity threat is loudest?
* Two variables move together in a repository mining study. Which validity type is most at risk?
* Why does one striking study justify curiosity but not a rollout?

**Answer guidance:** **Ideal result:** you flag external validity for the student sample, internal validity (confounding) for the mining correlation, and you explain that a single study carries all four threats, so replication is what earns a decision.

## Section 5: From Evidence to Practice – EBSE

Evidence-based software engineering (EBSE) is the applied side: using the best available research evidence, weighed against your local context, to make decisions. The idea is borrowed directly from evidence-based medicine, where clinicians combine research findings with the specifics of the patient in front of them.

### Systematic reviews over single studies

EBSE leans on systematic literature reviews rather than individual papers. A systematic review searches the literature with a documented, reproducible protocol, so it cannot quietly cherry-pick the studies that agree with the author. When you want the field's current best answer to a question, a good systematic review beats any single study, including the exciting one making the rounds.

### Evidence plus context

Evidence never makes the decision for you. A practice that works in a study of large distributed teams may be wrong for your four-person startup. EBSE pairs the external evidence with internal knowledge of your constraints, culture, and goals. The failure mode on both sides is treating one input as the whole answer: cargo-culting a study, or dismissing all research because "the situation here is different."

Industry frameworks such as the DORA (DevOps Research and Assessment) metrics for delivery performance and the SPACE framework for developer productivity are attempts to bring this discipline into everyday practice, giving teams validated constructs instead of homegrown proxies. Related engineering practices that generate this kind of evidence appear in [Fundamentals of Monitoring and Observability][observability] and [Fundamentals of Software Testing][testing].

### Quick Check: Applying Evidence

Test your understanding:

* Why does EBSE prefer a systematic review to the single most-cited paper on a topic?
* What is the failure mode of applying a study's result without weighing local context?
* Why is "the situation here is different" both sometimes true and often an excuse?

**Answer guidance:** **Ideal result:** you can explain that systematic reviews reduce cherry-picking, that ignoring context leads to cargo-culting, and that local difference is a real moderator of external validity but is also the easiest way to dodge inconvenient evidence.

## Section 6: Common Mistakes

Common mistakes turn honest measurement into misleading numbers. Recognizing them protects you whether you are running a study or reading one.

### Measuring what is easy instead of what matters

The most common failure is starting from available data instead of a question. Commit counts, lines of code, and ticket velocity are easy to collect, so teams collect them and then invent a meaning. GQM exists precisely to reverse this order.

### Confusing correlation with causation

A repository mining study finds that projects with more tests have fewer bugs. The tempting conclusion is "write more tests to get fewer bugs." The honest one is that careful teams both write more tests and produce fewer bugs, and the tests may be a symptom of care rather than its cause. Without randomization, you have an association, not a lever.

### Ignoring the denominator

"This tool found 200 bugs" means nothing without knowing how many existed and how many the old process caught. Raw counts hide rates. A number without its denominator is a number designed to impress, not inform.

### Trusting a single study

One study, however clean, carries all four validity threats. Treating a lone result as settled fact, especially a surprising one, is how the field accumulates confident folklore. Ask whether anyone has replicated it.

### Quick Check: Catching Mistakes

Test your understanding:

* A team celebrates that "code coverage rose to 90% and bugs dropped." What two mistakes might be hiding here?
* Why is a raw defect count nearly meaningless on its own?
* What question defuses most correlation-as-causation claims?

**Answer guidance:** **Ideal result:** you spot possible correlation-without-causation and a missing denominator, you explain that a defect count needs a rate to interpret, and you reach for "what else differs between these groups?" to defuse causal overreach.

## Section 7: Common Misconceptions

Common misconceptions about empirical software engineering include:

* **"Data speaks for itself."** Data never speaks; measurement choices and analysis put words in its mouth. The same commit log supports opposite stories depending on what you decided it meant.

* **"More data means more truth."** Volume does not fix bias. A million rows from a skewed source give you a precisely wrong answer. Repository mining is powerful and still inherits every quirk of how the tools logged events.

* **"Empirical means quantitative."** Numbers are one kind of evidence. Case studies and interviews are fully empirical; they observe reality systematically without reducing it to a metric, and they answer "why" questions that numbers cannot.

* **"A statistically significant result is an important result."** Significance says an effect is probably not zero. It says nothing about whether the effect is large enough to care about. A significant one percent improvement can be real and irrelevant.

* **"Research findings apply directly to this team."** External validity is never automatic. A finding is a starting hypothesis for your context, not a verdict on it.

## Section 8: When Formal Empirical Methods Are Overkill

Empirical rigor has a cost, and not every question earns it. Understanding when to skip the machinery keeps you from performing science on decisions that do not need it.

**The decision is cheap and reversible** - If trying something takes an afternoon and undoing it takes another, just try it. Reserve careful study for choices that are expensive or hard to walk back.

**The effect is obvious** - If a change makes the build ten times faster and everyone can feel it, you do not need a controlled experiment to confirm the obvious. Save the effort for effects too small to see by eye.

**You lack the sample to conclude anything** - A three-person team cannot run a statistically meaningful experiment on itself. Be honest that you are gathering anecdotes, useful ones, and do not dress them as findings.

**The question is about values, not facts** - Whether to prioritize speed or stability is a judgment call about what you want, and no dataset settles it. Empirical methods inform value decisions; they do not make them.

Even when you skip formal study, keep the empirical habit: state what you expect to happen, watch what actually happens, and update. That lightweight loop captures most of the value at a fraction of the cost.

## Laws, Bias, and Fallacies

Empirical software engineering exists partly because human reasoning about software is predictably distorted. These named effects are the distortions you will meet most often, both in studies and in the arguments around them.

### Laws and named effects

* **Goodhart's Law** - "When a measure becomes a target, it ceases to be a good measure." The moment a team is judged by commits or coverage, people optimize the number and hollow out the concept it stood for. This is why construct validity erodes the instant a metric gains stakes.
* **Campbell's Law** - The social-science sibling of Goodhart: the more a quantitative indicator is used for decision-making, the more it distorts and corrupts the process it was meant to monitor.
* **Hawthorne effect** - People change their behavior because they are being observed, not because of the treatment. A team may improve simply because someone is studying them, which quietly threatens internal validity in any watched study.
* **Twyman's Law** - "Any figure that looks interesting or different is usually wrong." A dramatic result is more often a data artifact or measurement bug than a discovery. Treat surprises as prompts to check the pipeline first.

### Biases

* **Confirmation bias** - You notice the studies and data that fit what you already believe and wave away the rest. It is the reason a documented, reproducible search protocol matters more than a smart person's reading list.
* **Survivorship bias** - Studying only the projects that shipped, or the companies that survived, hides the failures that would change the conclusion. "Successful teams do X" is worthless without knowing how many failed teams also did X.
* **Publication bias** - Studies with positive, exciting results get published; null results sit in drawers. The visible literature therefore overstates effects, which is one more reason to trust systematic reviews over eye-catching individual papers.
* **Automation complacency** - Once a dashboard produces numbers, people trust them without questioning how they were computed. The metric's authority comes from its rendering, not its validity.

### Fallacies

* **McNamara fallacy** - Deciding only by what is easily measured, then assuming what cannot be measured does not matter. Design quality, morale, and learning resist metrics and still drive outcomes, so discarding them because they are hard to count quietly throws away the things that matter most.
* **Texas sharpshooter fallacy** - Firing at the barn, then drawing the target around the tightest cluster of holes. In data terms, mining a dataset for any significant pattern and reporting it as if you had predicted it. This is why hypotheses should precede the data.
* **Base rate neglect** - Judging a result without its background frequency. "The new check flagged 50 risky commits" is uninterpretable until you know the total number of commits and how many were actually risky.
* **False dichotomy** - Framing "trust data" and "trust experience" as opposites. Empirical software engineering combines them: evidence tests experience, and experience interprets evidence.

Every item here has the same practical moral: state your question and expectation before you look, measure the concept rather than its convenient shadow, and hold any single number at arm's length.

## What to Remember

* **Empirical software engineering replaces authority with observation** so claims about software can be tested instead of merely asserted.
* **The research design decides what you may conclude** - only randomized designs support causal claims; mining and surveys show association and attitude.
* **Measurement is the hard part** - almost everything worth studying is a proxy, and construct validity decides whether the proxy means anything.
* **Threats to validity are the real content of a study** - read for construct, internal, external, and conclusion validity, and treat a single result as a hypothesis, not a verdict.
* **Evidence plus context makes the decision** - neither cargo-culting a study nor dismissing all research is reasoning.

### How These Concepts Connect

The pieces form one chain. A question sets the goal; the design determines what evidence is reachable; measurement decides whether the data means what you think; validity analysis exposes where the chain is weakest; and EBSE weighs the surviving evidence against your context. The laws and biases explain why each link tends to fail. Weaken any link and the conclusion breaks, no matter how much data sits behind it.

### Getting Started with Empirical Thinking

If you are new to this, start with a narrow, repeatable habit:

1. **Pick one belief** your team holds about how it works.
2. **Write down the question** it answers and what you would expect to see.
3. **Find the cheapest honest measure** of that concept, and name how it could mislead.
4. **Look at the data** and ask what else could explain it.
5. **Hold the conclusion loosely** until something independent points the same way.

Once that loop feels natural, apply it to bigger questions and read others' studies the same way.

### Self-Assessment – Can You Explain These in Your Own Words?

Before moving on, see if you can explain these in your own words:

* Why randomization is what lets an experiment claim causation.
* What construct validity is, and why a metric can fail it.
* The difference between internal and external validity.
* Why Goodhart's Law makes stakes corrode metrics.

If you can explain these clearly, you have internalized the fundamentals.

## Future Trends

Empirical methods in software are changing as the data and the tooling change. Two shifts are worth watching.

### Larger, messier data and causal inference

The scale of available data keeps growing: entire ecosystems of public repositories, continuous production telemetry, and developer tool usage. The frontier is moving from describing associations in this data toward causal inference techniques that try to extract causal claims from observational data without a randomized experiment. These methods are powerful and fragile, and they raise the bar for how carefully confounders must be handled.

**What this means:** more findings will come from observed behavior at scale rather than lab experiments, so the skill of reading for confounders becomes more valuable, not less.

**How to prepare:** learn what a confounder is and why randomization matters, so you can tell a careful causal-inference study from a dressed-up correlation.

### Studying AI-assisted development

As AI coding assistants become part of everyday work, the questions turn to them: do they make developers faster, do they change defect rates, do they shift what skills matter. These are empirical questions, and early answers will be noisy, contradictory, and heavily shaped by who was studied doing what.

**What this means:** you will see a flood of confident claims about AI and productivity built on weak designs and short-term measures.

**How to prepare:** apply the same lens you would to any claim - what was measured, on whom, compared with what, and who might disagree.

## Glossary

## References

* [Experimentation in Software Engineering][exp-book] by Wohlin et al., the standard text on designing software experiments and reasoning about validity threats.
* [Guidelines for conducting and reporting case study research in software engineering][case-study] by Runeson and Höst, on how case studies produce credible evidence.
* [Evidence-Based Software Engineering][ebse-paper] by Kitchenham, Dybå, and Jørgensen, the paper that framed EBSE for the field.
* [The State of DevOps Report (DORA)][dora], an example of large-scale empirical measurement of delivery performance, updated annually.
* [The SPACE of Developer Productivity][space] by Forsgren et al., on why productivity needs multiple validated dimensions rather than one proxy.
* [Related: Fundamentals of Statistics][stats], [Fundamentals of Metrics][metrics], and [Fundamentals of Data Analysis][data-analysis] on this site.

### Note on Verification

Empirical results age, and single studies get overturned. Prefer recent systematic reviews over individual papers, check whether a finding has been replicated in a setting like yours, and treat any dramatic number as a prompt to inspect how it was measured.

[stats]: https://jeffbailey.us/blog/2025/10/19/fundamentals-of-statistics/
[metrics]: https://jeffbailey.us/blog/2025/11/09/fundamentals-of-metrics/
[data-analysis]: https://jeffbailey.us/blog/2025/10/19/fundamentals-of-data-analysis/
[observability]: https://jeffbailey.us/blog/2025/11/16/fundamentals-of-monitoring-and-observability/
[testing]: https://jeffbailey.us/blog/2025/11/30/fundamentals-of-software-testing/
[exp-book]: https://link.springer.com/book/10.1007/978-3-642-29044-2
[case-study]: https://link.springer.com/article/10.1007/s10664-008-9102-8
[ebse-paper]: https://ieeexplore.ieee.org/document/1300292
[dora]: https://dora.dev/research/
[space]: https://queue.acm.org/detail.cfm?id=3454124