It’s a common pitfall in software engineering to over engineer and/or prematurely optimize a system. In my career, I’ve seen these inclinations from junior engineers all the way up to VPs of engineering. Which is to say that it hasn’t been uncommon to find myself debating the merits and demerits of a solution in terms of what I would call its excesses. During these discussions I often find myself explaining the principles of a term that I’ve only recently learned others have already coined, namely Pain-Driven Development (PDD).
Put simply, PDD is allowing whatever is most frustrating or difficult about your project to guide what you work on next. This could take a variety of forms, from fixing production bottlenecks to improved tooling to process changes within or across teams. It begins as a gut-level idea of what could be improved (“doing x each week is so arduous”), although it can also be made more informed with data (“x takes me an average of 4 hours per week”).
As a practice, it eschews over engineering and premature optimization, and is therefore a close relative of YAGNI, MVP, KISS, and others. Rather than relying on your best guess about the future of your project to determine what to invest more time into, you let your up-to-the-moment knowledge of the project’s challenges inform your next actions. By letting the present rather than the future guide your decision making, you actually know that what you’re working on is worthwhile. In this way, PDD is also akin to Just In Time (JIT) manufacturing, in which producers make items as demand arises rather than making a surplus to meet a projected demand.
This isn’t to say that PDD deprioritizes thoughtful solutions. Designing systems to meet immediate requirements is still essential, but emphasis is placed on the flexibility of a solution and the nimbleness of the software development process overall as a way of being able to make changes as new needs are felt. Rather than building the supposed Perfect System up front, more time can be spent on, for example, tooling and testing, so that when pain points arise (whether they’re those of users or internal to the project), it’s easier to address them.
Where might PDD falter and/or what are some of the cons of relying on PDD? For one, without the ability to think through pain points and accurately estimate their level of effort, you can find yourself rabbit holing on issues that you might not be in a position to fix. For example, although you might wish to swap out your object database for a relational alternative, attempting to do so within a system of sufficient size and complexity is often not realistic. It’s important to realize that, and to be scrappy about what you can manage to accomplish.
PDD also comes in for criticism for being short-sighted, and I don’t think this is totally unfair. PDD is a terrific guide for what to work on next, but it’s not a replacement for stepping back and looking at a project or product holistically. The micro and macro views compliment one another, and unequivocally both are necessary.
For me, Pain-Driven Development has become an indispensable mental framework. It helps me to think critically when designing solutions and to prioritize effectively once a project is off the ground. It’s also an intuitive process, which makes it easy to leverage even when I’m juggling multiple responsibilities at once. Finally, PDD works well with others; we all have pain points in our work, and discussing them leads to fruitful conversations and impactful solutions.