@sanityinc @deech @spacehobo

I’d love to do a proper study on this. I find it hard to define the difference between necessary and unnecessary complexity. My favourite examples of this relate to text. Consider C++’s std::string vs Objective-C’s NSString. The former is a contiguous block of memory storing characters (it’s actually templated over the character type). I’ve worked with some very smart developers who are completely convinced that this is all that you need for text. In contrast, NSString provides an abstraction over the underlying storage (it doesn’t need to be contiguous) and character set (the interfaces are exposed as Unicode [not quite sensibly because it predates Unicode needing more than 16 bits per code point, so it uses UTF-16 code units where it should use code points]). I’ve worked on codebases where changing the underlying storage for a specific NSString usage gave a significant, measurable performance improvement. But implementing NSString correctly requires a couple of orders of magnitude more code than implementing the C++ strings APIs.

Similarly, the OpenStep NSTextView is a (slightly dated) thing of beauty that provides all of the text rendering that you need to build a DTP package. Most GUI toolkits provide something more primitive and it’s mostly okay for 80% of programs, but the remaining 20% all implement subtly different and incompatible versions of the missing functionality.

The problems I’ve seen with open-source codebases from big tech are somewhat different and they usually fall into two categories:

First, they assume that refactoring API consumers is easy. Google famously puts all of their code in a monorepo and runs at-scale automated refactoring tools over the whole thing. If you got an API slightly wrong, it doesn’t matter, you deploy the new one and refactor all consumers to use it. That doesn’t work in a distributed collaborative environment. Consumers of your library or framework may have much longer update cycles than you. If you ship a bad API, they will use it (and maybe thousands of downstreams will use it), so it will cause a lot of problems that could have been avoided by putting a few days more thought into the design. Refactoring them all is hard because they’re scattered across a load of different open and internal codebases owned by different people.

Second, they either don’t generalise or generalise from too few examples. If you created a library that has one internal user, the line between library code and application code is blurry. If you put things in the library, every subsequent consumer may be constrained by that choice. A good API lets you express the delta between what most people want and what you want. But if there’s a single consumer, that delta is either everything or nothing. And there’s no difference in code complexity based on where you put the things: it exists in exactly one place either way. So the best choice is to start by making everything explicit. Sure, 90% of consumers will end up with exactly the same thousands of lines of setup, but that’s better than providing the wrong defaults and having a thousand lines of changing the defaults in every consumer. But the maintainers then don’t see this duplication affecting their employer because their employer has only 2-3 consumers of the API, and maybe they have wildly different requirements for defaults (or maybe they just factor them out into some shared wrapper).

The best advice I’d give to junior programmers wanting to do something F/OSS is to contribute to a project that has thousands of downstream consumers and strong backwards ABI compatibility guarantees (decade plus ideally, but at least maintained stable releases with multi-year support) before you work on anything else. The intuitions that you pick up from this will be incredibly valuable and will improve every other project that you work on.