On the State of Modern Development (2017)
Estimated reading time: 4 minutes.
I finally figured out what bothers me about a lot of these modern development trends. Specifically referring to modern package usage and management in languages like JavaScript and PHP—e.g. Composer or NPM.
It came from finally realizing a distinction between ‘simple’ and ‘easy’.
- Easy means that the task requires little effort.
- Simple means that it possesses little complexity.
I think it's safe to say that the focus of software development and engineering to date has largely been about reducing complexity as a path to producing better software. It's been on creating simplicity rather than ease.
Look at software design patterns. More often than not, they will require more effort expended to implement—it's always going to be easier to just directly instantiate an object rather than implement a factory pattern—but we use them anyway because they make the software less complex.
Less complex software is much more accessible and much more readily reasoned about and understood. In order to develop good software, the software must be understandable. A large part of most coding standards that are trying to specify how to develop reliable or secure software will be dedicated to writing clear, predictable code. For instance, NASA JPL's standards for coding in C document (DOCID D-60411; March 3 2009) is almost entirely composed of sections such as Predictable Execution, Defensive Coding and Code Clarity.
The case is pretty clear that less complex software is more reliable, more secure software.
Yet almost the entirety of modern software development practices seem to disregard this, instead accepting an explosion of complexity in exchange for a bit of ease in development.
It's so easy to just pull in an external package or library to solve the issue at hand. You never need to see any of the complexity it brings with it. You don't look at what's going on under the hood.
The ease that comes from using these libraries all comes at a cost. These libraries can generally make things easy because they're able to make assumptions about your use case. If those assumptions should ever be invalidated, then all of the complexity that the library was shielding you from starts to bubble back up to the surface. If anything should ever break for any reason, you either hope that the library's author has provided documentation that directly addresses your issue (in my experience, unlikely) or you open the hood and dive into all the complexity hiding beneath. Especially when dealing with the immature libraries that have flooded some of the modern package managers, you will be fighting against the Law of Leaky Abstractions "All non-trivial abstractions, to some degree, are leaky."—Joel Splosky, The Law of Leaky Abstractions. The idea being that no matter how much we try and hide away an underlying complexity, there will be details of it that can't be ignored and will leak out of the abstraction back up into your software. —and you will lose.
In order to build good software, you or your team need to actually understand the product you are working on. We can use strategies to break the software up—black boxing the software into modules or plugins—but somebody still needs to understand the consequences, trade-offs, and side effects of the decisions that are made. For a completely contrived example as to how this can go wrong if your team doesn't understand the packages you're using: what if the oft-mocked leftpad were—instead of locally padding the string—making a call out to the left-pad.io service to pad the string for you. That would introduce some very serious security, speed, and reliability consequences all hidden behind an innocuous function call. That complexity cannot be ignored.
This modern method of adding seemingly-infinite One project I have laying around my hard-drive used for putting together fairly basic websites is pulling in dependencies totalling over 10,000 files with just over 250,000 lines of code among them. Wat. amounts of poorly understood complexity to your program before you lay down your first line of code is completely ass-backwards to everything I understand about how to build good—fast, reliable, secure—software. These dependencies are a tool to be carefully considered and used when necessary, not our first approach.
Okay. Old man rant over I think. Get off my lawn.
Further Reading
What was so mystifying about "Senior Dev" code was not that I didn’t understand it, but that I could understand it immediately, it was fundamentally dumb, and it seemed like there had to be more to it. "Where’s the rest?" I remember thinking. "How does this do all of that?" [...] The greatest lessons I’ve learned are that writing dumb code is actually hard, and that it pays exponential dividends to do so.
Every package that you use adds yet another dependency to your project. Dependencies, by their very name, are things you need in order for your code to function. The more dependencies you take on, the more points of failure you have. Not to mention the more chance for error: have you vetted any of the programmers who have written these functions that you depend on daily?
Take on a dependency for any complex functionality that would take a lot of time, money, and/or debugging to write yourself. Things like a database access layer (ORM) or caching client should be dependencies because they’re complicated and the risk of the dependency is well worth the savings and efficiency.
But, for the love of all that is programming, write your own bloody basic programming functions.