How much do we really understand about the computers we work with? When building complex programs, we end up building deep and intricate layers of abstraction that the rest of our software can stand upon. We’re building tall towers that rest upon a foundation that is very deep. Part of what makes open source software so powerful is that 99% of that abstraction has been built by someone before you. You don’t have to roll your own web server if you want to start a web business. All you have to do is add the last topping: the cherry on top of the 7 layered cake that is your creation.
Do you spend most of your day working on port 80? If you’re building web applications, you spend a large portion of your time working at an abstraction level that removes you far from the machine. For the purposes of web development, this is usually a good thing. Most of the time, you don’t want to worry about how many kernel syscalls a web request is making to fetch some customer data from the database. Shielding yourself from this level of granularity is a good thing. You want to focus on your core problems, and most of the machine is better left hidden.
If it’s technological innovation you’re after, it’s going to be difficult to find it if you can only work at one level of abstraction. If you’re not willing to dive into the complexities of the machine, you’ll be stuck solving the same old web problems again and again. The vast ocean of problems you could otherwise work on will start to look like a small pond.
This concept feels a lot like the 80/20 rule, wherein 80% of the problems are solvable working at the highest abstraction levels, but the remaining 20% are at the lower levels of abstraction. If you want to push the boundaries of your technology, you have to first understand where the boundaries are. This means being willing to get your hands dirty. This means a necessity to find enjoyment working with a tool or programming language that might not directly apply to the next hot web app, but will add another support beam to your knowledge structure. Having a little mechanical sympathy will go a long way in building technologies that open up new horizons.
What does this look like? It might be the web developer who teaches himself C. Or perhaps it’s the flash developer who teaches herself assembly. These pursuits seem fruitless in the short term, but in the long term pay off in spades.
You must have curiosity. It can be difficult to teach yourself how to work with low-level bytecode when you’re used to working with high level languages. There will be times of tedium that you must overcome. In my mind, the easiest way to overcome those barriers is to remain curious and tenacious. Taking a deep dive down your software stack means having the drive to understand how the whole jigsaw fits together.
Today, pick a component of your software stack that is unfamiliar to you. Study it and learn it. Knowing your tools goes a long way in making the right technical decisions of the future. The world will seem larger and more exciting if you do.
I’m commiting publicly to understanding the internals of Redis, and will write about my findings here. Will you join me in commiting to learn more about your software stack?