A reflection on my 20 years of software development
Some things changed. Others didn't. And while I made many mistakes during these years I also learned a lot about both the market and myself.
I received my first paycheck as a software developer in July 2004. It's been 20 years now.
During these years I've been a developer, a business owner, a consultant, a trainer, and learned (mainly by making mistakes) many things about both the technical and the social side of software development.
What I wrote down here is based on what I lived in this market, which means: it's based on many years of experience, but it's also biased by these same experiences. But as I've seen things changing and some things not changing, I'd like to try to elaborate my reasoning on what are the causes and consequences of these outcomes.
So, let's start!
What changed in 20 years?
Well… Obviously, technology changed. And with that, everything changed.
When I started programming we didn't have iPhones or Android phones, and curiously enough I was working with mobile computing. Pocket PC 2002 was the operational system and I was writing C++ code that needed to be highly optimized because of all the memory restrictions we had back then.
But most importantly: the organizations changed. And most of the changes I saw were caused by technology innovations.
Complicated, then simpler, then complex
If you ever wrote a web application using J2EE and Enterprise Beans you can remember how bureaucratic it was. The deployment wasn't straightforward, we needed to write a lot of code to do the simplest thing and changes of a database schema were a nightmare. The thing was we were mostly creating private web applications because the web wasn't what it is today (not even Orkut had been created back then).
At first, these pain points incentivized organizations to push for bigger teams, a specialized database management department and also an operations department.
On the other hand, developers started to create tools to solve these pain points: ORM frameworks such as Hibernate started to become popular, then frameworks such as Rails started to introduce easier ways to create evolutionary databases and also a more direct approach to create a web application.
These changes empowered the developers and allowed small teams to be even more productive than big corporate teams were. With that we saw a shift in the market: everybody wanted smaller teams with the so called Full-stack Developers. Cloud computing was available for everyone for less than US$5/month with Heroku, DigitalOcean and AWS, and a lot of startups started to pop up everywhere.
But then… everything changed.
The web evolved and what was a simple REQUEST/RESPONSE with HTML and CSS became a complex puzzle using Ajax, jQuery, and then Angular, Ember, React, Vue, Svelte. The companies started to require frontend developers.
On the backend, things got even more complex. The giants in the industry were facing the challenges caused by their own success: how to handle such a massive amount of simultaneous users? How to store efficiently store this massive amount of data?
Then again, the developers came to the rescue and created solutions such as Docker, Kubernetes, Grafana. The giants shared their success stories and the medium-sized companies decided to add all this complexity to their products in order to achieve the same results. In part, this movement was also caused by the developers: we wanted to work with the cutting-edge technology used by the giants of the market, even if it was clearly not necessary for our servers that were handling way less users than Google or Facebook.
Developing a simple software felt… unnecessarily complex. Most companies I worked with back then weren't creating products for the masses, but niche products with little to no necessity of such complexity. And that's actually when I started to work more with consultancy and training again. I was constantly advocating for simpler models whenever they were good enough.
I still am.
What didn't change in 20 years?
Many things:
We still need software developers
People are still trying to figure out how to measure productivity in the software development field
Estimates are still a problem for most people
People are still arguing over which is the best text editor/IDE/framework/language
And well… software development still brings me joy :)
I don't feel I said everything I wanted to say. I still need to reflect more on how I changed during these 20 years.
Maybe I'll write more about that in the future.