The software Developers Dilemma: Chasing the Silver Bullet
Categories: Software, Professional, Consulting, AgileOne of the best coined terms in Software Development is the phrase: “There is no Silver Bullet” - meaning, there’s no magic way to slay the wherewolf, or doing software development in a perfect way. Yet, as I step out of the software development community and into the world of management consulting, I realize that the best developers out there, the ones committed to their field, eager to learn more and those who have an almost perfectionistic approach to their craft, holds on to this vision of finding the Silver Bullet - taking the role as the alchemist in search for gold.
I list a few dilemmas the silver-bullet-chasing software developer faces (and yes, I used to be one of them):
Creating Business Value vs “perfect code”: “Why do you code?” is a good question to ask a software developer. The honest answer would probably be because it’s fun, challenging and creative. Another, more formally correct answer would be because you want to create business value from IT. Consider that no business would care to pay a software developer who doesn’t provide business value tells us that the developer should be utterly focused on the value delivered from the programming. But value decrease with cost. Cost increase proposional with the time spent creating value. And here’s another thing: Innovative theory states that the effort to value ratio drops with time, improving and improving will at some point not generate enough value! Instead of chasing “perfect code” the developer should be more in sync with what business value is really being added?
Software product deprecation versus stable code: Software deprecation is a serious problem. It strikes when software evolves. As changes to the software is being made, complexity of the code and integrations increases. The time spent to add business value decreases to the time spent just making the product compile, run and work properly. The answer, of course, is stable code. Preferably modular where you can add business value as “components” like plug and play. However, there’s another very important aspect to consider: most software products are not created to last a life time. 5 years is often significant. Who can tell what kind of technology or what the IT landscape looks like five years from now? Even more, what business challenges will a company face in 5 years time? Any company thinking about investing in a new software product or web solution should consider the lifetime of their project, and think of this lifetime as “controlled software deprecation”. If so, it becomes completely waste (or YAGNI - You Aren’t Gonna Need It) to chase off after perfect code. Software developers should think in a similar fashion: if the product isn’t meant to last a lifetime, there’s no need to put all that extra effort into creating a great architecture with a fully configurable modular component software! Create what you need, no more no less. Anything else is simply bad for business.
The Agile Religion versus eliminate waste/YAGNI: I admit it: I was probably one of the earliest agile evangelists out there. In 2001 when I first heard about “lightweight” processes I was completely hooked: this is going to change software development forever! In three ways it did: 1) Software teams added some much needed understanding of mechanisms to how problem-solving creative teams work, 2) Business side understood that the outcome from creating software is at best uncertain and cannot be fully planned and budgeted from, and 3) the placebo effect of “believing” that agile rocks the world gave motivation, trust and controll back to software developers (who previously were functioning like machines or puppets controlled by the business people). Yet, the agile believers took XP, Scrum and other methodologies very seriously and said: “This is what we’re going to do, and we’re going to do it fully!” But following a methodology or process absolutely is exactly what agile is all about not doing. Reducing formal requirements, increasing rapid feedback and contribute to a collaborative environment is good but it all has to fit the corporate structure, the development culture, the business need from IT. Agile devleopment is a toolbox, and a good agile manager will use these tools in the best way possible, preferably combining several techniques but with a simple purpose: deliver business value the best way possible. Because, what is most important: following a process to its absolute or delivering the value asked for?
The silver bullet techniques versus adapting to change: With agile comes techniques like refactoring, pair programming, test driven development, self-documenting code and so forth. All good techniques. But like the previous point, using these techniques in all cases (like XP states) states two things: 1) There’s no room to maneuver. The techniques are enforced rather than adapted, hence working with an opposite psychological effect from what its intent was, and 2) behaves like it’s a silver bullet, which is the one true fact we’ve established does not exist! Insteand, these techniques are wonderful tools a software programmer should know about but used where it’s applicable. Not with a religious-approach that it solves all world problems.
Sometimes just simply programming works best: Yes, it may very well be! Adding all techniques, agile, creating a huge architecture, may sound good. But if the assignment is well understood, utterly simple and perhaps repetitive, why bother? If the software product is a support tool that can, in time, be replaced, why put all the effort into it? If the software can be created in a simple manner, make it compile, and accept that bugs will happen in production, then this may be the simplest approach that generate the most value!
No methodology og techniques can remove the effect of good leadership: Self-managed teams are great! They generate tremendous creative energy along with a dedication and commitment to the work. However, getting there is a long journey that might take years, it’s simply not enough to form a Scrum-team and state “we’re self-managed!”. But good leadership with creative teams is not about micro-management, it’s about getting the team to that ideal state where everyone’s a leader. Until reaching that point, the good leader must address team motivation, use what is available from the agile / software engineering toolbox, keep the team focused on the task at hand and be clear on expected results.
My points being:
1) eliminate waste, that also goes from new techniques or methodologies that’s not applicable in to the situation.
2) Create business value first, and consider that optimal code does not allways provide business value. Good leadership place expectations and focus on the business value. Proud developers will do what it takes to meet those expectations.
3) just following best practice for creative problem-solving teams like rapid feedback and a well established collaborative culture goes a long way - there’s no magic to that
June 17th, 2009 at 12:59 pm
Hej Bård
Just followed on Twitter (@kvan). I’m working at IBM in Denmark as an IT architect. Good article here with a nice clear focus, thanks.
I’m curious about your transition to consulting. Where are you working, what are you doing and…is it fun?
June 18th, 2009 at 3:42 pm
Hi Kevin, I saw your follow on twitter and returned the favor. Thanks
My transition to management consulting came through additional studies in technology management, and an interest in WHERE the IT decisions were being made (after all, it’s the technical guys who must suffer from bad technological decisions from management). So I got a chance to start working for PA Consulting in Norway, a great consultant company with 35 offices around the world and very big in Denmark by the way. However, the approach here is much more administrative and we base our recommendation on numerous of sources and best practices related to what we see fit with a certain organization. But to my knowledge, the technical part of IT is still seen a bit like a “black box” for many people in managerial positions: budgets are expected to be followed, technical problems can be solved by getting the engineers to work “harder” or put more people to the task, and possibilities of IT comes with promises from external sales people - not from good ideas from own ranks.
It’s therefore very interesting for me to bring my technical expertise into this setting but it also requires me to be more patient than I’m capable of. Still, working on a good software project like I used to, where you get to build, architect, compile and see your work in action is definitely extremely rewarding and fun. Giving good advice to company leaders is definitely very interesting and educational… if you see the difference.
Thanks for following kevin
June 21st, 2009 at 12:47 am
As promiced during our beer session, here are some comments to your post (probably accompanied by a lot of spelling mistakes.., it’s late..):
Your assumption here is that perfect code is at odds with business value. I STRINGLY DISAGREE! Clean code and high quality will produce MORE business value. The only way to go fast in software development is to go well. The longer/bigger the project, and the more business value involved, the truer this is.
This view is based on experience, and is not the future we would ask for. If we had the option to have our software and investments last longer than five years, and have the ability to evolve as well, we certainly should and would want that.
Oh yes, I completely agree. It’s close to the definition of perfect code. Follow this from day one, and if you have clean code, you can incrementally add value every day at a good pace. If the code is not clean, your productivity will drop too fast, and no more value can be added too soon. YAGNI (you ain’t gonna need it) is an agile value - huge arcitectures is the old way.
I agree, we have to watch out for the tendency to just follow practises and not think. Agile is often misunderstod as just another formal method, when in reality it is another philosophy completely. And at the base of it all is the realization that we always improve and always use what is best suited there and then.
You state some good oppinions in this blog post, but your headings are too tabloid for my taste, and you have some presumptions I can’t agree with. The dilemmas arn’t really there. I really recommend you read up on the Software Craftmanship movement, and listen to some of Robert C. Martin’s talks. If everyone tried to follow his principles the software industry would be a much better industry to be in - something we could count on to deliver high and lasting business value.
Best regards,
T-Man
I value Craftsmanship over Crap
June 22nd, 2009 at 3:06 pm
Thanks for your reply T-man! Your deep insight is always a welcome contribution. And yes, my heading was tabloid with a desire to provoke a little (which always works when talking about religion). Yet there’s some seriousness in it.
I agree with most of your consideration. Let’s refrase the importancy of the dilemma: WHERE is your focus as a developer? To deliver business value (hence, design a solution and implement something that adds business value in terms of increased revenue or cost reduction for your company / client) or is your focus on “perfect” code with the belief that perfect code will generate the best business value? Thing is, you can have both of course (clean code and business value) but if you need to focus on ONE thing over the other, what will it be?
And when I introduce the dimension of Software deprication, things change. Yes, this is based on experience. Ideally software would work forever but based on the ideas when it was created and with the limitations it saw during inception. If, for instance, your software product was estimated and budgeted to last for five years, how would this affect your thoughts on creating “perfect” code?
And even though I don’t show it in the text above, qualitative software craftmanship is a rare commodity these days. Blame the tools, blame the schools, blame the business people. I’m glad there are software developers out there who take REAL pride in their work - makes me sleep better at night knowing my Internet bank, facebook or the traffic lights will keep working tomorrow. Yet, finding the balance between creating business value (no more, no less) and the effort of creating “perfect” code is very important to consider.
After all, how much compiled code of what you write directly ends up generating business value when doing TDD? (yes, I know… how much time debugging/fixing code/cost of fixing poor design do you spend NOT doing TDD). At least it’s worth thinking about…
June 23rd, 2009 at 12:29 pm
In case any developers are reading this, I feel obliged to keep making my point. Talking about how reduced quality can be a good thing is dangerous!
The thing about short horizons (the depriciation consideration) is that what was intended as a two year solution more often then not ends up as a 10 year behemoth that you have to maintain and try to extend. Even quick solutions you are asked to throw together for a single user, that is only supposed to work for a month or two, ends up as huge systems with hundreds or more users in no time at all.
If we just had a way to take something made for small scale/time, and enhance it to make it better easily. Or if we just had the guts to say NO.
As a developer I am aware, very aware, of my responsibility to do work that adds value. “Write software that matters!” And I know there are a lot of devs that loose track of that. But as a dev it’s also my responsibility - and maybe even a more important responsibility - to inform the business of the consequences of going without quality. And it should be my ethical responsibility to say NO when quality is cut without a concrete plan to pay back the technical dept.
Let the hot, unexperieced developers with the red capes fly in and do the dirty projects. Them crashing and burning will hopefully make people listen to sensible craftsmanship.
About TDD the debate is over. It works, and it might even be considered unprofessional not to do it. If done right (yes, it can absolutely be done poorly) you will reliably be able to release faster with fever defects. Being able to change it quickly later on is just a bonus.