Saturday, November 29, 2008

Covering your tracks - let your codebase follow your skills

We've all been in situations when somebody dug up a piece of code from 1997, in a far far away module, authored by us and still containing some not the best of the breed code construction. We've spent too much time catenating Strings instead of StringBuffers, we left behind empty catch blocks, we double-iterated over a map first for the key and then for the value, you name it. We would never do it now, but there were times when we weren't that expert. When it happens there is usually a lot of joy for others when it's spotted, not so cool for us though.

There are several things we can get out of it: first of all if we want to see our programming skills evolving we just need to review our code in chronological order - it's all documented there. Now as we already know how to fix ourselves a nice evening of memories, full of emotions, recalling how fragile and naive developers we were some day, let's think about some more practical aspects of that discovery. Thus the second: people are still running this code and depend on it, and the third – if your portfolio is to be complete it doesn't look to good.

It seems like a good idea to – whenever you learn something new – review your earlier work and update it according to new information you’ve just gathered.

How would you go about that?

You can read the whole code, spot where you used a 'deprecated' technique and replace it with a 'better-new' approach. A single review of such kind may be months to years in time cost depending on how productive you are, and there is a high risk that your manager will beg you to make sure that you will not learn a single thing afterwards.

Then you can use string analyzing tools: simple 'grep' or any other string find software would reduce this time greatly (if you are using home-made one please make sure that you are using StringBuffers for heavily changing strings). You will still get a lot of noise (false positives) and miss a lot of problems (false negatives), which will require a lot of manual analyzing - nevertheless it will be much faster than reading through the whole codebase.

And then there are dedicated tools for that - greps which have kind of regular expressions based on the structure of programming language you work in. That's the fastest I know. Choosing such tool you need to make sure that (one) you are able to integrate it with your work, (two) you can define the pattern you want to find easily, and (three) you are able to define patterns in a way which allows you to reduce noise to minimum (patterns are not to generic (false positives), or too specific (false negatives)).

Once you have such product integrated with you - as soon you learn something new, you can make sure that code you created in the past follows.

Friday, November 28, 2008

Static Analysis - is it ROIng or not ROIng

Today I would like to talk about pattern based code checking (aka Static Analysis aka Coding Standards) - a very powerful technique which allows you to improve quality of your code in no time.

The idea is pretty simple - programming is an activity of coding based on the grammar, which is somewhat regular, and that means that programs will have similar patterns in their code. This is where bad and best practices come from. Now if we have means to analyze the program structure (given by existence of grammar) and we have patterns of different quality - we can promote good over bad patterns and that improves the overall quality of our software.

How would you do that - the easiest technique is to find and eliminate bad patterns in existing code. There is a wealth of products which not only can analyze the structure of your code but have tens or hundreds of such best practices built in. They integrate with our development IDEs, allow for extensions (defining your own rules), manage results of analysis, sometimes have auto-fix feature (apply standard refactoring to replace bad practice with its' better equivalent).

Now, there is a lot of discussion on whether it is necessary to use static analysis in your development lifecycle. The arguments are that this practice is not about chasing errors which would necessarily trigger off at some point of a user experience and kill our application in front of her eyes, this is more about preventing errors by writing in 'better style'. If we look at it from the return of investment side, we have to consider three costs - cost of the product, cost of introduction of the product, cost of using it.

As for the cost of the product - it's usually in whereabouts of several thousands dollars. Just think about it as a vehicle to make junior developer into senior one by putting it at its belt. Do the math how quickly it pays off.

As for the cost of introduction - back in the eighties there may have been some learning curve involved in this (as everything was command line based then), but now it just plugs in into your development environment, you work with results by clicking, it jumps with you to specific places in the code which require modification, explains what to do and why to do it. From time to time I train people on various kinds of testing. Static analysis takes about two-three hours (including rules configuration, suppressions mechanisms etc.). Again I'm leaving the math for you.

As for the cost of using it - processing time is pretty much about the same (up to three times longer) as normal compilation and it suits kind of similar purpose (checks whether code is sound and you can progress based on it with your development), so if you allow your team to compile code as they go, you should be ok with them running static analysis. Than there is working with results - first of all usually there is one click to assess whether violation should be fixed or not - it takes literally seconds. No false positives (of course if software doing analysis is not buggy itself).

And now for the ROI - depending on sources and lifecycle phase we have the cost of single defect removal varying from 1,000$ to 20,000$ and more.

That leaves it as a no-brainer. After three, four defects are removed, we have our price of the product paid off and we are left with developer directly connected to the oracle of coding wisdom.

Tuesday, November 25, 2008

Prima donna - 50 ways to kill your project

While we are talking about developers - there is one group, which 'deserves' a separate entry. These are software prima donnas - boys and girls who have one major goal - to look smarter than the others. Much smarter that is.

As you can imagine with such agenda there is not much good they can contribute to the project. It was kind of obvious to me since my day one. But I had no idea about how much harm to the project a single prima donna could do...

Let's look at them for a moment - it's easy to spot them – get the whole team in one room and talk to them about anything - if you see a person with silent disapproval on the face - this is your prima donna working hard for her throne. What always amazed me was the fact that the subject of the talk was irrelevant, the person talking was irrelevant - prima donna had always this unhappy 'I would do it much better' face.

Now, don't get me wrong - it's not like every project has them - in my whole professional life I worked closer with only three of them.

They are rare, but they are deadly.

Their number one goal is asserting their superiority. They have all kinds of power tricks in their arsenal: misleading, information hiding, gossiping, marginalizing others’ achievements, coloring up own achievement, over complicating things to arrive as a great savior of the day afterwards, asking questions without good answers, refocusing toward my areas of expertise, implying others' lack of competence, learning some very detailed part of knowledge and dragging it out whenever possible, you name it.

If they manage to achieve that they try to find followers - they are a building opposition business now. First of all they spread 'being unhappy' atmosphere. After all they need to be in opposition to something - the existing order of things. That means that current state of affairs has to be a bad one. If it isn't it has to be made one. And as we all know, unhappiness spreads easily so soon enough you will have an unhappy crowd instead of the team. And then it's usually too late to save a project.

Now, is it possible to fight the 'sneaky prima donna sabotage'? Of course it is - all you have to do is to show that the prima donna superiority is a fabricated one. Provide enough visibility on prima donna work effects (software creation related) and two things will happen: first of all everybody in the team will see that there is nothing special in prima donna performance, this is just as normal person as we all are; and secondly that would also force prima donna to focus on what he should be focusing on - software development. That should put prima donna covert operation into sleep (until better times come).

Or if you want a long-term solution - as soon you spot prima donna operating just split with her. You have my word - she will find another job and will be as unhappy there as she is with you. That's what prima donnas do.

Monday, November 24, 2008

In search for developers - curious craftsmen wanted!

We looked at what a good manager is supposed to be about. Let's now try to figure out what makes developers write better quality code. Things like skills and brilliance may certainly help. But are those the core components? I don't think so.

If I was to choose perfect features of my team member I would go with curiosity, engagement, openness and thoroughness in no particular order (or maybe in alphabetical). All right - I can add that some programming experience would be a plus as well.

Let's start with openness - no software developer is an island, thus communication is everything in software business. I talked about it already but I can't emphasize it enough. Unless you don't need extensible maintainable software, its' creators have to communicate with their peers and the rest of external world. So they do have to have two following skills: being able to talk to others (and actually practicing it) and being likeable enough to be talked to. I'm not saying that they should be party stars or anything like that. Nerds with working I/O would do.

Now, what I found is that creating something of high value and high quality was always going in pair with having fun working on it (no matter whether I was a developer, architect or manager). I think that basically goes with the feeling that you are doing something right. Moreover, in these projects I had people who no matter what they were doing, they were always putting their 100% in it. It was somehow easy to get them engaged. At some point I figured out that they had engagement ingrained in them. And believe me you want to work with such people. People who don't give a damn, who are 'just earning money' are not fun to work with. They are inclined to take shortcuts, think short term and to look at you with 'what this has to do with me' face in the time of crisis. If you can - work with people who care about what they are doing.

As I mentioned earlier building up quality is all about incremental improvements, innovations which make your code better and better. And to innovate you have to stay up to date, learn and try new things. Think about your current quality as the status quo, comfort zone your developers live in. If you want them to get out of this cave they need to be curious enough to take action which would push them out. Given that they are wondering enough to see shadows and hear echoes in the first place. No progress without questioning, in short.

And finally thoroughness - this is the difficult one - most of the folks go into this business because of high payoffs for creative actions - you build up several routines and you are already running something which paints beautiful outputs to your screen, you change a bit of it and the outcomes are completely different and even better adjusted to what you wanted to get. This is what we all are for in this business. Unfortunately, if we want to make something more out of it than a creator's toy - after we have the prototype which works with input of our choice, we need to put ten times more work to make it work with all the inputs in all situations - and this is not so much fun any more. Ratio is different based on how you modeled domain in the first place, but still creative piece is always far shorter than the process of handling non-standard cases afterwards. To make things worse we know how to get next adrenaline fix - we just need to move on to creating next piece. It requires a lot of discipline to finish it up - I've been there - it took me many years to develop thoroughness and ingrain it into my character and I still have slips from time to time. Fortunately being able to do the task completely offers also good chemistry strike through your body and additionally good managers will be looking for you everywhere.

To sum this up - features commonly searched for like skills and creativity are for sure important, but at least my experience says that some character traits (which by the way would make you stand out in anything you put your hands to) are far more important than that.

Friday, November 21, 2008

On good manager - how to spend windowed office wisely

Last post was about insecure software manager vicious circle, so to balance it today I will talk about what I believe are the most desired features in managers’ CVs to scan for.

Task of finding a good manager is one of the important ones as difference on this element of the puzzle can give us results varying from a successful project on time (if you have your project delivered even earlier it's most likely time to wake up and go to work) to projects abandoned out of frustration by the whole crew, completely unmanageable, which you need to burn to the ground to avoid disease spreading. So unless you have magic idea how to capitalize on high turnover (if you have and you are still making software you are apparently missing something) you should look for a manager who would perform well.

So what makes a difference in manager performance? I would probably go for the following mix: integrity, ability to keep an eye on the target, ability to glue the team, ability to facilitate communication and minimize its costs. Let me elaborate a bit on that.

She has to have the integrity as there will be pressures from all directions - we have customers pushing for features, project sponsors pushing for return on investment, team pushing for nice neat innovations, economy pushing for savings - manager has to know the balance and has to level everything to keep it.

Similar story about maintaining the vision on what it is we are producing. Everybody else get their own partial perspective on the project - there is money view and features view and program characteristics view, internal construction view. Project sponsor is funding it to get more money out of it, developer is writing code to get PS3 for his 1-year old son. Project manager is the person who has to see from the project success perspective. Now this is a difficult task, because requirements, resources keep changing as we go (as we didn't put external world on standby). Project manager has to keep this running target under control and in sync.

Now let's look how the team works - from the workflow point of view, we work in iterations: participants are agreeing what they are working on (parts) and how it will connect to the rest of a whole (interfaces). Then they work on their parts and when finished they integrate the parts with the whole. Keep in mind that we are working on the mathematical model of a high complexity - there is a lot of communication required to make it happen and any miscommunication costs a lot. Project manager has to carefully develop environment, which promotes easy flowing communication. All disrupting factors like interpersonal conflicts, prima donnas, special procedures for communication have to be resolved at all costs. Manager should also seek to find ways to provide people with information in a way, which doesn't get them out of their zones or put them on documentation pages for hours. She should also at all costs make sure that nobody feels overlooked there.

Now to the gluing part - I've worked with many teams in my life, and I can see two patterns really - either people are engaged into the project and work as a team, are proud of being a member of the team, help each other, or they are there to earn money, doing what they are asked to but never going an extra inch in project cause. Performances of both teams differ drastically - I've never done proper measurement, but I think factor of ten is not unreasonable. If you want to check which of the two your team is, check whether most of the team is sitting on their chairs 5.03pm - if they are then there is a pretty good chance they are engaged. By the way if they are still there at 6pm you have another kind of problem, but this is for another occasion. If you are a manager and cannot check whether they are there at 5.03pm because you are in the car already you not only can safely bet they aren't but you have also serious suspicion for what is causing it. To be honest I've never figured out what does the gluing - I had teams working on similar projects on two completely different ends of this scale. The only hints I can give here is: treat people seriously, be honest and be friendly.

And remember you are there no to demand. You are there to help.

Wednesday, November 19, 2008

Continuously degrading quality - insecure manager kills it all

Software programs tend (still) to be created by people - so it's quite likely that different people would create software of different quality. There is a whole bunch of people who are involved during project creation - we have product managers, project managers, project sponsors, architects, team leads, software developers, graphic designers, docs people, qa people etc. Let's divide them into two groups: managers (people responsible for overall outcome) and developers (people who develop parts of actual product). Trying to refocus from how big this oversimplification is let's try to figure what are the desired characteristics for the manager. Or maybe let's try an easier path and leave the desired stuff for later - what actually should be avoided at all costs when you are looking for a good manager to take on your project.

Based on my experience if there is a single thing I could say to a software manager sitting next to my death bed (I don't think I'll get that lucky...) - that would be: 'do no harm'.

Why is that? Because managers are the people who do most harm during software project creation. They are usually responsible for project failures. Am I saying managers are idiots? Not at all. Managers are usually the most experienced and wise people in the project. But...

Managers have extremely hard life: they are not much smarter (if at all) than the rest of the team in which they are supposed to be power figures. Remember that we are talking about ecosystem in which the main survival skill is mathematics. It's not easy to compete with pale-face, skinny-bodied mathema-males. But the manager has to have respect in this world to lead the project to desired outcome.

Additionally managers do not have measure work units which they could present to their masters if required. There is no I did 20 pounds of managing today which is well above my last year average. They are supposed to define a course and if the boat deviates from it put corrections in place.

So project management seems like easy money. Start the thing up, anticipate problems and help team prevent them or overcome them. Make sure communication within the team doesn't go off limits and protect team from the external world.

The rest of the time sit in the office and do not distract others.

This is where it gets weak when experienced by insecure manager:
if I'm not doing anything apparently there is something wrong and my masters would not be happy with me. I need to help my team. I will get involved in some technical meetings, I will argue with technicians on technical stuff, to defend my position in the team as a power figure I will be winning arguments using 'because I say so' approach. Seeing morale plunging I will design some sophisticated work review procedures and add some procedures to uniform the team. I will talk to developers with sad voice about their performance to make them improve. At the end I might not be happy and they will probably seem miserable but whatever it takes to deliver the project. Maybe late and maybe not really tested, but we for sure all worked hard every minute of our time...

So if you are a manager please remember 'do no harm' - especially if you are insecure. If you don't calculate both benefits and costs of introducing new practice, don't introduce it. If you are not sure of the outcome of the new experiment don't start it.

And if you are insecure fight your insecurity with all power you have for Christ sake - you are a manager now, you are supposed to lead people.

Monday, November 17, 2008

Attempts to define quality

So software quality can help us grow our business. Can make our customers loyal. Gives us an edge when it comes to competition.

Let's define what we mean by software quality. Software quality from the user perspective is its correctness (it does what is it supposed to), reliability (it does actually work even if we deviate from tutorial), scalability (larger inputs don't get it out of balance), completeness (models whole domain it represents, ratio of quirky NYI message boxes/produced assertions is rather small), intuitiveness (user spends more time with the program than its documentation). Software quality from developer point of interest means also readability (software can be still read after several sequences of improvements), extendibility (extension of the domain doesn't require creating of separate program), maintability (if defect is found there is no need to shut down the business).

And many other factors really. Anything what could support 'My program is better because' cause in ideological war between two geeks or sales-reps (on CRM system choice) would do.

So software quality can be described as a vector of certain aspects of quality and our overall goal (quality engineering) would be to first normalize the vector (figure out which aspects are how important) by most likely putting weights to them and then try to maximize resulting value given resources we have or are obtainable (people, time etc). Why do we need this measure which combines all aspects into one - very often quality requirements are contradictory - to speed up application you need to sacrifice readability, to increase reliability you need to sacrifice speed etc. Don't be scared in the real world this measure is most often described in sentences like: 'reliability is the most important for us' or 'it has to be the fastest processing product' which gives you (a developer) a good framework to assess where to go when the fork on the road appears. This is also the place where projects can get dead-on-arrival status right away - if you (a manager) combine two contradicting aspects together and force developers to 'accept' them - you (still the same manager) build strong foundation for failure. Not to mention sacrificing morale of developers, who (usually) have better understanding what can and what cannot be achieved together when it comes to code construction.

But back on the subject - quality seems to be a task of finding a maximum in some vertex space. When we take a closer look at any one of the aspects we find that it's characteristic is kind of monotonous - if you can apply A to improve it, and then B to improve it - overall improvement will be aggregated. That means that continuous improvement would be our method of choice. Which is a good thing as we live by continuous improvement all our lives (or at least I hope we are). Now let's think what we can be improved - it's obvious like in the old joke: code can be improved. How do we go about improving code - we need to modify something in process of its' creation or we modify it post-mortem (which is also altering process of code creation). And that's where the fun begins...