I found some interesting website which provides visitor with nice return on investment calculator: http://www.semanticdesigns.com/Purchase/CloneCalc.html
The basis for the estimate is Brenda Baker paper “On Finding Duplication and Near-Duplication in Large Software Systems.” You can find it for example on IEEExplore (http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=514697&isnumber=11405).
Bottom line: on average 13%-20% of the code can be removed (as is a result of code duplication) making maintenance of the project significantly cheaper. Assuming that code maintenance is proportional to its' size - up to 20% cheaper.
So if you are spending $70,000+ on your software engineer and have 5 of them maintaining the code invest $5,000 on code duplication removal product and get your money back in less than a month. Within a year you will be $30,000 up in the blacks. On average of course. Not bad.
Showing posts with label static analysis. Show all posts
Showing posts with label static analysis. Show all posts
Wednesday, December 17, 2008
Wednesday, December 3, 2008
User always chooses the wrong path - on data flowing through your code
Software development is so much fun
Developing the code is a very cool activity. Because of how creative and limitless it is and because of instant rewarding nature. You figure out what needs to be done - you model it, write it down, and execute almost immediately. Caboom! New shiny '4' is produced in your console as the answer to your '2' and '2' input. You just did an amazing piece on integers adding. You can move on to the next adventure.
And then software development suddenly is mundane
But then, can you really? I would suggest to try '3' and '2' inputs as well to make sure that your algorithm is adding (we expect '5') and not multiplying (which would get us to '6' and a bit of frustration when trying to claim money back from our savings account) or even better with '3' and '4' to make sure that it doesn't add '2' to the first argument. This for some time can be cool experience - as you are still playing in 'what I could’ve missed' game, but far before we get to '2147483647' + '1' the boredom sneaks in and just kills all the fun.
What makes the whole thing worse is that you are locked in what you know about the code you've just written, and you haven't managed to develop your brain significantly since then, so you are locked in the same mindset with all limiting consequences.
Bad news is that we have to do it and there is no way around it, good news is that there are tools and techniques available to shortcut it. Today I would like to talk about one of them called 'data-flow analysis'.
Data flow analysis - what is it about?
So what is it - it is a static code analysis in a sense that we don't run our software to get results, but what it is trying to do is to mimic potential paths through the code in search of some specific path or data patterns which are for some reasons interesting. When you think about it - this approach is much more powerful than just testing some of the paths - here we have all of them analyzed.
The idea is: let's assume we can collect all possible paths through our application and then let's define subset there which would collect 'something went wrong' paths. Extremely powerful idea - if entirely realizable, it would be equivalent to testing all possible inputs and conditions. And the world would be a different place, where software is cheap. And big part of software developers would be selling coffee in Starbucks.
Data flow analysis - how useful it is?
Unfortunately for software users and fortunately for software developers and currently selling coffee wanna-be-actors it is not entirely possible. Calculus required is too complicated, space of all possible paths and inputs too big to control. Does it mean it's useless? Not at all.
It's actually extremely useful and commonly used - the trick is to limit the 'all paths' set by imposing maximum path length, size of all possible within path transitions etc. And we still can get extremely valuable results - these algorithms know transitions or path segments which you usually don't anticipate - like for simple setups which lead to raising exception from standard functions, rare paths which lead to leaked memory and resources, weird user scenarios which lead to pumping your collections with excess of data. Running your code through such algorithms is actually an eye-opening experience - there is so much you haven't anticipated getting your 'man of an hour creativity reward' earlier this day.
And remember - no matter how weird and rare these paths seem to be, these are the very ones people will follow as soon as they start using your software. Users are vicious when it comes to using our software - they don't add '2' and '2', they just keep adding whatever they fancy with no respect to the inputs we test it against. Unless you know how to change this behavior, data flow analysis can help you 'prove' your software (in a limited but still powerful way).
Use it.
Saturday, November 29, 2008
Covering your tracks - let your codebase follow your skills
We've all been in situations when somebody dug up a piece of code from 1997, in a far far away module, authored by us and still containing some not the best of the breed code construction. We've spent too much time catenating Strings instead of StringBuffers, we left behind empty catch blocks, we double-iterated over a map first for the key and then for the value, you name it. We would never do it now, but there were times when we weren't that expert. When it happens there is usually a lot of joy for others when it's spotted, not so cool for us though.
There are several things we can get out of it: first of all if we want to see our programming skills evolving we just need to review our code in chronological order - it's all documented there. Now as we already know how to fix ourselves a nice evening of memories, full of emotions, recalling how fragile and naive developers we were some day, let's think about some more practical aspects of that discovery. Thus the second: people are still running this code and depend on it, and the third – if your portfolio is to be complete it doesn't look to good.
It seems like a good idea to – whenever you learn something new – review your earlier work and update it according to new information you’ve just gathered.
How would you go about that?
You can read the whole code, spot where you used a 'deprecated' technique and replace it with a 'better-new' approach. A single review of such kind may be months to years in time cost depending on how productive you are, and there is a high risk that your manager will beg you to make sure that you will not learn a single thing afterwards.
Then you can use string analyzing tools: simple 'grep' or any other string find software would reduce this time greatly (if you are using home-made one please make sure that you are using StringBuffers for heavily changing strings). You will still get a lot of noise (false positives) and miss a lot of problems (false negatives), which will require a lot of manual analyzing - nevertheless it will be much faster than reading through the whole codebase.
And then there are dedicated tools for that - greps which have kind of regular expressions based on the structure of programming language you work in. That's the fastest I know. Choosing such tool you need to make sure that (one) you are able to integrate it with your work, (two) you can define the pattern you want to find easily, and (three) you are able to define patterns in a way which allows you to reduce noise to minimum (patterns are not to generic (false positives), or too specific (false negatives)).
Once you have such product integrated with you - as soon you learn something new, you can make sure that code you created in the past follows.
There are several things we can get out of it: first of all if we want to see our programming skills evolving we just need to review our code in chronological order - it's all documented there. Now as we already know how to fix ourselves a nice evening of memories, full of emotions, recalling how fragile and naive developers we were some day, let's think about some more practical aspects of that discovery. Thus the second: people are still running this code and depend on it, and the third – if your portfolio is to be complete it doesn't look to good.
It seems like a good idea to – whenever you learn something new – review your earlier work and update it according to new information you’ve just gathered.
How would you go about that?
You can read the whole code, spot where you used a 'deprecated' technique and replace it with a 'better-new' approach. A single review of such kind may be months to years in time cost depending on how productive you are, and there is a high risk that your manager will beg you to make sure that you will not learn a single thing afterwards.
Then you can use string analyzing tools: simple 'grep' or any other string find software would reduce this time greatly (if you are using home-made one please make sure that you are using StringBuffers for heavily changing strings). You will still get a lot of noise (false positives) and miss a lot of problems (false negatives), which will require a lot of manual analyzing - nevertheless it will be much faster than reading through the whole codebase.
And then there are dedicated tools for that - greps which have kind of regular expressions based on the structure of programming language you work in. That's the fastest I know. Choosing such tool you need to make sure that (one) you are able to integrate it with your work, (two) you can define the pattern you want to find easily, and (three) you are able to define patterns in a way which allows you to reduce noise to minimum (patterns are not to generic (false positives), or too specific (false negatives)).
Once you have such product integrated with you - as soon you learn something new, you can make sure that code you created in the past follows.
Friday, November 28, 2008
Static Analysis - is it ROIng or not ROIng
Today I would like to talk about pattern based code checking (aka Static Analysis aka Coding Standards) - a very powerful technique which allows you to improve quality of your code in no time.
The idea is pretty simple - programming is an activity of coding based on the grammar, which is somewhat regular, and that means that programs will have similar patterns in their code. This is where bad and best practices come from. Now if we have means to analyze the program structure (given by existence of grammar) and we have patterns of different quality - we can promote good over bad patterns and that improves the overall quality of our software.
How would you do that - the easiest technique is to find and eliminate bad patterns in existing code. There is a wealth of products which not only can analyze the structure of your code but have tens or hundreds of such best practices built in. They integrate with our development IDEs, allow for extensions (defining your own rules), manage results of analysis, sometimes have auto-fix feature (apply standard refactoring to replace bad practice with its' better equivalent).
Now, there is a lot of discussion on whether it is necessary to use static analysis in your development lifecycle. The arguments are that this practice is not about chasing errors which would necessarily trigger off at some point of a user experience and kill our application in front of her eyes, this is more about preventing errors by writing in 'better style'. If we look at it from the return of investment side, we have to consider three costs - cost of the product, cost of introduction of the product, cost of using it.
As for the cost of the product - it's usually in whereabouts of several thousands dollars. Just think about it as a vehicle to make junior developer into senior one by putting it at its belt. Do the math how quickly it pays off.
As for the cost of introduction - back in the eighties there may have been some learning curve involved in this (as everything was command line based then), but now it just plugs in into your development environment, you work with results by clicking, it jumps with you to specific places in the code which require modification, explains what to do and why to do it. From time to time I train people on various kinds of testing. Static analysis takes about two-three hours (including rules configuration, suppressions mechanisms etc.). Again I'm leaving the math for you.
As for the cost of using it - processing time is pretty much about the same (up to three times longer) as normal compilation and it suits kind of similar purpose (checks whether code is sound and you can progress based on it with your development), so if you allow your team to compile code as they go, you should be ok with them running static analysis. Than there is working with results - first of all usually there is one click to assess whether violation should be fixed or not - it takes literally seconds. No false positives (of course if software doing analysis is not buggy itself).
And now for the ROI - depending on sources and lifecycle phase we have the cost of single defect removal varying from 1,000$ to 20,000$ and more.
That leaves it as a no-brainer. After three, four defects are removed, we have our price of the product paid off and we are left with developer directly connected to the oracle of coding wisdom.
The idea is pretty simple - programming is an activity of coding based on the grammar, which is somewhat regular, and that means that programs will have similar patterns in their code. This is where bad and best practices come from. Now if we have means to analyze the program structure (given by existence of grammar) and we have patterns of different quality - we can promote good over bad patterns and that improves the overall quality of our software.
How would you do that - the easiest technique is to find and eliminate bad patterns in existing code. There is a wealth of products which not only can analyze the structure of your code but have tens or hundreds of such best practices built in. They integrate with our development IDEs, allow for extensions (defining your own rules), manage results of analysis, sometimes have auto-fix feature (apply standard refactoring to replace bad practice with its' better equivalent).
Now, there is a lot of discussion on whether it is necessary to use static analysis in your development lifecycle. The arguments are that this practice is not about chasing errors which would necessarily trigger off at some point of a user experience and kill our application in front of her eyes, this is more about preventing errors by writing in 'better style'. If we look at it from the return of investment side, we have to consider three costs - cost of the product, cost of introduction of the product, cost of using it.
As for the cost of the product - it's usually in whereabouts of several thousands dollars. Just think about it as a vehicle to make junior developer into senior one by putting it at its belt. Do the math how quickly it pays off.
As for the cost of introduction - back in the eighties there may have been some learning curve involved in this (as everything was command line based then), but now it just plugs in into your development environment, you work with results by clicking, it jumps with you to specific places in the code which require modification, explains what to do and why to do it. From time to time I train people on various kinds of testing. Static analysis takes about two-three hours (including rules configuration, suppressions mechanisms etc.). Again I'm leaving the math for you.
As for the cost of using it - processing time is pretty much about the same (up to three times longer) as normal compilation and it suits kind of similar purpose (checks whether code is sound and you can progress based on it with your development), so if you allow your team to compile code as they go, you should be ok with them running static analysis. Than there is working with results - first of all usually there is one click to assess whether violation should be fixed or not - it takes literally seconds. No false positives (of course if software doing analysis is not buggy itself).
And now for the ROI - depending on sources and lifecycle phase we have the cost of single defect removal varying from 1,000$ to 20,000$ and more.
That leaves it as a no-brainer. After three, four defects are removed, we have our price of the product paid off and we are left with developer directly connected to the oracle of coding wisdom.
Subscribe to:
Posts (Atom)
Posts
