Blog home

A total rewrite: costly, time-consuming, but worth it?

Joel Spolsky wrote a famous blog post back in 2000 called "Things You Should Never Do, Part I ", where he wrote the following: "..the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch."

Back in 2005 we found ourselves in a situation where we had to decide the future of our content management system. We had been developing content management systems since the company was founded in 2000. Our CMS was developed using Active Server Pages, and consisted of around 80,000 lines of VBScript code. It was moderately successful, being one of the leading content management systems in Norway, and we had an international partner network with partners located on all continents. The vast majority of our partners were located in Europe, US, Canada and Australia.

We landed some high profile clients, for example El-Al, the Israeli national airline, who ran their main website on our CMS for several years, with many million page views per month.

Technically, our solution was very advanced in many areas, but there were some fundamental problems that we struggled with:

  • The whole administration interface only worked in IE, and it used a lot of client side VBScript. In 2005, browser alternatives started to appear (Firefox 1.0 was released in November 2004). Creating a standards compliant interface meant rewriting the whole interface.
  • Since the whole solution had to be easy to run on shared hosting, the whole application was basically a big script, without any external dependencies like COM objects, and far from an n-tier architecture. So it was hard to maintain and extend.
  • SQL overload - Since there was no central point of entry to the database, caching was very hard to do efficiently. This meant that everything had to access the database on every request. It was common that a single page view needed somewhere between 10 and 60 SQL, killing scalability. This is something Drupal still struggles with today, even though they are moving in the right direction.

In addition to the pure technical problems, there were a few other factors that played a role in our eventual decision: 

  • It was starting to become a problem to recruit developers to work on ASP based solutions. And would we be able to keep the existing developers happy in the future continuing to use ASP? 
  • We saw a trend that websites were becoming more complex and in many situations more of a web application than a simple informational website. We wanted a solution that would be flexible enough to be used for these applications, and enable regular CMS users to manage the content in these web applications in the same way they manage the content in the rest of the website. While some of the ideas were based on technology we had in the existing solution, we took those ideas a lot further.

So in early 2005 we made the decision to reduce development on the old ASP based system to a minimum, and to focus on creating a completely new CMS based on Microsoft .Net 2.0, which had just been released as a beta version.

Before the decision, we did consider making the transition gradual, by rewriting the system in .Net, module by module, and improving the architecture was we went along, but in the end we decided against it, mainly due to the messy nature of such a solution, and the fact that it would be harder to make innovative and breaking changes to the core system. Some might argue that it's insane to base such a monumental decision on how elegant a solution is, but remember that we were developing the core of a system that probably would be used for more than a decade. Any technical debt in the core of the system is bound to come back and bite you - hard.

We started development of an early prototype of the core framework back in February/March 2005.  Our original timeline was to test parts of the core framework in a project that was due to be launched in September 2005, and release the first version of the full CMS in the summer of 2006.

We delivered the project with the prototype framework as promised, but as we continued to develop the rest of the framework, we saw that it took a lot longer than anticipated. There were several reasons for the delays, but the most important were:

Tight finances.
We're a relatively small company with finite financial resources. From the start of the project, the project was financed by profit from the consulting part of the company that implemented our CMS in different web projects, and international license revenue from partners around the world. This limited progress in the project, but luckily we qualified for R&D grants from Innovation Norway, several years in a row, which helped quite a bit. It gave us the required time to make the base framework very solid and well-thought out.

R&D is time consuming and difficult to estimate.
When we started the project, the goal was to create a new system that would fix the shortcomings in the old CMS, and be prepared for the changes in the marketplace that we anticipated. How to do that, and exactly what had to be developed wasn?t clearly specified. We had some ideas on what the system had to be capable of, but it's a long way from an idea to a complete and tested feature. This resulted in a lot research of various technologies and writing quick prototypes to test various implementations.

Feature creep.
The lack of a clear specification also made it hard to limit feature creep. Since we didn't have a clear picture on how the different features would be implemented, we often discovered that when we implemented one feature, it "forced" us to implement another feature.

We started the pilot project on the new CMS in October 2006. It was due to launch before Christmas 2006. In the end we launched it in March 2007. By that time, the framework was fairly stable, but the CMS user interface was very raw, and it missed a lot of functionality that we had in the old CMS. Truth be told, it was also quite buggy. But it was mainly the editors that noticed the rough state of the CMS. It's still online today, using a much improved version of our CMS.

Since the first project launched in 2007, more than two hundred web projects using our new CMS have been completed in Norway. During those projects, the system has gradually matured and grown. The experience we gained from using the system for real projects from such an early stage has been invaluable in the later stages of development. Looking back at all those projects, it has been a very difficult and challenging period, especially in the beginning, but we're finally in a position where we're very happy with our product. During the last couple of years we've seen that the difficult decisions we made back in 2005 and 2006 are starting to pay off.

Conclusion
Do we think Joel was correct that a rewrite is "the single worst strategic mistake that any software company can make"

Not categorically, but we experienced a lot of the hardship and trouble that he talks about, so we can easily see how many companies could get into trouble during such a process. If the board of directors had known in 2005 that we wouldn't be ready to re-launch in the international market until 2010, I doubt we would have gotten the green light for the project. But now we clearly see that the end result is so much better than if we had done a gradual transition. That might not be true in all cases. We had a clear idea on how we wanted to improve the existing solution. If you?re rewriting an application just to change platform, without any fundamental improvements that would be hard to do in your existing codebase, I agree totally with Joel. This fits together with another quote from Joel:

"It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time."

If this is true, then a rewrite of an existing application will always be a huge mistake. Joel is basically saying that developers don't benefit from previous experience, a notion that I think is plainly wrong... All the good developers I've ever worked with have been more than capable of seeing what works and what doesn't work in a solution, and would have been able to do a better job on the second try.

In our case, the technological shift from VBScript to .Net was also a major factor in making the end result much, much better than the old version.

There are also other examples of total rewrites that have been successful. Apple's transition to OS X is a very good example. The classic Mac OS was an operating system with some good ideas, but the core architecture had a lot of problems. Did Apple make a bad decision to start from scratch with OS X? I don't think so. They brought over and improved the best bits from OS 9, and merged it with the solid Darwin core (much like we did). During and after the switch, they had a few rough years, but since the launch of OS X, they have been on a more or less constant upwards trend.  With the international launch of Webnodes.com behind us, I'm confident we'll follow the same trajectory.

What do you think? Did we make a mistake in rewriting the whole system?

In the coming weeks we'll go into more detail on the different features we've developed in WAF (Webnodes Adaptive Framework).

5/20/2010
Posted by: Vidar Langberget
Categories: CMS  Webnodes.com 
  
Comments (33)
Not sure
by Brian
5/21/2010 - 12:36 AM

i'm surprised you decided to start from scratch. Upgrading your old system might have been messy to begin with, but you would be in a much better spot much sooner.

Every situation is different
by Tom K
5/21/2010 - 12:43 AM

I'm sure there are situations where both viewpoints are valid. The trick is to evaluate a situation correctly so that the correct route is taken.

Evaluation of every situation will be different
by Srikanth Shirodkar
5/21/2010 - 12:41 PM

I fully understand where Vidar is coming from.

I have seen other situations where a full re-write has been successfully done - the most common case being that the application grew dynamically with the business itself over a decade or so.

Design principles or lack of them - from a decade ago, spaghetti code, User Interface design no longer being relevant and being tightly coupled with business logic, subsystems being devolved, small features taking very long to take to market - these are most common reasons why a re-engineering is justified.

Sometimes it makes no sense to putter along in this way. Every case has to be evaluated specifically - so I would not agree to Joel's categorical statement.

A risk analysis - with mitigation and contingency conversions to time and budget, a testing strategy, instituting a user group for decision making, prototyping of the UI, careful planning of schedules, resource loading - these go a long way in re-engineering projects.

Mac OS X is bad example
by Dmitry Leskov
5/21/2010 - 7:01 PM

Mac OS X is based on the Mach kernel and BSD, relying literally on decades of Unix development before and during NeXT times:

http://en.wikipedia.org/wiki/History_of_Mac_OS_X

Many factors here
by Jan Erik Paulsen
5/21/2010 - 8:00 PM

Joel Spolsky has no Innovation Norway money in his pocket. A rewrite could paralyze the company and restrict cash flow. And what happens when customers understand that CMS 2.0 is right around the corner.. but not quite. Suddenly CMS 1.0 is selling less. Two products could result in twice the support burden. Data migration might become an issue. Some customers might refuse to upgrade. Developers with inside information might quit.

Webnodes decision is fine because Microsoft dropped support for old school VBScript based ASP. It is logical, future compatible and apparently financially sustainable. Joel Spolsky simply believes feature creep and new bugs will kick him out of his skyscraper office in New York. His operating margins are probably thinner than yours. He has to cover health insurance, etc.

http://www.joelonsoftware.com/items/2008/12/29.html

A rewrite could imply that you already have a development process problem. You are blaming the house for being crocked. Bad house ! Or you grew too fast and added to many features too soon. If so, a rewrite will probably produce a better product.

PS
Why is there no URL field in the comment form :)
http://labs.teppefall.com

The nature of software engineering.
by Salman Haq
5/21/2010 - 11:35 PM

There is a rule of software engineering that all programs are constantly being re-written. I've been through three re-writes of our product in six years. Every time we started, we knew that the new version was going to be revolutionary compared to the old version. It turns out that at the end of the day, most customers viewed the improvements as evolutionary. But that's okay because the new versions did improve significantly upon the previous versions.

So how did we successfully complete this feat not once but three times? A solid customer base and strong revenues. They were critical in supporting the R&D team and their endeavor. Without it, we would have sunk ourselves (and we probably did come close but I just don't know it).

Our reasons for doing the re-writes were the same as yours - technical obsolescence, unmaintainable code, and engineering mistakes. Each re-write was fun because we got to experiment with new technology, learn new things, and apply old lessons. At the same time, it was also quite taxing because of the pressure. Every software project takes twice as long, even when you plan for the extra time.

http://bitshaq.com

Not Quite
by Simon
5/22/2010 - 3:22 PM

Actually, Joel redacted this quite early on and in fact when talking to Jeff Atwood changed his opinion openly.

Read this post from 2008: http://www.codinghorror.com/blog/2008/10/programming-is-hard-lets-go-shopping.html

The talent to re-write is a reflection of the team
by chase turner
5/22/2010 - 3:35 PM

I recall Dr. Alan Kay saying the Smalltalk-80 development team would re-write the Smalltalk kernel -- from scratch -- every 6 months or so in the late 1970's. In this way, all members of the development team were "current" with all aspects of the system, and the forced re-write from scratch ensured that salient and necessary features continued on, whereas less-suitable features got dropped. Not to mention, it also allowed them to refine their infra-structure for build/deploy/test/etc.

15+ years later, and my having worked in an enterprise-sized development shop where the sheer scale of the system, coupled with an increasing pace of body-shop assignments, code rot, forced-re-engineering of existing code under pressing customer escalations, multiple-branch code streams with significant numbers of backports and other issues, etc … there is a significant attraction to the notion of re-writing from scratch, setting aside a budget for a 5 year "leap frog" team whose mission is to do exactly that … and then take over the production release for the next 5 years while another team "leapfrogs" to the future system.

All rewrite, all the time.
by Reader
5/22/2010 - 3:44 PM

Although rewriting code is essential, a FULL rewrite indicates deeper problems. It indicates an inability to rewrite offending code.

In my own development, I will rewrite any module and it's interface whenever I think of an improvement. This can break everything else that uses it, but usually only takes a few minutes to track down all those broken places and fix them to use the new interface.

The result is that the product very quickly becomes more and more streamlined until it is a piece of code I can be truly proud of. Something that is sleek, efficient, and easy to modify. It also allows you to recognize areas where more code can be reused, as often the rewritten code is shorter and sweeter, leading to fewer bugs in the end.

If you're afraid to make changes like this for fear of breaking your application along the way, I believe you're destined to actually take LONGER overall simply due to working with a more clumsy base of code.

Bottom line: rewriting code SAVES time in the long run.

Mac OS Comparison is bogus
by itomato
5/22/2010 - 5:07 PM

Mac OS X doesn't represent a complete OS rewrite - that would have been Copland.

Instead, the development manager brought in to oversee the (hopelessly behind schedule) Copland OS development recommended "Apple look outside the company" for a solution.

Openstep would be that solution - which itself had been evolving since 1987, where it began as a 'from scratch' effort.

Mac OS is a rewrite and extension of Mach and NeXT libraries to work on newer hardware.

by Xeno
5/22/2010 - 5:08 PM

Have to agree. We are a financial company. When I came on, there was a critical app done in Lotus using MySQL and all the data was in first notmal form, unchecked for data integrity and redundancy all over. The developer initially had no clue about Db development and had decided to handle his own primary keys, didn't understand foreign keys, and many other problems.

The front end was a spaghetti coded mess from a prepackaged solution that was so bad it took over a minute to load pages and the entire thing would crash daily.

The only solution was to rebuild not only the database and their structure which took a redundant database in 1st normal form to third normal form and reduced the size (by introducing relationships where their were none) from 2GB to 200 MB

Did anyone thank me though? No. Everyone ended up being pissy about the entire thing even though everything worked immensely better. They had no idea what kind of mess they had created for themselves and what I had just delivered them out of.

So sometimes, you HAVE to rebuild from scratch when the original team of devs build an app was so bad that the application no longer supports the companies needs.

Mac OSX is a complicated example
by Peter Desnoyers
5/22/2010 - 5:28 PM

The transition from OS 9 to OS X was even more frightening than the one from Windows XP to Windows 7, but there wasn't a Vista to put a public face on it. I can recall seeing an early demo of Pink (the first try at a re-write) in 1990, before the earthquake, but it wasn't until 2001 that they released OS X, derived from an entirely different code base.

For more details you can check out http://lowendmac.com/orchard/05/1026.html

Not surprised.
by NH
5/22/2010 - 6:12 PM

I have been through a similar project - but the code I had to sift through was more aged - up to 25 years old in some cases.

More can be read over at Slashdot: http://developers.slashdot.org/comments.pl?sid=1661266&cid=32306026

by Greg C
5/22/2010 - 7:39 PM

I'm in the process of trying to get my company to let us rewrite our software. I will not be able to convince them of a full all at once rewrite because of the time involved and the lack of progress in our current code base during this process. The good thing is logic in one language is not so different from another in most cases. The rough part for us will be 3rd party GUI components that won't work and will have to be replaced. I just wish I could convince my boss that it is worth the time investment. We spend so much time now on upkeep because of the rats nest of code written by horrible long ago fired programmers.

by James Schend
5/22/2010 - 7:49 PM

A couple things here:
1) Everything you said about Mac OS is wrong. Every single sentence of it.

The only feature (I can recall) Apple ported from Classic to OS X that wasn't already in NeXT was the Labels feature in Finder, and they certainly didn't improve it in the process. Dozens of Classic features were never ported, and the entire spatial philosophy of the Classic OS was thrown in the trash.

I actually switched to Windows during that period. (I gave them until 10.4, but when it was obvious that spatial was gone, I jumped ship.)

2) So let me get this straight: the only reason you stayed afloat during the rewrite was government loans? And you're telling us that Joel was *wrong*? Sounds like he was spot-on to me.

To your point about developer experience, developers (well, here in the US at least) come and go. If you're relying solely on developer experience over a 2 year period to ensure the quality of the product is better than the previous product, you're gambling. Maybe it's a safer gamble in a country where you can get government loans to bail your ass out of dumb decisions, but here in the US it's more sink-or-swim.

I do agree that Joel's article is flawed, it doesn't list exceptions. (And I believe exceptions exist.) But it's just advice, not the bible... you're not supposed to blindly obey anybody's advice, you're supposed to make your own decisions.

But this example leaves very little to the debate. From my perspective, you've done nothing but prove:
* Joel is probably right
* Your company is very, very lucky

Enterprise Architect
by Scott S. McCoy
5/22/2010 - 7:54 PM

I've faced this problem a good many times over the years, even in fairly large volume. The only time I've seen a rewrite really produce a reasonably improved project — it was indeed not really a rewrite at all. What happened is rather a new back-end system was built to supply not the same, but a fundamentally improved functionality which was similar enough in nature to be controlled by the same control interface. The back-end system launched independently and through a variety of integration points ran in parallel to the existing back end, with new distribution partners integrating with the new back end. The next major project included rebuilding the user interface. In the mix, the new system was on an entirely different platform (it moved from an ASP/ASP.NET system to Java). This large, iterative cycle resulted in a far better product. And starting with a new back end meant the most important changes happened first, allowing the business to scale in ways that weren't otherwise possible.

by Waldo Nell
5/22/2010 - 8:13 PM

I agree with the author. And I agree with some commenters. Deciding when to rewrite software is a delicate choice that depends on several factors.

Firstly, the reasons why a rewrite is a good choice:

1) Old system is based on old / legacy technology that is not providing the infrastructure necessary to grow the product (I have experience in this case where we spent 1.5 years writing a large rebates system - it was developed in 4GL/Informix and we rewrote it in J2EE/PostgreSQL - it was a huge success and on time but this was because we had fixed requirements and little scope creep, and a different lead developer)
2) Bad architecture choice initially, and due to experience you can rewrite it to be more flexible, modular and extendible.
3) Unmaintainable code written by unexperienced developers.
4) Bad platform choice - I have a client where I developed a sales leads system using Java swing as a rich client app. I warned them back in 2003 that the right way is a web based system. They did not believe me at the time and was worried about performance issues. 7 years later and we have endless trouble with the client/server version as people try to run it across a WAN, the architecture does not support firewalls that well due to Java's RMI dynamic port issues, and the performance was never optimised for WAN purposes. The architecture does not lend itself to further optimisation. So 7 years later and the client is ready to invest the time and money to port the system to a web based system. In this case the language stays the same, but we are moving from a rich client/server based model to a web based platform. Some code can definitely be reused (maybe 30%) as the whole backend was properly separated in a 4 tiered system. But as you know a lot of the work is in the front end and the user experience, so this has to be rewritten.)

There are several reasons not to rewrite code. Just jumping from say .NET to J2EE or vice versa, would not necessarily help. Jumping from assembler to .NET will if there is no performance benefits, as you would gain the benefits of higher levels of abstraction, easier code maintenance, rapid development, portability etc.

Like everything in life one cannot state these things in absolutes. It depends on context.

Re: A total rewrite: costly, time-consuming, but worth it?
by Vidar Langberget
5/22/2010 - 8:24 PM

James Schend:
1) I never said they brought over all the features from OS 9. I said the best parts. You might think they didn't bring all the best features, but do you disagree that it was their intent to bring over the features they believed were the best?

A few people have commented on the blog post and said that Apple didn't write OS X from scratch. Well, I didn't say that either. I said start from scratch. It depends on how you define "start from scratch". No code, AFAIK, was brought over from OS 9 without changes. Yes, as I mentioned, they used Darwin/NextStep as the foundation for the new OS, but do you know many operating systems that haven't used any already existing components?

You could of course debate the degree they used parts from NextStep compared to other new operating system, but that's missing my point.

My point was that changing the platform/codebase can have a huge impact on the success of a company, which is why I believe we made the right decision to rewrite our application.

2) The support from Innovation Norway was basically tax credits for research. We spent time and money to reseach specific parts of the architecture, and the following year, we would get back a certain amount of the taxes we paid the previous year.

The company would have survived, and we would have been able to complete the project without the support from Innovation Norway, but the support enabled us to spend more time on certain parts that we and Innovation Norway thought were innovative. We hope and believe that those key strenghts can be the determining factors for our future success.

As for the experience of the developers, I believe that's a key part of the whole equation. The cost of loosing a good developer is huge. I know the labour market in the US is more volatile, and people change jobs more often than here in Norway, but there are ways to keep developers happy in a company, even in the US.

by James Schend
5/22/2010 - 8:35 PM

Vidar: Unless you're using an extremely basic definition of "features" (like: windows, menus, cursors), then no Apple did not bring the best features from Classic to OS X. Heck, Classic's best feature was its spatial design, which has never been present in OS X.

I think it would be a hard case to make that Apple's "intent" was to satisfy Classic users, since OS X turned out so different from Classic, and most unique Classic features has never been ported to it. If that was truly Apple's intent, then they've failed completely.

Obviously their intent was to make a good product. And I'm not arguing that OS X is a bad product; obviously it's not.

But Apple didn't rewrite Mac OS from scratch; they also didn't start from scratch. There isn't a single technology in OS X that didn't already exist in NeXT... all Apple did is buy a competing product (and their developers!) and re-brand it as "Mac OS."

As for retaining developers: No, you can't retain developers if you're broke because you don't have any customers because you haven't shipped a product in a full year and your competitors are eating your lunch. That was the point.

Your company was lucky; you were able to retain customers by getting government money (regardless of whether an accountant would call it a loan or not... whatever.) Or possibly because the job market for developers in Norway sucks.

Either way, that wouldn't work in the US or, I suppose, most parts of the world.

Burning the Disk Packs
by Casey Ransberger
5/22/2010 - 8:38 PM

I think Joel is mostly right, but he presents his argument as black-and-white in a world that isn't. Sometimes burning the disk packs is a good idea: especially when one is dissatisfied with the architectural status quo: and if you think about it, we're talking about the very small overlap between research and business. Research is a very different kind of investment from what business people are accustomed to making, because the point of research is to learn, not to ship a profitable product. Research tends to have a very long tail in terms of return on investment. See the 40 years we've spent in industry failing to arrive upon the architectures which were discovered at Xerox. When you burn your disk packs, you're making an investment in research with the expectation that your research will inform a successful product -- recognizing that there's really two projects involved helps to explain the risks involved.

Re: A total rewrite: costly, time-consuming, but worth it?
by Vidar Langberget
5/22/2010 - 8:51 PM

James Schend: The job market doesn't suck at all here in Norway. I'm 100% sure I could get a dozen job offers within a week if I tried. But I like the job, and I like to develop a software product and be part of the international product launch and help the company grow. And last but not least, the pleasure of seeing a lot of people using the stuff you've developed is very rewarding.

We've tried to hire developers several times during the last year, but the number of qualified applications we get is very low.

As for government money, I understand that it might be hard to get that in the US, but it's much easier to get investors in the US than here in Norway, so I think it's fairly even..

Rewrites can be good or bad. It's all in the Plan.
by Fred Mitchell
5/22/2010 - 9:25 PM

Whether a rewrite is a good idea or not depends. It's all in the planning.

You're a growing company soon to face scalability issues, new feature demands, new customer issues, new competition, and the like.

You have a software team comprising members with various levels of expertise and temperament. You have the owners of the company that probably doesn't understand or care about all the technical details. And you have managers whose necks are on the line pending the outcome of the project.

And there is NEVER enough time. NEVER.

Having a good plan will be the determinant of whether you will be a shining star or be broken. Having vision and foresight are essential. Having a good motivated team is as well.

To say it's a non-trivial problem is a gross understatement.

Also, another issue you are up against is that the legacy code most likely is poorly documented, and the original people who wrote it are no longer around. My last project was literally plagued with this issue -- scripts running on servers whose details were unknown -- yet everything would come crashing down if you switched off those servers! The scripts themselves spaghetti code, and in my case, really horribly written PHP code being activated by other scripts using lynx or elinks browsers to pass commands along. I kid you not.

But at the same time, there's a lot of business problems that were solved and implemented by the badly-written under-documented code. All sorts of unstructured wierdo crap that was essential to the business operation.

Been there.

Sometimes you simply don't have a choice but to rewrite. But to do so without solid engineering and planning is a big mistake. One writer mentioned that there's no guarantee that the rewrite will be better than the legacy. IT HAD BETTER BE, and you had better be able to SHOW that through your plan.

by NuShrike
5/22/2010 - 9:30 PM

Joel was right and wrong. For Netscape, it was strategically bad but for Gecko, it was the most successful decision ever.

Not only did it break the monopoly IE had on the market with the now near ubiquity of Firefox, but it also broke open chances for other engines to come to fruition such as KTML->WebKit.

Sometimes it's necessary to bit the bullet and dig deep just that most business managers are neither trained or too risk adverse for the long-term strategic planning necessary for game-changing decisions.

Note how most car companies refused to rebuild the petro-burning car platform from its century of development, and it took some others billions of dollars and a decade of work to come up with a new sustainable hybrid system that totally broke those original companies' backs when it came to the PR and economic crunch.

Long-term changes are hard, but looking only at the short-term horizon is also a strategic mistake.

All things are relative
by Bjorn
5/22/2010 - 9:32 PM

I'm happy that it worked out for you because you took a big risk. But the benchmark should really be - where would you be today if you had decided to not start from scratch but instead put all that time into a serious refactoring the existing code? My firm belief is that you would have years of time to spare. But that is based on 10 years of professional personal experience and nothing else..

Mac OS X just rebranded NeXT??
by Nickster
5/22/2010 - 9:38 PM

"There isn't a single technology in OS X that didn't already exist in NeXT..."

*cough* Quicktime? Java? Quartz? Carbon?

"all Apple did is buy a competing product (and their developers!) and re-brand it as "Mac OS."

Oh, of course. That's why it was released immediately after Apple bought it...oh, wait, it took over 3 years of development, didn't it?

Sometimes there's just no choice
by Badtux
5/22/2010 - 9:53 PM

In the mid 90's I worked on a software system that was written in a DBASE-style database and tied closely to the Xenix operating system. Clearly we had no choice but to rewrite from scratch, since you simply can't migrate from a flat-file data management system on Xenix to a SQL-type data management system on Linux without rewriting. The current solution by that employer is based upon Java and MySQL -- completely different from what the mid 90's solution looked like. Was the transition difficult? Sure. But you gotta do what you gotta do. Obsolete is obsolete, in the end, and there simply isn't any migration path from dBase BASIC to Java SQL other than a complete re-write.

Re: A total rewrite: costly, time-consuming, but worth it?
by Vidar Langberget
5/22/2010 - 10:41 PM

Bjorn: I agree that to determine if we did the right thing, you need to look at the result. But do you take a look at where we are today, or where we are in two years? The answer might be different.

We signed our first partner in Portugal yesterday. If the current trend continues, we'll have the same number of international partners in a few months as we did back in 2005.

by Al
5/23/2010 - 7:10 AM

"..the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch."

another example of bullshit blanket statements. so 2000! it is not hard to imagine an article stating that "single worst mistake in software company is to hesitate to make radical changes to the code when it is absolutely required". statements like these are meaningless and potentially harmful.

Al
visualcriteria.net

the design must fit
by Simon Asselbergs
5/23/2010 - 10:43 AM

As an owner and also productmanager of a small company you can think of a very good solution and have a very sound productstrategy and developmentpath. None the less, business is dynamic so sooner or later the flexibility of the strategy will be tested by scenario's you didn't think of or developments in the business going a different pace as you anticipated (by immense success or the lack of it). This means that the design becomes less fitting for the real solution, it needs a overhaul. There are 3 ways to deal with this:

1. Ignore it. The quality of the code will become a mess, maintenance becomes more a problem and also custom work on it for clustomers. It maybe more cosst effective on the short run, but customers will recognize that the product lacks quality. If you have a monopoly, it's maybe possible to pull it off, but don't expect much enthousiasm from the people in or outside the company. This kind of development gives me reasons to hate software development companies.

2. Rewrite from scratch, but with good research on standing bugs and required features, business cases, usecases and design first. It is very costly, but if you have a good design specification it makes it much easier to get it right this time (and if it wont the next time).

3. Rewrite from scratch important cores, every 6 months. This also means refining(rewriting) design specs. But after a while you will see that certain parts of the api's will be prove to become very stable and the product becomes very flexible (less complexity) and performant, making it easy to do bug fixes, to adopt features that clients really need and to weed out all the unused stuff. It may look expensive, but you earn it back in maintenance and competitiveness. Quality on the long run always wins. The short term is the same challenge to every small business.

by Michael Ansley
5/23/2010 - 2:58 PM

I think this decision was not a 're-write' decision, but rather a 'change the technology' decision, and the reasoning behind it was solid. Although superficially there is a divergence from what Joel discussed back in 2000, in the end, I think both positions can be reconciled; Joel's real issue was re-writing because 'it is a mess'... which does not seem to be the case here. Well done on successfully managing such a change. I have been through at least one similar project and it was a financial disaster, and very nearly a strategic disaster for the company concerned.

by Sam Watkins
5/24/2010 - 2:11 AM

I actually believe that the only way to achieve really high-quality software is to evolve it as far as you can, then rewrite it from scratch, with reference of course to the previous version. There are some things that can't be fixed by tiny evolutionary steps, you will need to redo the whole thing, perhaps several times, or at least make some massive large-scale changes to the system such as rewriting it in a different language or a different style.

Financial sense vs quality
by Bill Lambert
5/24/2010 - 5:01 AM

I am a big fan of rewrites, but I will qualify that by saying that I mostly work on small projects. The kind of apps that start out messy in the first place, are developed in a mad rush, and then shamefully extended far beyond their means over the next few years. It is in those particular scenarios that I think a rewrite is good housekeeping, and it sounds like the situation Webnodes was in.

Ask any developer what the greatest flaw is in their biggest app, then ask them how they could fix it. Most of them will either say it's unfixable, or that it requires starting over scratch, with the wisdom they acquired over the many years of supporting and patching the old app.

If you, as a programmer, never revisit old problems with new solutions, you are stagnant. You are relegated to writing the same thing, making the same mistakes, every single time. Don't lie and tell me that, at the end of a development cycle, you didn't feel like there were things you could improve, or would have done differently to make your life easier. Most often, when I deliver a project, I feel the desire to start writing the v2.0 right away. It would be faster, easier to use, easier to integrate... if you go from start to finish without learning anything, you're doing it wrong.

Joel Spolsky's comment isn't wrong or right, it is simply valid from his perspective. He is a very vocal member of the community, but how many of you have actually used his products ? Joel talks mostly about how he is awesome and how everyone else is wrong or stupid. How often does he write about his failures ? Or should we believe he never makes any mistakes ? I personally cannot relate at all to his point of view.

I put far more stock in Jeff Atwood's words and opinions over at Coding Horror. Maybe it's because he mostly writes from a developer's perspective, not that of a boss. I don't know Jeff's stance on code rewrites, but I'd like to think it would be a more balanced approach rather than religious mantra.

Joel was absolutely right, but...
by Christopher E. Stith
5/26/2010 - 9:53 AM

It's a strategic mistake to decide to do a complete rewrite. However, sometimes we have no strategic decisions to make. If you have no choice but to rewrite and hope, then that's not a strategic decision. It's a last-ditch effort.

(Where do you folks figure "last-ditch effort" comes from, anyway? That's right -- when the enemy in a large dug-in battle crosses the field into range of your last defensive ditch, it's up to effort to save you because your strategy has already failed.)

Making the "choice" between having only an obsolete product you can't figure out how to rewrite incrementally and risking everything to rewrite it from scratch is only really a choice if you have the proverbial golden parachute. An incremental rewrite is the choice to make unless you just can't make that choice.

Add comment
Title:
Name:
Email:
Comment:
Captcha Image