Spaghetti Architecture and the Importance of Starting Over
I'd first like to show you an oversimplified analogy:
Lets say that you work for a company, and you produce shapes, depending of course on what the market desires. When you first come to market, you see that people want things that roll. So, you build a circle. It rolls all around, and your customers are happy. A few months later, people are more interested in squares; they can be stacked after all. You take your existing product, attach some corners to it, and you release your square. People are happy, your shapes are stacking up to the ceiling. Before you know it, people really want the structural integrity of the triangle! Again, you take your existing product, add some pieces to it, and you now have a triangle.
I'm sure you can start to see what the problem is. With each new major product revision, instead of starting over, you simply add more stuff to what you've already got. Sound familiar? I like to call this Spaghetti Architecture. It is derived from the phrase Spaghetti Code, but instead of just code, it can be applied to larger architectural decisions and even to business practices of non-technical companies.
My suggested fix for Spaghetti Architecture is not to start from scratch with each new revision, but instead to start from scratch at some semi-arbitrary interval, perhaps every few years or before every major version. Sure, that release 3 triangle is ugly, and harder to maintain than a normal triangle, but it would be too much effort to start from scratch all the time.
Starting from scratch is a scary thing. It will take a lot of work to do, the squares and circles you've grown to know over the years will be gone. Everything will need to be tested again. But, it is necessary.
With each additional layer of complexity added your product, the codebase is that much more complex. These problems really start to show for large companies. Suppose you are a bank, and you've been using the same software for a couple decades. At the heart of your business is a FORTRAN application. Around that is some COBOL. Years later you added a GUI for bankers and a bunch of VB6 code is now mission critical. Finally you added a web interface and did that with a bunch of .NET applications. By now, making the slightest changes to your system require tweaks to four different languages, and a ton of testing.
One day a new banking company comes along, lets say Simple. While you've got that circle square triangle product, they already came out with the pentagon everyone now desires. It will take many years for your business to wrap a giant pentagram around its existing triangle square circle, and by the time you're done, it will probably not be the best pentagram it could have been, not to mention hexagons might be the cats meow.
We call the adding up of these shapes technical debt. You see it every day; perhaps your one-to-one relationship turns into a one-to-many. The system wasn't originally intended to do that, so you kinda slap some code on it and call it good. When that relationship later turns into a many-to-many, you've really shot yourself in the foot. The code starts to look horrible and slows down.
Always keep your different systems modular. If you find yourself building a server to perform some functions of your business, clearly define what the application is supposed to do, and the communication interfaces. Always use standardized messaging protocols, the lighter the better. Abstract your databases if you can, it is much easier for a new technology to make HTTP requests than needing to implement a proprietary binary driver. Keep things well documented; you don't need a monolithic SOAP API which explains code to the IDE's of developers. What you do need are human-readable API calls and awesome documentation.
If you keep things simple enough and documented properly, a new hire should be writing apps within a week. Look at GitHub's API. While it isn't perfect, it is documented well and easy to work with. Imagine if all of your internal services are this easy and well documented! How many new features could you be churning out given a minimal number of developers?
It is also important for the companies IT department to be able to prioritize projects. Rewriting an application will not make the company money, but it will keep the company fast and able to adapt as markets change. The same thing goes for regular maintenance, which can slow down and possibly remove technical debt. This will one day make or break the company.
Really, what it all comes down to is this: At any given point in time, you know what your product should be. You don't know what it will be like in the future. With each major revision to the product, you are adding unnecessary complexity to re-shape your product. This complexity adds up and makes development slower. If you don't eventually start from scratch, you will find yourself becoming un-innovative and sluggish due to the effort required to update these antiquated systems.