I used to work for a small company that had a single, highly profitable core product. As you would expect, this small company wanted to be a big company with a diversified portfolio of products. The business leadership was continuously proposing new ways to leverage the company’s core abilities to serve other markets and user bases. Sales and marketing would push for fragile contortions of the existing core product in order to pursue various opportunities. These attempts to enter new markets made perfect sense from a business standpoint, but they were playing havoc with the existing system and development team.
As a way to address the situation, the engineering department came up with an initiative to divide the capabilities of the core product into a series of abstract components. These components could then be tied together with a little bit of code specific to the problem domain. We wanted to quickly create independent systems to serve distinct user bases. We would be able to pursue all of these new opportunities without mucking up our existing code base. It was a super sexy idea, and it was very easy to sell internally to both the business folks and the developers.
The first capability marked for abstraction was a portion of the core product that handled a long-running (3 to 5 day) transaction with a group of 3rd party service providers. 5 to 10 percent of the transactions would result in a high-touch customer service situation with our end users. This was one of the vital value-add services of the product, and a core competency of the company. We gave the new component a catchy name, and the journey to create a platform had begun.
How did it go?
A dedicated 3 to 5 person team worked on the new component for over three years. The component was never used by the existing core product. It never saw any significant usage in a production environment. More than three years of development on the new component never yielded a single tangible moment of benefit to the company. The sexy idea of going from product to platform had been a complete failure.
What went wrong?
The new component didn’t have any major technical flaws. It was well engineered, had a solid suite of automated tests and reliable deployment scripts. This was not a component that failed because the software was buggy or unusable. This was an organizational problem.
The team working on the new component never had the motivation to integrate with the existing core product. It was much easier and more fun for the team to just keep adding features to their clean little sandbox. Furthermore, the team was made up of developers relatively new to the company that felt little or no responsibility for the core product, which was older and much harder to work with.
Meanwhile, the teams working on the core product were also not motivated to integrate the new component. The integration always represented more work than simply modifying the capability already in the core product. Time after time, features that the new component already had were added to the existing core product because that path was easier and faster in the short term. To further compound the problem, none of the core product teams had much knowledge of the new component. They had not been involved in the development of the new component, and therefore felt no responsibility for its success. It is tough to convince a development team to spend time to learn new system in order to gain features that their code already has.
As time passed, the problem got worse. As development of the new component continued, it became more aligned with other new development projects and less compatible with the core product. One glaring example of this was how the new component was hosted in production. Two years into development, the new component was moved to the cloud. This move was a good one for the component itself and any future clients. However, the existing core product was still strictly hosted with on-premises servers. The idea of using cloud servers even for a single component of the core product was very controversial within the company. This was due to cultural and regulatory factors more than any technical issue, and had been a thorn in the side of engineering for quite some time. These factors were severe enough that once the new component moved to the cloud, any realistic hope for it to be used with the existing core product was extinguished.
How can I avoid screwing up in exactly the same way?
Decide if the benefits of the abstracting into a platform are really worth it.
Are the needs of your company’s separate products truly identical? In our case, it turned out that the production hosting requirements were different enough to be a problem. We may have discovered other differences after two production systems started using the new component, but we never even made it that far.
Will the complication of integrating with independent platform components overshadow the benefit of re-using the implementations? It can often be easier to maintain two simple solutions than one complicated one.
Who is going to be in charge of retrofitting existing products to use the new platform components? The platform team won’t necessarily be excited about working in the old system, and the old system team won’t necessarily be excited about working with the new platform components.
Is your company willing and able to staff the ongoing maintenance of the new, shared components? It takes a lot of organizational maturity to manage the budget and technical direction of components that serve a variety of initiatives. In our case, one of the initiatives was able to steer the technical direction of the new component to the point where it was nearly useless to the one product that made money. The component could not survive independently, and needed to be a direct part of a more complete initative in order to have a cohesive technical direction.
If you still want to have a go at turning your existing system into a platform, I have a couple of suggestions.
Do not write the stand alone component from scratch. Start with what the existing logic, and make that capability available to other systems. Having real users validate your development changes at every step along the way is the surest method for staying on track. Delivering small, frequent changes to production is just good general development advice, but it is doubly true in this case. Software that does not support a current revenue stream is really easy to abandon. Get value out of your development!
Know that maintaining platform components is going to increase the overall complexity of the systems involved. It will also increase the complexity of the interactions between development teams. Maybe by a lot. By following the “Don’t repeat yourself” principle, you might make yourself much less efficient. Know that the complexity is coming. Be ready to either lead your team through it, or run away as soon as the benefit obviously isn’t worth the trouble.