Incent GmbH, headquartered in Berlin, Germany, operates as a platform that extends deals and promotions through an employee benefit program to its clientele. Esteemed companies like Siemens, Vodafone, Rewe, and McDonald’s register on this platform, enabling their employees to access and explore a diverse range of offers available.
Founded in 2014, this platform was developed using the Symfony2 framework and PHP 5.6. Its operational infrastructure relies on a MySQL5 database. The application is hosted on AWS, utilizing a replicated cluster comprising three EC2 machines, each operating on a Debian5 operating system.
Problems
The Incent platform suffered from several issues related to the features and functionality that it offered to its audience.
Technology
In 2022, PHP5.6 was phased out, leading Zend to recommend migration to PHP 7 or 8. Simultaneously, Symfony2 became outdated, necessitating an upgrade to Symfony 6 due to the lack of community maintenance.
Complicating matters, Debian5 reached obsolescence, prompting the need for redeployment on newer versions. The SSL community’s shift from TLS 1.0 to TLS 2.0 posed challenges, particularly as Debian5 lacked the required packages for the upgrade, resulting in CURL failures over HTTPS.
The situation intensified when one of the three EC2 machines broke down in mid-2022, emphasizing the critical need for a comprehensive upgrade.
Performance
The application initially adopted a monolithic architecture, combining the website frontend, administrator back office, and backend into a single entity. Hosting on robust AWS servers with high CPU and RAM capacities led to significant costs.
Challenges arose as certain features became outdated and struggled to handle increased data volume or relied on unmaintained third-party libraries. The application faced collapse during peak hours, particularly when dispatching newsletters, leading to disruptions.
The system resembled a ticking time bomb, vulnerable to imminent issues and interruptions.
Standardization and Documentation
Throughout its lifespan, the application underwent maintenance by multiple development teams, leading to the introduction of divergent coding standards with each iteration. The result was a cluttered codebase, making it challenging to comprehend both the functionality and the reasoning behind design choices.
Further complicating matters, none of the previous teams documented their work, leaving the application devoid of comprehensive documentation.
The absence of documentation posed a significant challenge, requiring the remaining team to decipher the intricacies of the codebase and align it with contemporary standards during the transition.
Unscalable
Incent faced challenges in expanding its market reach and boosting revenue due to the obsolescence of its technology stack. Introducing new features became difficult, hindering scalability within the existing infrastructure.
A crucial decision was made to reconstruct the application and pivot from the original technology stack. This strategic shift aimed to facilitate the seamless integration of new features and position the application for sustained growth and innovation.
Cost
The application faced high server costs and performance issues, prompting the previous engineers to introduce a new layer with Redis and Memcached for speed and shift logging to a MongoDB database. However, their solution was flawed.
Instead of improving the code, the team rented large clusters of MongoDB Atlas and instances of Redis, Memcached, and ELB load balancers on AWS. This led to a substantial monthly cost increase, reaching 4,000 USD.
Incent ended up paying double the potential cost at an average of 8,500 USD per month.
Solution
In addressing these challenges, my recommendation involved a comprehensive overhaul of the application.
The entire system overhaul was an extensive and carefully planned strategic reconstruction. It encompassed modernizing both the frontend and backend systems, adopting cutting-edge technologies to ensure scalability, performance, and flexibility. The frontend development utilized Vue.js for both the back office and the main website, creating dynamic, responsive interfaces that improved the user experience and enhanced system interaction for operators and end-users alike. The shift to a microservices architecture enabled the modular, independent development of various components, significantly improving the system’s maintainability and scalability.
The previous version of the application could support up to 200 concurrent users, with an average page load time of 4.5 seconds. With the new system in place, performance was dramatically improved—the application now supports up to 800 concurrent users while serving pages in under 2 seconds. This boost in efficiency was the result of optimized resource allocation, better load balancing, and improved backend operations.
Several key improvements were introduced as part of the overhaul. AWS load balancers were replaced with a unified traffic management solution using Nginx and HAProxy. Database indexing was optimized to reduce the overall cluster size. The entire application was deployed on a Kubernetes cluster, with Redis and Memcached migrated from AWS-managed services to run as components within the cluster.
This redesign not only boosted performance but also resulted in a 50% reduction in hosting costs.
Additionally, a new shop feature was integrated into the platform. The shop is designed to attract new customers, enhance the shopping experience, offer a wider variety of products, and expand the company’s revenue streams. The company also aimed to offer the shop as a licensed syndicated plugin for its customers, enabling them to integrate the shop widget into their employee portals.
The shop was built from the ground up and integrated with multiple product providers, payment gateways, and third-party sales and reporting tools. Node.js, with the Fastify framework, was used for backend microservices, while MongoDB and Elasticsearch handled data management. The interface was developed using Vue.js. CI/CD pipelines were set up via Bitbucket pipelines, and the shop was hosted on the same Kubernetes cluster as the existing portal.
Stages
I led the entire reconstruction of the application, strategically staging it across multiple phases, and successfully completed the project within two years.
Stage 1
This stage took 6 months
Admin Back Office Reconstruction
- Rebuilt the admin back office using VueJS.
- Simultaneously developed a new backend using PHP8 and Symfony6.
- Recreated all necessary backend operations as APIs to support the admin back office functionalities.
Code Cleanup
- Removed legacy code of the backend operations and the admin back office from the existing application.
- Consolidated the legacy application to contain only the code related to frontend websites.
Containerization and Deployment
- Dockerized the newly reconstructed admin back office and backend.
- Deployed all three components (Admin Back Office, PHP8 Backend, and Dockerized Frontend) on a Kubernetes cluster for efficient and scalable management.
Stage 2
This stage took 3 months
Frontend Website Reconstruction
- Rebuilt the website frontend using VueJS.
Extend the new Backend
- Recreate all backend operations from the old legacy on the new backend that was created in stage 1 as APIs
Containerization and Deployment
- Dockerized the newly reconstructed frontend website.
- Replace the old Dockerized frontend with this new version.
Stage 3
This stage took 3 months
Microservices
- Break down the new backend into several microservices
API Gateway
- Introduce an API Gateway in front of all the microservices and have it handle authentication, authorization, access permissions, cloud awareness, and traffic management.
Containerization and Deployment
- Dockerize each microservice and the new API Gateway
- Deploy these new components on the kubernetes cluster
Stage 4
This stage took 9 months
Shop Backbone
- Developed new microservices for the shop component using Node.js and the Fastify framework.
- Installed, configured, and integrated MongoDB Atlas and Elasticsearch IO, enabling the backend microservices to handle all shop operations across both platforms.
- Designed and built interfaces for operators, third-party product providers, and merchants, allowing them to inject and synchronize their products and catalogs into the shop’s database.
- Integrated third-party services, including sales platforms, payment gateways, and marketing tools.
- Added support for various purchasing methods, such as credit cards, PayPal, vouchers, coupons, and more.
Shop Interface
Designed a new shop interface using Vue.js. The same codebase was utilized to package the Vue app into both a website widget and a standalone single-page web application.
Containerization and Deployment
- Dockerize each microservice and Interface
- Deploy these new components on the kubernetes cluster
