--- title: "Automated Syndication System for 25+ Real Estate Portals" description: "Full ETL pipeline extracting real estate listings from PIM Akeneo, transforming them into each partner's required format (XML, CSV, JSON), and delivering to SeLoger, LeBonCoin, BienIci, LogicImmo and more - 4 years of continuous operation." locale: "en" canonical: "https://portfolio.josedacosta.info/en/achievements/diffusion-portails-immobiliers" source: "https://portfolio.josedacosta.info/en/achievements/diffusion-portails-immobiliers.md" html_source: "https://portfolio.josedacosta.info/en/achievements/diffusion-portails-immobiliers" author: "José DA COSTA" date: "2019" type: "achievement" slug: "diffusion-portails-immobiliers" tags: ["PHP", "Symfony", "Akeneo PIM v2", "REST API", "XML", "CSV", "JSON", "FTP/SFTP", "GitLab CI", "Docker", "Kubernetes", "MySQL"] generated_at: "2026-04-23T15:45:28.432Z" --- # Automated Syndication System for 25+ Real Estate Portals Full ETL pipeline extracting real estate listings from PIM Akeneo, transforming them into each partner's required format (XML, CSV, JSON), and delivering to SeLoger, LeBonCoin, BienIci, LogicImmo and more - 4 years of continuous operation. **Date:** January 2019 - 2023 **Duration:** ~4 years **Role:** Technical Lead then Project Manager - Export Distribution **Technologies:** PHP, Symfony, Akeneo PIM v2, REST API, XML, CSV, JSON, FTP/SFTP, GitLab CI, Docker, Kubernetes, MySQL ### Key Metrics - Partner Portals: **-** - Migrated, integrated and maintained - Export Formats: **-** - XML, CSV, JSON - Project Duration: **-** - Continuous evolution - GitLab Branches: **-** - Features and hotfixes documented - Daily Volume: **-** - Listings processed across all portals - Availability: **-** - Over 4 years of continuous operation ## Presentation _Project definition and scope_ ### Nature Automated ETL pipeline (Extract-Transform-Load) for multi-channel real estate ad distribution ### Domain Real Estate / PropTech - B2B (internal teams, partner portals) and B2C (indirect, end buyers) ### Scope - Automated data extraction from PIM Akeneo v2 REST API - Per-partner format transformation (XML, CSV, JSON) - FTP/SFTP automated delivery to 25+ partner platforms - Image management with multi-format adaptation (4/3, 16/9, panoramic, square, portal-specific ratios) - Property typology mapping (apartment, house, duplex, triplex, studio, T1-T5+) - Execution monitoring with email alerts and SOFT Monitor integration - Individual partner activation/deactivation capability - SKU matching algorithm for real vs. manually-created PIM programs ### Technology Choices & Rationale - [object Object] - [object Object] - [object Object] - [object Object] **Content:** The **"Export Ligneurs"** system is the **automated real estate listing distribution engine** of the Groupe Pichet. It extracts program and lot data from the PIM Akeneo, transforms it into the specific format required by each partner (XML, CSV, or JSON), and automatically exports it to real estate distribution platforms. The system serves as the **critical link between the company's product data and its commercial visibility**: every property listing published on major French real estate portals (SeLoger, LeBonCoin, BienIci, LogicImmo...) passes through this pipeline. Any interruption or data inconsistency directly translates into **lost leads and missed sales opportunities**. As the **sole technical owner** of this system, I was responsible for all architecture decisions, development, deployment, monitoring, and incident response - with full accountability for a pipeline feeding an estimated **400K euros/month in lead acquisition**. **Nature Label:** Nature **Domain Label:** Domain **Scope Label:** Functional Scope ## Objectives, Context, Stakes & Risks _Strategic vision and constraints_ ### Obj - Migrate all export feeds from legacy PIM v1.4 to the new PIM v2 Akeneo - Execute migration partner by partner with business validation at each step - Verify data consistency between source PIM and feeds sent to portals - Handle each portal's specificities (image formats, typologies, required fields) - Automate feed supervision (error alerts, execution reports) ### Context The project was initiated during the **knowledge transfer from Andoni L. (Kariba)** in January 2019. The existing system ran on the legacy PIM v1.4 and needed to be fully migrated to PIM v2 Akeneo while maintaining continuous service to all partner portals. The migration had to be performed **portal by portal** - each with its own format specifications, required fields, image constraints, and property typology mappings - making it impossible to execute as a single "big bang" migration. Each partner required individual validation by the business teams before going live. The system was embedded in a larger data ecosystem: upstream data came from Qualiac, G2P, and Oracle ERPs feeding the PIM, while downstream the feeds connected to 75+ lead suppliers generating an estimated **1 lead every 2 seconds** across all portals. ### Stakes The partner portals (SeLoger, LeBonCoin, BienIci...) are the **primary prospect acquisition channels** for the Groupe Pichet, feeding an estimated **400K euros/month lead pipeline**. Any interruption or error in the feeds directly translates into **lost leads and reduced commercial pipeline**. With 25+ partners to migrate individually, the project required sustained attention over multiple years while maintaining zero downtime on active feeds. ### Risk - [object Object] - [object Object] - [object Object] - [object Object] ### Key Architecture Decisions - [object Object] - [object Object] - [object Object] - [object Object] **Objectives:** Objectives **Context:** Context **Stakes:** Stakes **Risks:** Risks **Arch Decision Label:** Decision **Arch Rationale Label:** Rationale ## The Steps - What I Did _Chronological progression of the project_ ## Actors & Interactions _Collaborative ecosystem_ **Content:** As the **sole technical owner**, I coordinated directly with business stakeholders, external vendors, and partner portals. Each migration involved defining acceptance criteria, piloting validation cycles, and making go/no-go decisions for production deployment. This required translating technical constraints into business terms and vice versa. ## Results _Impact for me and for the company_ ### For Me - Full technical ownership of a business-critical system directly impacting revenue generation - primary lead acquisition channel for the company - Autonomous architecture decisions on a high-value scope with full accountability for system reliability and data accuracy - Proven ability to pilot a 4-year technical project with multiple stakeholders: business teams, external vendors, and 25+ partner portals - End-to-end system lifecycle management: architecture design, development, deployment, monitoring, incident response, and continuous evolution - Cross-functional leadership: defined validation processes, coordinated business and technical teams, established operational procedures for partner onboarding ### For Company - 25+ partner portals migrated from PIM v1.4 to PIM v2 Akeneo with zero service interruption - protecting an estimated 400K euros/month lead acquisition pipeline - 2 new partner integrations built from scratch (BienIci, Investimeo), expanding the distribution network by ~8% - ~2,000 listings processed daily across 27+ portals, feeding the primary commercial lead generation channel - 99.5%+ system availability over 4 years of continuous operation, with average incident resolution under 4 hours - Standardized property typology management across all partner feeds, reducing data inconsistency reports from business teams **For Me:** For Me **For Company:** For the Company ## Project Aftermath _What happened after delivery_ **Content:** **Immediate aftermath**: After the initial migration wave (2019), the system entered a **continuous maintenance and evolution phase**. New partners were added as business needs arose, existing feeds were updated to match evolving portal specifications, and anomalies were resolved as they were detected. **Medium term**: The system proved its resilience over 4 years of continuous operation, handling format changes from partners (BienIci specification updates, SeLoger field additions) and internal data model evolutions (new property types, price structure changes). **Long-term perspective**: The export system became a **foundational piece of infrastructure** at Groupe Pichet, directly feeding the commercial pipeline. The architectural choices - individual partner modules, solid error handling, automated monitoring - allowed the system to scale from the initial batch of portals to 27+ without requiring fundamental redesign. The modular architecture I chose made it possible for any developer to add a new partner integration by following the established patterns, without needing deep knowledge of the entire system. ## Critical Reflection _Honest retrospective analysis_ ### Positive - The portal-by-portal migration strategy was the right call - it minimized risk and allowed business validation at each step, with immediate rollback capability - Solid error handling and email alerting caught issues before they impacted business results - average detection time under 30 minutes - The modular architecture (one module per partner) made it easy to add, modify, or deactivate individual feeds without side effects on other partners - Establishing a standardized onboarding process for new partners reduced integration time from weeks to days ### Improvement - A centralized dashboard for feed status monitoring would have reduced the time spent checking individual email alerts - Automated integration tests for each partner format would have caught format regressions earlier - Better documentation of each partner's specific requirements would have accelerated onboarding of new team members ### Redo - I would implement a partner specification registry from day one: a structured database of each portal's format requirements, field mappings, and image constraints. This would have cut onboarding time for new partners by at least 50%. - I would add automated integration tests generating sample exports and validating them against each partner's schema before deployment. The manual verification cycles added 1-2 days per migration. - I would build a centralized monitoring dashboard with real-time feed status instead of relying on email alerts. The context-switching cost of checking individual emails was significant over 4 years. - I would push for an event-driven notification layer on top of the batch ETL, to alert the business team proactively when a feed completes or fails, rather than waiting for periodic manual checks. ### Lesson - In multi-partner systems, there is no "one size fits all" - each integration has unique constraints that must be respected - Long-running projects require a maintenance mindset from day one, not just a "build and ship" approach - Business validation at each migration step is non-negotiable when the system directly impacts revenue generation - As a sole technical owner, the most valuable investment is in system observability - the ability to diagnose issues quickly matters more than preventing every possible failure **Positives:** What Worked Well **Improvements:** What Could Have Been Better **Redo:** If I Had to Redo It Today **Lessons:** Key Lessons Learned