Why Choose Talend Solutions?

 

Next Steps

Tell us about your requirements:

+44 (0) 1494 616134

sales@talendsolutions.com

BeMyEye and Paddy Power Betfair sign new commitment - BeMyEye

 

Talend Solutions was engaged to provide expert assistance to:

  • kick-start Talend adoption;
  • provide governance and best practices;
  • architect, design and implement Talend-based solutions correctly first-time;
  • train, mentor and support the near-shore development team.

 

Talend Solutions led the coaching and mentoring of Paddy Power Betfair’s large near-shore team of data engineers, with key initial tasks including: installing and configuring software; specifying standards and best practices; designing, specifying and developing job frameworks and patterns.

 

The first project on the platform was to build a consolidated Data Warehouse, merging legacy Paddy Power (SQL Server) and Betfair (Oracle) data warehouses to a cloud-based standardized dimension model (star schema) for the joint business.  This was architected on an Amazon Web Services stack (EC2 servers, S3 data lake, Redshift databases).  The design involved developing Talend job patterns to:

  • Load (migrate) historical data.
  • Implement business as usual (BAU) processing of change data: including ingestion, standardization and build of the dimensions and fact tables multiple times during a given day.
  • Two copies of the star schema were required: one in the ETL Redshift cluster as a working copy for the ETL processes and one in the Query Redshift cluster for exposure to the end user community.
  • A data lake in S3 was established out of the process, hydrated with a daily full table file and a daily cumulative file for each table in each source.

 

The nature of the requirement demanded very similar jobs to be built for every entity to lift and land the data.  Traditional approaches would have developed numerous specific jobs.  Talend Solutions architected, designed and built a framework and pattern-based meta data driven approach that allowed us to develop common generic jobs which were driven by external meta data and used Talend's powerful dynamic row capability.  This approach radically reduced the number of jobs required to be developed, allowed development timelines to be dramatically shortened and supported rapid on-boarding of new data sets.  The approach also rationalised testing procedures and eased support and maintenance challenges.  Framework, job patterns and initial job implementations were handed over to data engineers, who we mentored and supported to enable project delivery.

 

Talend Solutions also:

  • Defined and produced development procedures and continuous integration processes (using Talend, Nexus, Jenkins, Maven and Gitlab) for Paddy Power Betfair.
  • Integrated Talend job execution with Airflow (an open source scheduling tool) allowing dependencies with external processes to be managed.

 

The project subsequently went on to become a Talend Data Masters Award Winner (https://www.talend.com/customers/paddy-power-betfair/).

 

The near-shore team are now a proficient and productive Talend development centre.