To improve efficiencies and reduce costs, the online content network needed to migrate their on-premises Teradata data warehouse to the public cloud. They also needed to improve their analytics capabilities to provide a 360-degree view of customer experience. They chose Google Cloud Platform (GCP) as the basis for their new data hub because of the significant cost savings associated with not managing on-premises hardware and software, while it also enabled them to stay current with the latest technology without requiring in-house time and expertise. For their analytics engine, they chose Google BigQuery. The challenge was to determine how to prepare and move multi-source customer data from their Teradata data warehouse to the Google BigQuery data analytics warehouse, while also integrating all their data sources in the new data hub. They needed a partner with expertise in Google Cloud Platform, migrations from legacy systems, data analytics, and all the tools needed to build a pipeline for the data transfer. The partner also needed to be able to both design and implement a customized analytics solution.

With unrivaled experience in migration planning and implementation, data analytics, and all the tools and required expertise, Pythian was uniquely qualified to act as advisor and resource hub. Pythian Analytics Services were chosen to develop a cloud-based analytics solution. The Pythian team designed and implemented a data pipeline that used Apache NiFi to integrate data from approximately 60 sources, including: • MySQL database • Oracle database • A number of APIs • Omniture DoubleClick for Publishers marketing and web analytics • Pardot B2B Marketing Analytics • Tableau reports • Neilsen ratings and viewing data The data was exported from the Teradata data warehouse to Apache Hadoop Distributed File System orchestrated by Google Cloud Dataproc, then transferred to Google Cloud Storage and loaded into Google BigQuery. Pythian also created a data dictionary using Apache Avro to define the company’s various data types.  

Pythian’s experience with migrations and analytics solutions, plus technical expertise in the selected technologies, resulted in a flexible and cost-efficient data analytics solution. In Google BigQuery the cost of storage and compute are kept separate, and on-demand pricing enables the customer to pay for only the storage and compute they use. Query response time is also better with Google BigQuery than Teradata, and BigQuery can adapt to any data type or format, plus convert formats, without additional charges. The online content network now has a more effective analytics solution with higher performance, increased flexibility and lower costs. They also have a better view of the end-to-end customer experience, an irreplaceable competitive advantage.