Data science keeps world's biggest trucks rolling

Teck, a major Canadian mining company needed to optimize haul truck operations by predicting failures. Having access to vast amount of IoT data from operating machines, Teck turned to AI and Machine Learning to produce unique actionable insights.


Business Need

A modern mining haul truck is a million-pound IoT device on wheels producing 2GB of raw data per day and this presents a major opportunity to use AI for optimizing performance. The company needed to harness the power of the sensor data to predict costly issues such as electrical failures and suspension degradation before they happened. This would involve analyzing terabytes of raw operational sensor and alerts data, coupled with maintenance, scheduling, and other truck lifecycle records to produce actionable insights.


Teck chose to partner with Pythian because of their proven expertise in data science and for their deep knowledge and experience implementing innovative solutions on Google Cloud Platform (GCP). By following Pythian’s iterative data science framework Teck could make well-educated investment decisions at each step of the project, while carefully managing the uncertainties typically found in data science projects. Collaborating with Teck’s subject matter experts, Pythian’s Enterprise Data Science team assessed opportunities to optimize the operational capacity of the machinery by applying Machine Learning solutions to IoT data flowing from the haul trucks in the field. After one month of assessment, Pythian and Teck jointly selected the top use case based on a risk/reward analysis. Pythian then ingested all historical data into GCP and within two months built a proof of concept (POC) solution for the top use cases. As the POC demonstrated good predictive performance, it was followed by two months of implementing the haul truck failure prediction and maintenance recommendation solution in production on GCP while also fine-tuning the models to be more robust. Pythian embedded the resulting insights into the company’s on-premises systems and made available for end users in their already-familiar tools to help them make timely maintenance decisions. Since the production launch, Pythian has been helping the company maintain the operational state of the predictive system closing the full cycle of AIOps. The use of GCP scaled as the project evolved, while Teck’s engineering team continuously acquired deeper and deeper experience with it. In the initial stages, Pythian made use of such products as Google Cloud Storage, Google BigQuery, Google Datalab, Google Dataprep and Google Compute Engine. For machine learning modeling, Pythian used well-recognized open tools such as Scikit Learn and TensorFlow. Google Cloud Bigtable has been used as a wide columnar sparse data store for thousands of sparse pre-aggregated features that allowed Pythian to iterate rapidly through large-scale feature engineering as part of modeling iterations. During production implementation Pythian deployed a custom API layer on Google Apps Engine, implementing unique business logic around machine learning models and exposing a business-specific predictive API via Google Cloud Endpoints for secure and authenticated consumption of resulted predictive microservices. Pythian also leveraged Apache NiFi to handle data ingestion from the company’s on-premise systems and Apache Airflow as job orchestration layer. (The latter is now available as a fully managed service in Google Cloud Platform - Google Cloud Composer.) Thanks to Google Stackdriver and appropriate instrumentation of each component, the state of all components of the predictive system are easily monitored. The collaboration between Pythian and Google has positioned Teck to unlock further value from its unique data and to continue to leverage advanced data analysis and AI capabilities of GCP. These new capabilities are turning Teck into a leading data-driven organization. The solution built by Pythian on GCP is easily expandable - beyond working with end users and enhancements to existing models, the next steps anticipated are growing the number of use cases in optimizing industrial assets maintenance and other optimization of mining operations as well as expanding into other business domains at the company. Notable aspects of the project include:
  • Available maintenance events were carefully analyzed to produce expanded labeled training data on time series data.
  • Pythian performed feature engineering from time-series data with a focus on dimensionality reduction. Given the low number of training samples and a high degree of data complexity, standard high-powered time-series approaches (RNNs, variant CNNs, etc.) were not feasible.
  • Pythian designed features and selected models based on low-complexity approximations of high complexity models that preserved relevant time-series information and avoided overfitting.
  • Pythian combined generic time-series data aggregations to handle a large variety of sensors and alerts at scale with domain-specific feature engineering.


Pythian used IoT data from truck sensors and other sources to produce a predictive solution that helps minimize costly downtime by predicting failures and recommending maintenance. The resulting solution was a machine learning application built on GCP with integrated IoT data ingestion and predictive microservices embedded into Teck’s on-premises products via REST APIs. The ability to anticipate events such as catastrophic electrical failures and suspension performance degradation allows the company to optimize maintenance schedules and reduce trucks downtime.


Google Cloud Platform, Google Cloud Storage, Google BigQuery, Google Datalab, Google Dataprep, Google Compute Engine, Google Cloud Endpoints, Google Apps Engine, Scikit Learn, TensorFlow, Apache NiFi, Apache Airflow