An overview of how SHAP feature contributions are calculated — Suppose you (Player 1) and a friend (Player 2) enter a Kaggle contest. You end up winning the first prize of $10,000. Now, you want to split the money fairly. Your friend suggests that you just split it equally. However, your hyperparameter tuning skills are superior. You believe you deserve…

The intuition, maths and code (R and Python) behind partial dependence plots and individual conditional expectation plots — Both PDPs and ICE plots can help us understand how our models make predictions. Using PDPs we can visualise the relationship between model features and the target variable. They can tell us if a relationship is linear, non-linear or if there is no relationship. Similarly, ICE Plots can be used…

Doing an exploratory fairness analysis and measuring fairness using equal opportunity, equalized odds and disparate impact — It is no longer enough to build models that make accurate predictions. We also need to make sure that those predictions are fair. Doing so will reduce the harm of biased predictions. As a result, you will go a long way in building trust in your AI systems. …

My process, tools and tips for writing data science articles — I find the blank page of every new article to be a bit daunting. There is so much work to be done. You need to do research, draft, edit and add graphics. In some cases, you will need to learn something completely new. …

How to incorporate the scientific process in your data science analysis — Business science, sports science, food science… we love throwing the word science behind other words. It makes those fields sound, well, more scientific. Yet, if we are not careful they will have more in common with pseudoscience. Data science is no different. On the surface, it may seem close to…

Using DeepShap to understand and improve the model powering an autonomous car — Autonomous cars terrify me. Big hunks of metal flying around with no humans to stop them if something goes wrong. To reduce this risk it is not enough to evaluate the models powering these beasts. We also need to understand how they are making predictions. …

What I learned about machine learning in production, transfer learning and a PhD in general — I started my PhD with the enthusiasm that only a first-year student could have. I was ready to push the boundaries of science! Calm down, Feynman. Before I could jump into my project, we had a 6 week-long bootcamp. This was filled with workshops, industry talks, social events and, importantly…

Cropping, grayscale, RGB channels, intensity thresholds, edge detection and colour filters — With feature engineering, we immediately think about tabular data. Yet, we can also get features for image data. The goal is to extract the most important aspects of the image. Doing so will make it easier to find a mapping between our data and the target variable. This means you…

Creating a tool that selects the RGB channels from image pixels — I know, I know… there are loads of colour-picking tools out there. Yet, I think you will find advantages to using one that works directly in a notebook. This tool is also fully customisable. We will build two colour pickers: Simple picker — selects a colour from a single image …