Why my first job in data science was not what I expected — New shirt, new shoes. I was ready for my first job in one of Ireland’s biggest banks. I was excited. Looking back, I had good reason to be. I was able to work on impactful projects and I learned an immense amount. In fact, the biggest lesson was: Data science…

How to incorporate the scientific process in your data science analysis — Business science, sports science, food science… we love throwing the word science behind other words. It makes those fields sound, well, more scientific. Yet, if we are not careful they will have more in common with pseudoscience. Data science is no different. On the surface, it may seem close to…

The intuition, maths and code (R and Python) behind partial dependence plots and individual conditional expectation plots — Both PDPs and ICE plots can help us understand how our models make predictions. Using PDPs we can visualise the relationship between model features and the target variable. They can tell us if a relationship is linear, non-linear or if there is no relationship. Similarly, ICE Plots can be used…

Doing an exploratory fairness analysis and measuring fairness using equal opportunity, equalized odds and disparate impact — It is no longer enough to build models that make accurate predictions. We also need to make sure that those predictions are fair. Doing so will reduce the harm of biased predictions. As a result, you will go a long way in building trust in your AI systems. …

A guide to the code and interpreting SHAP plots when your model predicts a categorical target variable — SHAP values give the contribution of a model feature to a prediction. This is also true when we use SHAP for classification. Except, for binary target variables, we interpret these values in terms of log odds. For multiclass targets, we use softmax. We will: Discuss these interpretations in more depth

What the plots in SHAP version 0.42.1 can tell you about your model — One of the biggest concerns about SHAP has to do with the package itself. It hadn’t been updated in a while and the GitHub issues were piling up. To the relief of many users, contributors have been more active. In fact, they’ve given us new charts — Violin and Heatmap…

Using Python to rotate satellite images while maintaining geolocation accuracy — Raster data is similar to normal image data. Except every pixel is linked to a location on the Earth’s surface. This complicates things. If we want to rotate the data we must also consider the underlying coordinate reference system (CRS). Distorting a raster without adjusting its geolocation will lead to…

The dangers of gender recognition software, inadequate medical models, and the amplification of transphobic content — Discussions around the risks of AI often gravitate towards the hypothetical dangers of artificial general intelligence (AGI) and doomsday scenarios. Robots are not going to take over the world. Yet, the current level of AI does pose tangible risks. Particularly to the trans and gender non-conforming community who have already…

Calculating cloud cover in your area of interest, removing clouds and inpainting them using another satellite image — Our technology has progressed to conquer the vastness of space. We’ve launched satellites, armed with state-of-the-art sensors, to monitor our changing planets. Yet, one adversary has humbled these advanced systems — clouds. Whether you want to compare the value of spectral indices or train machine learning models on satellite images…