Why my first job in data science was not what I expected — New shirt, new shoes. I was ready for my first job in one of Ireland’s biggest banks. I was excited. Looking back, I had good reason to be. I was able to work on impactful projects and I learned an immense amount. In fact, the biggest lesson was: Data science…

How to incorporate the scientific process in your data science analysis — Business science, sports science, food science… we love throwing the word science behind other words. It makes those fields sound, well, more scientific. Yet, if we are not careful they will have more in common with pseudoscience. Data science is no different. On the surface, it may seem close to…

The intuition, maths and code (R and Python) behind partial dependence plots and individual conditional expectation plots — Both PDPs and ICE plots can help us understand how our models make predictions. Using PDPs we can visualise the relationship between model features and the target variable. They can tell us if a relationship is linear, non-linear or if there is no relationship. Similarly, ICE Plots can be used…

Doing an exploratory fairness analysis and measuring fairness using equal opportunity, equalized odds and disparate impact — It is no longer enough to build models that make accurate predictions. We also need to make sure that those predictions are fair. Doing so will reduce the harm of biased predictions. As a result, you will go a long way in building trust in your AI systems. To correct…

Mapping pixel positions to geographic coordinates using Landsat metadata and Rasterio — Location, location, location! Not just a housing market cliché but also incredibly important to remote sensing. Whether it’s for monitoring environmental changes, analyzing urban development, or tracking crop health, precise geolocation is crucial. …

Streamline Landsat scene downloads with the landsatxplore Python package — Landsat satellites are among the most commonly used sources of Earth observation data. They have been providing high-quality images of the planet’s surface for over four decades. However, manually downloading these images can be tedious! Fortunately, with the landsatxplore package, you can easily download and process Landsat scenes with a…

Improve your technical and communication skills, learn to write readable code and start writing about anything — When I first started writing, I could hardly code. My little Python experience came from university projects. At the same time, I thought I was a great writer! Now looking back, I cringe at some of my earlier articles. This shows me that writing tutorials has improved both my technical…

How to deal with multiple spectral bands, large pixel values and skewed RGB channels when visualising satellite images — Satellite images contain an abundance of information. The downside is that it is not straightforward to visualise them. The data differs from normal images as they can have: Over 12 channels Large pixel values Skewed pixel values We will discuss these key considerations. We then factor them into a Python…

How to effectively run and debug code in a resource-constrained environment — Queue job, wait 24 hours, cuda runtime error: out of memory Queue job, wait 24 hours, FileNotFoundError: No such file or directory Queue job, wait 24 hours, RuntimeError: stack expects each tensor… AHHHGH!!! Debugging code on a high-performance computing (HPC) cluster can be incredibly frustrating. To make matters worse, at…