Code for Earth: Evaluating air quality
Driven by our commitments to sustainability and open source, we won the opportunity to work on a project run by the European Centre for Medium-Range Weather Forecasts (ECMWF) to create vAirify, a tool to evaluate Air Quality Index forecasts across the globe.
This initiative aligns with our social mission as a company and a B Corporation, as well as our ambitious efforts to safeguard the environment. Atmospheric pollutants can have a serious effect on people’s health and have been linked to conditions including asthma, heart disease and cancer.
The tool we’ve delivered – vAirify – will help enhance air quality forecasting and will also inform policy decisions to help drive improvements in air quality by reducing pollution. In these ways, it will make a contribution to improving public health.
The Code for Earth challenge
ECMWF established Code for Earth in 2018 to drive innovation and open source developments in the Earth sciences community. The programme sets challenges related to the broad scope of activities at ECMWF. Project teams then apply for the opportunity to take on one of the challenges and, if successful, work with experienced mentors from ECMWF and partner organisations to deliver the project.
We applied for the challenge to “develop an online dashboard for displaying and monitoring air quality forecasts for cities around the world” because it particularly played to our strengths in data collection, evaluation and visualisation. Scott Logic’s roots are in financial services, helping institutions harness and visualise vast amounts of data to derive actionable insights.
The Code for Earth competition offered the perfect opportunity to transfer our expertise from that realm into uncharted territory, delivering a tool that would help make a difference to people’s lives.
The ECMWF’s objective was for the dashboard to compare air quality forecasts with near-real-time, in-situ measurements of air quality for cities across the world, and to provide alerts of any potential issues. The forecast data would be drawn from the Copernicus Atmosphere Monitoring Service (CAMS), which provides forecasts to the public through smartphone apps and news services. The in-situ measurements would be drawn from OpenAQ, a non-profit, non-governmental organisation.
It was a pleasure to work with such a professional team. From the technical side of the project, I've improved my knowledge about lean project management techniques, efficient requirement collection steps, modern application deployment procedures, the importance of a comprehensive testing protocol and selecting a sensible level of documentation.
Miha Razinger, Analyst Consultant at ECMWF
Partnering to deliver tech for good
From the beginning, collaboration was at the heart of the project. The Scott Logic team that formed to work on the challenge submission welcomed a new member shortly after the competing projects were announced; Sebastian Steinig, a climate scientist from the University of Bristol, asked to come on board. Steinig brought the specialist expertise in climate science that our team lacked, helping accelerate their learning curve; in return, he benefited from our expertise in agile software engineering by joining the daily stand-ups and the weekly reviews and retrospectives with stakeholders.
With support from the mentors during a rapid discovery, the team found out how to access, process and interpret the forecast and measurement data. The focus then shifted to building the dashboard incrementally in order to present working software to the mentors at the earliest opportunity, allowing for rapid iterations based on their feedback. Following six weeks of data engineering and user interface design, the team was able to release a publicly accessible version of the dashboard on a virtual machine, allowing specialists to put it through its paces and provide expert feedback.
Validation in near-real-time
Connected to live forecast and measurement data from around the world, it wasn’t long before the accuracy and utility of the dashboard was validated by events. On the morning after the 4th of July, the team noticed peaks in PM2.5 pollution (Particulate Matter around 2.5 micrometres in size) for cities in the USA. Their deduction that this was due to smoke from Independence Day firework displays was borne out by academic research. The peaks didn’t appear in forecasts as the locations of the emissions were not known in advance.
A week later, one of the mentors reported that vAirify was providing helpful insights about Dakar, where Saharan dust was blowing out towards the Atlantic Ocean. A sun photometer was showing that ECMWF was underestimating the Aerosol Optical Depth and vAirify indicated that the PM10 count (Particulate Matter around 10 micrometres in size) was overestimated, which helped to evaluate the forecast performance.
The team’s test strategy provided further insights on how well vAirify was working by detecting some issues in the data that weren’t immediately obvious. For example, it revealed that units of the data being drawn in from some air quality sensors was in parts per million and from other sensors in micrograms per cubic metre. Having identified this, the team programmed in the necessary equation to transform and standardise the data.
Delivering early and ongoing value
From the moment the vAirify dashboard was made publicly accessible, it was used routinely, helping evaluate air quality forecasts. Ten weeks after commencing the project, the mentors confirmed that our team had already met the project goal. In the ensuing two months before the team presented the project at the ECMWF headquarters in Reading, they could focus on exceeding expectations by refining vAirify and making it easy to maintain, with full documentation on GitHub.
With support from the open source community, ECMWF and CAMS will take forward the development of vAirify. The tool will positively impact the work of the organisation in two principal ways.
First, it will provide the capacity to evaluate air quality forecasts in near-real-time against open source air quality measurements, so that it can work with governments and organisations around the world to make improvements. Second, vAirify will help to demonstrate the quality and usability of the open source air quality measurements, which has previously been less straightforward to do with the data available.
We at Scott Logic feel privileged to have had the opportunity to contribute to Code for Earth and, through vAirify, to play a small part in improving health outcomes for people around the world.
The development of the vAirify application is an important step in the routine evaluation of the forecasts that we provide in the Copernicus Atmosphere Monitoring Service. Our data is widely used around the world and vAirify provides us with a tool for better understanding the forecast performance, and in giving confidence to our communication activities on global air quality.
Dr Mark Parrington, Senior Scientist, CAMS Research Department, ECMWF