Model Evaluation on Gadi¶
Model evaluation is about measuring how fit for purpose a particular model is. Model evaluation in climate science is the process of assessing the performance and reliability of computational models that simulate the Earth's climate system. It involves comparing model predictions to observed data to determine the model's accuracy and usefulness. In doing so, we can understand how well a model represents real-world climate processes and make predictions about future climate trends. Such rigorous model evaluation allows scientists to identify model strengths, weaknesses and uncertainties, as well as refine models to enhance their predictive capabilities.
Data workflows¶
FAIR (Findable, Accessible, Interoperable and Reusable) data is required for model evaluation. Some of these evaluation workflows use data tools and catalogues outlined in the ACCESS data section. ACCESS-MOPPy can also be integrated with these evaluation frameworks on Gadi. Observations have been collated for model evaluation so that they can be ingested directly by the frameworks on Gadi.
Use on Gadi¶
Model evaluation is tipically carried out using Python. Therefore, scientists need suitable Python environments for their workflows, which need to be managed and maintained with the required dependencies for open source and developing software.
ACCESS-NRI supports the conda/analysis3 environment in the xp65 NCI project which includes commonly used Python libraries and have these evaluation frameworks already installed.
Evaluation Frameworks¶
ACCESS-NRI currently provides support for the following model evaluation frameworks on Gadi:
Evaluation recipes¶
These recipes provide workflows to reproduce diagnostics calculations and analysis visualisations of model outputs. This allows scientists to construct their own evaluation workflows by building upon existing community workflows.
They can use any Python library, including leveraging evaluation frameworks APIs. Diagnostic calculations can be ported between using common or scientific libraries and evaluation frameworks depending on a user's preference and familiarity with them. An advantage of using standard frameworks includes possibility to scale the analyses by running them in on multiple different datasets. This is a principle for the development of the Rapid Evaluation Framework(REF) for CMIP7.
Contributing to recipes and diagnostics¶
All evaluation recipes on community papers are open for contribution. To contribute to a specific evaluation framework or recipe, follow the contribution guidelines in its GitHub repository.
General steps include:
- Opening an issue in the relevant repository to discuss your idea
- Submitting a Pull Request to add your recipe, documentation, and links
Support¶
To get further support on Model Evaluation on Gadi, refer to User support and reach out on ACCESS-Hive Forum
