Ctrl+K

Transfer scientific algorithms into technical application packages for CRIS

Transfer scientific algorithms into technical application packages for CRIS#

Example CRAI (Climate Reconstruction AI)#

CRAI is a state-of-the-art deep learning based inpainting technology to infill missing values in climate datasets.

The method#

The infilling algorithm is trained with complete climate datasets and masks defining realistic regions of missing observational data.

../_images/pconv-unet.png

As shown in the figure, the neural network has a U-net architecture and makes use of partial convolutional layers instead of standard convolution layers. A partial convolutional layer is a combination of two successive operations:

  • a partial convolution: \(x'=\left\{\begin{array}{ll}\boldsymbol{W}^T\left(\boldsymbol{X}\odot\boldsymbol{M}\right) \frac{\text{sum}(\boldsymbol{1})}{\text{sum}(\boldsymbol{M})} +b, & \text{if sum}(\boldsymbol{M})>0 \\ 0, &\text{otherwise}\end{array}\right.\)

  • a mask update: \(m'=\left\{\begin{array}{ll}1, & \text{if sum}(\boldsymbol{M})>0 \\ 0, &\text{otherwise}\end{array}\right.\)

where \(\boldsymbol{W}\) are the convolution filter weights, \(\boldsymbol{X}\) the feature values, \(\boldsymbol{M}\) the binary mask values and \(b\) the bias.

Compared to other inpainting methods [1], the partial convolution technique offers several advantages:

  • it operates robustly on irregular holes

  • it can infill large holes

  • it produces semantically meaningful predictions

Installation#

CRAI is written in Python and based on the PyTorch framework and the NVIDIA CUDA® Deep Neural Network library (cuDNN). It is licensed under the terms of the BSD 3-Clause.

Step 1: clone the branch of the repository:

git clone -b clint --single-branch https://github.com/FREVA-CLINT/climatereconstructionAI.git

Step 2: create en environment with all the required dependencies using Anaconda

conda env create -f environment.yml

Step3: activate the environment

conda activate crai

Step4: install the Python package using pip

pip install .

Usage#

The software can be used to:

  • train a model (training)

  • infill climate datasets using a trained model (evaluation)

It can be used as:

  • a command line interface (CLI):

    • training:

    crai-train

    • evaluation:

    crai-evaluate

  • a Python library:

    • training:

    from climatereconstructionai import train
train()

    • evaluation:

    from climatereconstructionai import evaluate
evaluate()

For more details about the execution and the input arguments, please have a look to the documentation and demo examples hosted in the repository

Contents