The DeepRain project is a collaboration among the Jülich Supercomputing Centre (JSC) at Forschungszentrum Jülich, the German Weather Service (DWD), the Universities of Osnabrück and Bonn and the Jacobs University in Bremen. The partners investigate how modern methods of machine learning can be applied to improve precipitation forecasts in Germany. Precise predictions of rain and snow with a reliable indication of the expected amount of precipitation are still an extreme challenge for weather modelling, especially at the local scales where they are most relevant.
The project makes use of radar data, high-resolution topographic data, and ensemble forecast data from a numerical weather model and employs state-of-the-art information processing technology as well as advanced statistics for the evaluation of the new forecasts. DeepRain is funded by the German ministry of education and research (BMBF) under grant number 01 IS 18047 and runs from Oct 1st, 2018 to Sept 30th, 2021.
Goals
Approach
DeepRain explores new ways in the four areas: Data fusion, application of AI, validation methods, and big data IT technology.
Spatially resolved (grid point) data from different data sources are imported into the array database rasdaman, where they are then available for flexible high-performance extraction. The project will lead to enhanced query options for the rasdaman database.
Modern deep learning technologies and neural network architectures are adapted to the specific requirements of meteorological data sets. In contrast to typical image recognition or segmentation problems, DeepRain faces the challenge of limited input data (large fields, but small number of samples). Complex and time-evolving model errors make it a challenging task to select appropriate data structures and neural network architectures. Furthermore, precipitation is far from following a normal distribution, and heavy rainfall events are rare cases.
Advanced statistical techniques from climate model validation and postprocessing will be adapted to short-range weather forecast results on high resolution. Careful and robust evaluation of the deep learning results shall build confidence in this novel technique, or – if the results are not convincing – establish critical benchmarks against which future attempts to supersede classical downscaling methods have to be measured.
The number and size of the meteorological fields (about 600 TBytes total) and the complexity of the problem call for sophisticated data and work flows and provide the opportunity to test new data architectures and data staging concepts on the Jülich supercomputers JURECA and JUWELS. The DeepRain project will thus contribute to the design choices of future exascale supercomputing architectures.
Workflow
The graphics shows the workflow as employed by DeepRain. The various input data are first imported into high-performance database systems and processed there so that they are available as training, test and evaluation data for the neural network. The topographic data is static, all other data sets contain temporal changes up to the recent past. Predictions are also available from the weather model, which the neural network can use for downscaling. With the help of machine learning, a spatial mask is first generated for each prediction time step in order to identify those points where precipitation is likely to occur for further analysis. Then the predictions are divided into three classes for strong, medium and weak precipitation and for each grid point the probability for the occurrence of precipitation of the respective classes is calculated, whereby the coupling between adjacent grid boxes must be taken into account. The prediction results are made available via a web interface where they can be evaluated with independent data from station measurements.
DeepRain work packages
| WP1: Coordination and project management |  |
||
| WP2: Data processing and provision |  |
|
 |
| WP3: Method development for deep learning |  |
|
|
| WP4: Data consistency and classical downscaling algorithms | |
 |
|
| WP5: Evaluation and comparison of results | |
|
|
| WP6: System design and workflow analysis | |
|