PCRaster and Python version 2 have been loyal friends for a long time, but even good things won’t last forever. As the official support for Python 2 ends in less than three months, we no longer make efforts in building new versions of PCRaster against Python 2. The upcoming PCRaster 4.3 release (expected later this year) will exclusively support Python 3 on all supported platforms. Modellers really should migrate their model code to Python 3 in case they did not until now.
We will be in Vienna next week, our contributions are:
The LUE scientific data base for storing heterogeneous earth science data
Tue, 09 Apr, 14:00–15:45 Hall X1
Sustainable scientific software: experiences of the PCRaster research and development team
Wed, 10 Apr, 08:30–10:15 Hall X1
We are glad to announce the release of PCRaster 4.2.1. We fixed a memory leak in
pcr2numpy, thus long lasting model runs can be fired up again!
We are glad to announce the final release of PCRaster 4.2.0! We added support for Python 3, the new ‘multicore’ module providing multi-threaded local and focal operations, fixed several software issues and merged all our manuals into one documentation. For more information please visit the download page. Enjoy!
We received some user feedback since we shipped the PCRaster 4.2rc1, amongst others reporting one relevant issue in the Python module. The conversion of PCRaster non-spatial values to NumPy was defective in previous releases, this should be resolved now.
Please check out the PCRaster 4.2rc2 for Windows. In case no further issues are found the final version of PCRaster 4.2 will be released next week.
We are finalising the upcoming PCRaster 4.2 release. Briefly, the main new features are the new multicore module for PCRaster Python, support for Python 3, and enabling large file support on Windows systems.
PCRaster support for Python 3
The lifetime of Python 2 is coming to an end, Python 3 is now widely used and even conservative Linux distributions like RedHat will switch to Python 3 as default version in their upcoming major release. We now support Python 2 or 3 on Linux distributions. Python version 3.6 is now our supported version by default on Windows.
The PCRaster multicore module
The new PCRaster Python multicore module contains alternative implementations of a subset of the PCRaster operations that are capable of distributing their workload across more than one CPU core, therefore improving the runtime performance of those operations. Modifications to existing model scripts are not required. The multicore module can be enabled just by setting an environment variable.
Support for files larger than 2 GB on Windows systems
With previous PCRaster versions it was on Windows systems impossible to generate raster maps larger than 2 GB, this issue is resolved now. We also updated the PCRaster driver in GDAL, version 2.2.4 or later is required to use the new functionality.
Further improvements and Windows package
We unified the documentation of all our projects. We also spent significant efforts in improving our code base as well as the build system for different operating systems. Supporting several platforms should be more straightforward now.
Please check out the release candidate for Windows 64-bit. In case no major bugs are found, we plan to release the final version of PCRaster 4.2 within a few weeks.
Build your own PCRaster
We already mentioned earlier that we will no longer distribute compiled versions of PCRaster for Linux systems. Users will need to build their own package, but we strive to make that process simple. Debian testing users, for example, can try this:
$ sudo apt install cmake gcc g++ git libboost-all-dev libgdal-dev libncurses5-dev libpython-dev libqwt-qt5-dev libxerces-c-dev libxml2 libxml2-utils libxslt1-dev python-numpy qtbase5-dev xsdcxx python-docopt
$ mkdir pcraster42rc1 && cd pcraster42rc1
$ git clone --recursive https://github.com/pcraster/pcraster.git
$ mkdir build
$ cd build && cmake -DFERN_BUILD_ALGORITHM:BOOL=TRUE ../pcraster && cmake --build .
We will provide build instructions for other Linux distributions in another post.
Last week I attended the FOSS4G conference in Bonn. There were plenty of interesting presentations, and the venue (World Conference Center Bonn) was really impressive. If you are interested in our current work on estimation of human exposure to air pollution you can download our presentation slides and watch my presentation (ca. 25 min; mp4, webm). You can also contact us directly for more information.
We released the PCRaster version 4.1 already some time ago. We’ve been busy the last year with several projects, and we are finally getting closer to the next release. A lot still needs to be done in the next few weeks, but it is time for a short update. While we will continue to distribute binary packages for Windows systems, Linux users will face some changes:
Dropping LSB support
Until now, we provided Linux versions of the PCRaster software conforming to the Linux Standard Base (LSB). This approach is convenient for users as our executables will run on virtually any Linux distribution. For us developers, however, building LSB software is quite tedious. We were forced to use specific (outdated) compiler versions and we needed to compile and distribute a lot (about 25) of third party libraries ourselves. In addition, the LSB is rather conservative with respect to supporting new compiler versions, therefore not enabling us to use new and very useful features of the C++ programming language. However, we need those new C++ features for our current projects.
As PCRaster is open-source for quite some time now there is also no reason to distribute binaries any longer. We will instead focus on improving and extending our software and on distributing the code. We changed our whole build infrastructure to use more libraries installed on the system, and further try to reduce the dependencies to third party libraries making it easier for us to develop and add new features. However, no longer providing binaries will require the Linux users to compile the PCRaster software themselves. Don’t fear; we will provide build scripts that smooth the compilation process.
Distributing experimental features
We develop new software due to several research and development projects. Last summer we worked on a multicore module for the PCRaster point operations within the maps4society project. We continue our work on high performance computing elaborating distributed computing to further increase runtime performance of environmental models and parallel I/O. We are also busy with a new file format that is capable of handling heterogeneous data.
These exciting projects result in new, volatile software rather than stable versions. But we are convinced that our developments are of interest and useful for adventurous modellers. However, previously it was very difficult to distribute those new software developments to our users. The new release will include an optional flag to build these experimental features such that modellers can benefit from our latest projects more easily.
Supporting new platforms
Reducing dependencies to external libraries should result in a straightforward and easier build process for Linux distributions. The PCRaster software will therefore continue to run on all kinds of Linux machines (and yours). We would also like to add the PCRaster software to the OSGeo live CD to streamline the first time experience of new users. In the very near future we also will be able to support other UNIX based systems (the attentive reader might have noticed the operating system used for the screenshot). In a later stage we consider distributing our software via package managers such as apt or macports.
During the summer we have been busy with tailoring PCRaster towards high-performance computing, and we already have rewritten parts of the PCRaster algorithms due to our Maps4Society project. This refactoring required significant efforts and code changes. Sometimes, however, appending just a single word can yield performance improvements as well… as our colleague Edwin Sutanudjaja mentioned, a simple NOPRINT statement in several packages prevents Modflow to write a large amount of redundant information. For a model running the extent shown above (see also here; 21600000 cells, with 3920887 drains and 21600000 river cells) the Modflow runtime reduces from about
Nice savings, especially if you need to run several timesteps! The improvements will be included in the upcoming PCRaster release.
In the context of the Healthy Urban Living programme we implemented a set of Land Use Regression models to determine the spatial distribution of several air pollution concentrations (e.g. NO2, NOx). These models cover the entire Netherlands at 5m resolution (the figure below shows the PM10 concentration).
We combine these field-based concentration maps with human activity patterns, for instance to calculate the total exposure of individuals during their home-work travel. For more information join our presentation in the Air Pollution Modelling session next month at the EGU or contact us at firstname.lastname@example.org.