GS2 v8.1

GS2 v8.1 is our first feature release using the new git + Bitbucket workflow, and it's a big one! It's over 280 pull requests - and over 2000 commits - ahead of the current bugfix release 8.0.6.

v8.1 offers many new features, enhancements and performance improvements, along with significant expansion of testing and documentation, and lots of bug fixes!

This release enables larger, faster, more robust simulations, without changing the familiar interface for users or developers. But since this is a feature release, we have allowed some changes that might require user changes to input or analysis scripts, or that may alter results slightly - so do check out of the breaking changes section below.

Features and Enhancements

• Smart defaults - it's now possible to change the default value of a variable, based on the input from another variable. For example, we now set write_fluxes and write_fluxes_by_mode to true for nonlinear runs (nonlin=true) and to false for linear runs. This should hopefully lead to more intuitive behaviour. If you have suggestions for extra smart defaults then please report this in an issue.
• New simpler option for hyperviscosity, simple3D, which has the form D_hyper3D * (|kperp|/ max |kperp|)^P_hyper3D that respects continuity along the field line, and does not require MPI communication.
• Users can now specify both spatial grid resolution nx, ny and wavenumber resolution nakx, naky (provided the anti-aliasing constraint is met), making it easier to select sweetspots (... but can still define only one of these and let GS2 choose the other).
• Users can now specify the number of passing pitch angles by specifying npassing in the le_grids_knobs namelist.
• The explicit source terms are now stored in the output. This is the default, but can be turned off by setting include_explicit_source_in_restart in the init_g_knobs namelist to false.
• Users can now specify a random seed for initialization by setting seed in the knobs namelist. This makes it easier to exactly reproduce runs.
• Users can now control the netcdf sync frequency (the frequency at which updates are written to disk) by setting nc_sync_freq in gs2_diagnostics_knobs.
• GS2 now saves the layout string in output and checks for consistency in restarted runs.
• The new "run configuration" type allows programmatic control of code inputs, allowing better documentation and testing.
• The user can query the built GS2 executable for the version, build configuration and usage instructions. Try ./gs2 --help for more details.
• Netcdf output files store some useful metadata such as GS2 build version, time and date of run, unique run id. These are stored as global attributes, so will appear at the bottom when doing ncdump.
• Fields local initialisation significantly improved to avoid issues with communicator fragmentation which prevented the use of fields local on large problems. We'd encourage you to try field_option = 'local' for nonlinear simulations as the initialisation can be a factor of 20x faster than with field_option = 'implicit'.
• Each run will generate a unique id which is reported to the screen, various ascii outputs and stored in the netcdf outputs. This should allow better data management, making it easier to associate data from the same run etc.
• A new ascii output file, <run_name>.used_inputs.in, is written early in the run. This stores the values of all input configuration parameters used in the run. This allows users to check which defaults have been applied etc.
• The ability to use compression in netcdf files has been introduced. This is not currently applied by default, but can be enabled by passing USE_NETCDF_COMPRESSION=on to make. This can reduce the size of restart files by a factor 2. This has been enabled by a switch to the NETCDF4 file format. Existing analysis tools should continue to work however this generally means one should not access the netcdf files whilst the run is ongoing by default. For example, running ncdump on an in use file is likely to return a HDF error. To avoid this and allow access during a run one can set HDF5_USE_FILE_LOCKING=FALSE in the environment (e.g. export this in bash)*.
• A new species type, 'hybrid_electron', has been introduced and results in an electron type for which the passing particles are adiabatic and the trapped are kinetic. This has been tested but is currently considered experimental.
• More verbose timer output at the end of the run. This might help identify if your run has an excess fraction of MPI, for example, indicating that you may be using too many cores.
• Additional matrix inversion implementations including LAPACK based and block recursion.
• Fully replaced ruby with python, removing ruby as an optional dependency.
• There are many other small tweaks and features for testing and convenience.

Documentation

We've made significant improvements to our documentation:

• We've expanded our user manual and developer manual.
• We've expanded our inline comments and now have documentation of input variables.
• Source code is parsed and put on the documentation website that builds automatically for master and next.

Testing and infrastructure

We've expanded our test suites and improved our building and continuous integration frameworks.

• New expanded GitLab testing framework to replace Jenkins
• New more accurate Coveralls test coverage tracking of pfunit tests to replace Codecov
• Makefile is now tidier and more robust. Git hashes for utilsand Makefiles are now stored during the build process, making it easier to track down bugs.
• We've removed many instances of imported-but-unused variables from the source code, making the build output quieter and easier to read.
• We've added tests and benchmarks, including for
  • indexing routines
  • ceq and leq geometries
  • theta grid
  • zonal flow residual linear test
  • box mode linear itg test
  • le_integrate tests
  • fix the fields_local benchmark
  • tidy and expand pfunit tests

Performance improvements

8.1 is not really a performance release. We have two big performance improvements coming up in future releases - an improved field solve, and better data layout - but 8.1 still includes some smaller improvements:

• We've vastly reduced creation and destruction of MPI communicators in the local field option, improving initialization performance, and enabling larger simulations which were previously failing due either to running out of communicators or to communicator fragmentation.
• We now store response matrices for reuse (for example, when the timestep halves and then doubles again). This can save orders-of-magnitude in reinitialization time when the timestep is changing frequently. For nonlinear runs we recommend setting both dump_response = T and read_response = T. This will save the response matrices from each timestep and reuse these when revisiting a timestep size. You may wish to set response_dir to a location to store these files to avoid a large number of files in the main run directory.
• We've improved indexing by removing "select" cases from tight loops.
• We've removed the less performant version of some routines (see breaking changes).
• We now avoid unnecessary initialization of unused data layouts.

Bugfixes

Here are a selection of the more notable squashed bugs. We've:

• tidied the source code, improved code structure and inline comments, and removed unused code
• fixed bugs in the ceq and leq geometries
• fixed the initial theta grid size for eikcoefs
• ensured FFTs are now bitwise reproducible by default
• added the missing benchmark option to the build_gs2 script
• reduced the bouncefuzz parameter. bouncefuzz adds a small tolerance to the inequality determining whether a pitch angle is forbidden, to account for roundoff error in lambda and B. Reducing bouncefuzz makes the calculation more accurate, so any changes will actually be a bugfix. This was introduced to prevent a numerical instability.
• Fixed box mode (e.g. nonlinear) runs with nperiod > 1.
• Fixed nonlinear source treatment when repeating an existing time step. Previously we would shuffle the time history even on failed time steps, resulting in errors in the nonlinear source on a time step change.

Breaking changes

Here are a list of changes that might need users to alter their input/analysis scripts, or that might change results:

• opt_redist_init is no longer an option - we only support the optimal version of the redistribution initialization routine. Users need to remove opt_redist_init from layout_knobs namelist in input files.
• For consistency, the new diagnostics now uses lower case for rplot, zplot, rprime and zprime. This has the potential to break analysis scripts.
• Changes to defaults: write_fluxes and write_fluxes_by_mode now default to true for nonlinear runs and false for nonlinear runs.
• Runs now abort if the user attempts to load a restart file with a different layout than the one set in the rest of the code / input file.