carputils Overview

The increasing complexity of in silico experiments as they have become feasible over the past few years prompted for the development of a framework that simplifies the definition and execution of such experiments which become quickly intractable otherwise even for expert users. The major design goals were

Encapsulate the technical complexity to enable users to perform experiments at a more abstract level
Once built, limited technical expertise should be required to carry out in silico experiments, thus relieving users from undergoing months/years of training and thus save time to be spent on actual research
Expose only steering parameters required for a given experiment
Enable the easy sharing of experiments for collaborative research, re-use of used/published experiments or simply for the sake of remote debugging/advising
Provide built-in capabilities for generating high quality documentation of experiments

../_images/InsilicoExperiment_Definition.png

Simplified definition of in silico experiments within the CARPentry modeling framework. Discrete representations of hearts are stored in repositories from where they are loaded when needed. Not only the meshes themselves, but also derived discrete entities such as element/vertex lists or surfaces are stored there to be used for defining electrodes or boundary conditions. The location of all tools for performing the experiments and for analyzing experimental data are encapsulated in the python-based carputils package which is also used to describe the experiment, that is, link the exposed input parameters used to steer the experiment to create appropriate sets of commands which are used to execute the experiment. To share an experiment only the CARPentry experiment ID and the exact experimental protocol are needed to be replicated elsewhere.

carputils is a Python package containing a lot of CARPentry-related functionality and a framework for running CARPentry simulations and testing their outputs en masse. Provided functionality includes:

Scripting environment for designing complex in silico experiments
Generation of CARP command line inputs with:
- Automatic adding of MPI launcher
- Debugger and profiler attachment
- Automatic batch script generation and submission with platform-specific hardware profiles
Compact model setup using Python classes
Efficient IO of common CARPentry file formats to numpy arrays
Failure-compliant mass running of tests
Automatic comparison of test results against reference solutions

Structure

The CARP examples framework encompasses a number of repositories, the purpose of which can be initially confusing. This section aims to explain the structure.

A typical installation might look like:

software
├── carputils
├── carp-examples
│   ├── benchmarks
│   └── devtests
├── carp-meshes
└── carp-reference
    ├── benchmarks
    └── devtests

carputils: A repository containing most of the Python functionality used in setting up and postprocessing CARP simulations. The bin subdirectory should be added to your PATH environment variable, and the top level directory should be added to your PYTHONPATH. On first run of an experiment, or when using the automatic clone script, the settings.yaml file will be generated, which must be tailored to your setup.
carp-examples: This is a normal directory which contains 2 (or more) Python packages whose modules define experiments. Each run.py script in these packages runs an experiment. In fact, as long as carputils is set up correctly, you can move these run scripts anywhere on your system (along with any input files they depend on) and they will work, but in order to run regression tests they must be inside these packages and carp-examples must be added to the PYTHONPATH.
carp-meshes: This is an SVN repository containing meshes used by examples. As it is very large, you may instead use the --get-mesh flag with run scripts to get only their needed meshes. Note that most run scripts generate meshes programmatically, in which case this is not needed. The location of this repository is set in carputils/settings.yaml with MESH_DIR.
carp-reference: These directories contain the reference solutions for regression tests. The repositories under carp-reference must have the same name as those under carp-examples. These repositories are only needed for running regression tests. The location of carp-reference is set by the REGRESSION_REF settings in carputils/settings.yaml

Note

The actual structure above need not be conformed to. It only matters that carputils and carp-examples are in the PYTHONPATH, carputils/bin is in the PATH, and carp-meshes and carp-reference are at the locations specified in settings.yaml. Other than that you can put them where you want.

Concepts

The following concepts are used in the carputils framework:

sim: A single CARP execution - consecutive, related runs are considered to be multiple sims
job: One or more related sims launched by a single execution of a carputils run script
example: A carputils run script, capable of launching a single job per execution, though different jobs can be launched by changing the command line arguments
test: A fixed (according to the command line arguments) run configuration of an example plus several checks to determine the accuracy of the output
check: A method to determine the accuracy of the output of a test - most commonly by comparing with a previous reference solution but also may be by comparison with an analytic solution and other custom methods

Generate HTML Page

This documentation is based on Sphinx and can be generated using the Makefile in the carputils/doc folder.

cd carputils/doc
make clean
make html

The html webpages are found then in the folder carputils/doc/build/html. To view, point your favorite browser to

firefox ${CARPUTILS_DIR}/doc/build/html/index.html