AD Model Builder/Tips and Tricks

From QERM Wiki
Jump to: navigation, search

Contents

About this page

This wiki page was converted from the Tricks, Tips and Ad Model Builder (a Living Document) written by André Punt. Conversion to wiki format is intended to breath more life into the document, but it may include formatting errors in the short term.

Also, some of the information here may be outdated, so keep up to date with developments by the admb-project.

Introduction

AD Model Builder (henceforth ADMB) provides a powerful framework for the estimation of parameters (and their variances, co-variances) for (highly) non-linear models. As such, its use can speed up the process of model development markedly. Unfortunately, there are many "undocumented features" associated with ADMB that can hinder progress (and even cause major bugs). This document outlines the experience of several ADMB users and is meant to be a living document. If you have anything to add to this document, please draft suitable paragraphs and email them to Robin Thomson (robin.thomson@csiro.au).

Some of the libraries used by ADMB (referred to as AutoDiff) can be used as libraries in standard C++ code. However, it is more common to develop models using the ADMB template (the TPL file) and using the ADMB preprocessor to generate a cpp file. This document concentrates on the latter. There is an NMFS "modelling site" (http://www.refm.noaa.gov/stocks/modeling.htm) that (sometimes) provides useful information about ADMB and its problems, and a list server--see the NMFS site for further details.

Key aspects of ADMB

There are four main uses for ADMB:

  • determination of the values for the model parameters that minimise some objective function (usually a likelihood function),
  • estimation of the variance-covariance matrix for the parameters (and chosen variables of interest),
  • calculation of likelihood profiles for quantities of interest (parameters, model outputs), and
  • development of posterior distributions based on a Bayesian analysis (using the MCMC algorithm).

The above order represents both the order of development and the frequency of use (and testing) by Dave Fournier. Therefore, the modules for estimating parameters are generally reliable (subject, of course, to the normal problems of model fitting). The variance and covariances for the model outputs are also considered reliable and robust. The likelihood profiles for the parameters appear correct. However, the likelihood profiles for other model outputs (e.g. biomass in a given year for a typical fisheries model) are questionable as (at one stage), these appeared to approximate Bayesian posteriors!

The results of the MCMC algorithm should be used with considerable caution. This is because the variant of the MCMC algorithm implemented in ADMB is not documented and many practitioners have encountered what must be errors in the results. My (Andre Punt) recommendation is only to use the results of the MCMC output if (a) the standard diagnostics (see Punt and Hilborn: Rev Fish. Biol. Fish. 7:35-63) indicate things are OK and (b) preferably if the calculations have been repeated using other MCMC (or SIR) software. I (AP) have implementations of SIR and MCMC in FORTRAN while Sally Wayte (CSIRO) has these in C++.

C++ compilers

  • Your ADMB CD will contain several directories, each with a different implementation of ADMB for a different C++ compiler. You may find that different implementation will behave differently. The gnu and Borland implementations seem to be the more robust than the microsoft one, possibly because they have been used more by Dave Fournier. So if you're having trouble with one compiler, try another. The gnu compiler is provided on the ADMB disc. You can install more than one version of ADMB on your machine, just put them in different directories and make sure that your path command points to whichever one you're using at command line.
  • The MS C++ compiler when used at the command line might give an INTERNAL COMPILER ERROR whenever you use a number or a variable declared as a non-differentiable type, within an internal function e.g. log(2.0) or exp(val) where val is declared as a double. You can fix this by declaring a dvariable called temp and passing the value that you want to include within the function to temp first. Or you can use MS C++ through the Visual Studio or start using another C++ compiler, like gnu C++.

Modifying your path command

  • The notes below refer to "the path". This is a variable that tells the operating system where to look for any files that the user might refer to. It will look first in the current directory and then will search through the directories listed in the PATH variable. You can see this list by opening a DOS window and typing path at the DOS prompt.
  • The old fashioned way of modifying the path is to edit (or create) a file called AUTOEXEC.BAT in the root directory (i.e. C:\). All versions of windows support this method but note that they all have a PATH variable with a pre-existing list so that you need to be careful not to overwrite this list. Open or create C:\AUTOEXEC.BAT. Add the following line:
 SET PATH=%path%; c:\admb\bin;
where admb is the directory where you installed the ADMB files. %path% is a placeholder for the pre-existing list so that your directoryis added to this list, instead of replacing it.
  • Alternatively, in some versions of windows you can modify the path by clicking on the start button, go to settings, control panel, system, advanced, environment variables. Find PATH and add your directory. Different versions of windows will differ a little in where the environment variable are specified.

Setting up C++

  • Before attempting to install ADMB, make sure that you have a working version of C++ installed on your computer.
  • You will need to add your C++ compiler's "BIN" and "LIB" directories to your path.
  • If you are going to use gnu C++ and have not already installed it, follow the instructions in the ming32 directory on the ADMB CD. To test the installation, open a DOS window and at the command line type gcc. You should get the following error message: GCC.EXE: no input files. If you don't then you have not installed gnu C++ properly, check your path and the extra environment variable mentioned in the installation notes.
  • If you have Microsoft C++ type cl in a DOS window and you should get a message saying usage: etc. If you just get an unknown file error message then you have not included the directory where CL.EXE resides in your path.
  • You may prefer to use C++ though its environment (e.g. the Microsoft Visual Studio), rather than at command line. Instructions on setting up the Microsoft Visual Studio to use ADMB libraries are given in Appendix I.

Setting up and using ADMB

  • Identify the directory on the ADMB CD that corresponds to your C++ compiler. If you have a newer version (v5 and above) of ADMB you will probably find an executable on the CD with a name like MSCDSK.EXE. Go to the directory where you want your ADMB files installed and type D:\ MSCDSK -d where D:\ is your CD-ROM drive and -d is the option that tells MSCDSK.EXE to install your files into BIN, LIB, INCLUDE subdirectories. If you forget the -d all the files will be copied into your current directory and the ADMB batch files won't work because they will be looking for BIN, LIB, INCLUDE subdirectories. Alternatively, copy all the files and sub-directories from the CD into a new directory on your computer e.g. "C:\ADMB".
  • Add the "C:\ADMB\BIN" directory to the path command on your computer.
  • You are now in a position to convert *.tpl files to *.cpp files by typing tpl2cpp program_name.tpl at the command line. (Where program_name is the name of your executable.)
  • The next step is to convert the program_name.cpp file to an .exe. This can either be done at the command line or through your version of C++'s environment e.g. Microsoft Visual Studio. Note that gnu C++ does not have an environment.
  • To use the CL command line compiler, Modify the "MYVCC.BAT" file in the "ADMB\BIN" directory so that it refers to the correct ADMB and C++ include directories. Modify the "LINKVCC.BAT" file so that it points to the "C:\ADMB\LIB" directory. There are also files called "MYVCCS.BAT" and "LINKVCCS.BAT" which produce "safe" versions of your program.
  • For the gnu compiler the batch files to be edited are called "MYEGCO.BAT" and "LINKEGCO.BAT". The "safe" equivalents have the same name but with the "O" changed to an "S".
  • You convert the program_name.cpp file to a program_name.exe file by typing "ADMB program_name" at the command prompt. This runs a batch file, ADM.BAT which in turn runs MYVCC.BAT and LINKVCC.BAT.
  • You may also need to copy some C++ *.H files into your "C:\ADMB\INCLUDE" directory. If you get an error message that looks something like cannot find IO.H when you try to run ADM.BAT find out where IO.H is and copy it into "C:\ADMB\INCLUDE". There should be about 6 of these files that need copying over. This is not a problem with newer versions of ADMB, v5 and above I think, and is only a problem with the Microsoft C++ compiler.
  • Another problem you may get the first time you try to run ADMB is an error message indicating PC.H not found. Find the reference to PC.H in the FVAR.HPP file and comment this line out.
  • You will probably get a warning about "unknown pragma" you can ignore this.

Input files

The key ADMB input file is the program_name.DAT file. Warnings here are:

  • Avoid the use of TABs when entering data because at least some version of ADMB will crash if TABs are encountered, this applied to the .tpl file as well.
  • Use the program_name.DAT file to read in all constants. It is horrible modifying a program that includes constants that have been "hard wired". An additional reason is that reading in array dimensions means that you don't have any arrays that are larger than they need to be.

Ensuring that models are differentiable

For ADMB to work, it is necessary that the function to be minimised be continuous and differentiable with respect to the parameters. This may seem like a trivial constraint but, for several fisheries problems, it is easy for the model not to satisfy this constraint.

  • Try not to use IF statements that have a model quantity (parameter or derived variable) in the IF clause, e.g. if (biomass > 0.5) x = 1; else x = 2; If you used this in an ADMB program, the function would not be differentiable when the parameters chosen equated to biomass=0.5! Note that it is OK to have if statements that are based on variables read in the DATA SECTION.
  • A particular instance of the above occurs with extinctions (biomass less than catch). This can occur with models that remove the catch each year exactly and set the biomass to something like 0.01 if the catch exceeds the biomass. One way to avoid this problem messing things up is to help ADMB to keep away from situations where the population is already extinct by adding a penalty to the likelihood function. This penalty could be of the form:
[need to replace equation]
where [need to replace equation] is the actual catch during year y, and
[need to replace equation] is the model-estimate of the catch during year y.
This term is ideally zero but will penalise cases in which the population went extinct so the model catch differs from the actual catch.
  • Keep the biomass from becoming zero (and causing errors when taking log's) by ensuring that the survival rate (1 - C/N) stays above some positive number x. This can be done using the built-in function posfun(). The syntax is y = posfun(x,eps,pen); where y <= x and >= eps and pen is a penalty function whose value is = 0.01*square(x-eps). x and pen are dvariables. You will need to add pen times some number (say 100) to the likelihood function as a penalty function. Before calling posfun() set pen = 0. because its value might be cumulative (depending on the version of AD Model Builder that you're using).
  • Another related problem is the use of (iterative) numerical algorithms (say to solve a non-linear equation) inside the function to be minimised (think about the ADAPT-VPA method). You must not run the algorithm until some tolerance (which would then be a function of the model parameters) is reached. Rather when using (iterative) numerical methods, pre-specify the number of iterations to use.
  • Avoid taking the maximum of a set of numbers.

Creating libraries

You can create your own library files containing commonly used functions. The names and argument types for these functions are contained in a header file (.h) and the name of this file is included in the .tpl.

  • the name of the header file goes into the GLOBALS_SECTION in the .tpl. For example:
GLOBALS_SECTION
#include <myheader.h>  
  • when tpl2cpp is run myheader.h is included at the top of the resulting .cpp file, before admodel.h is included therefore myheader.h must include admodel.h. An example of myheader.h follows:
#if ndef _MYHEADER_  //ensures that this code isn't included twice
#define _MYHEADER_
#include <admodel.h>
void get_numbers_at_age(dvariable& log_popascale; int nyrs, int nages, dvar_vector& log_initpop; dvar_matrix& S);
#endif
  • The code for get_numbers_at_age is stored in a separate .cpp file and must be valid C++ / Autodiff code. As the user will not see this code, it should not alter the values of any global variables. An example of mycode.cpp follows:
#include <admodel.h>
void get_numbers_at_age(dvariable& log_popascale; int nyrs, int nages, dvar_vector& log_initpop; dvar_matrix& S)
{
// ~ code ~
}


  • This code is called from the template file as follows:
PROCEDURE_SECTION
::get_numbers_at_age(log_popscale,nyrs,nages, log_initpop, S)
// the :: isn't compulsory but may be necessary is there is a clash of names
  • mycode.cpp is compiled as a "static-library application" and the mycode.lib is added to the string of library files, either in the link batch file or the link project settings depending on which compiler you're using.

Debugging

A useful trick when debugging is to read in a variable called "Diag". If this is zero no diagnostics are output and different levels of "Diag" correspond to different levels of diagnostic output. This means that you can keep the diagnostic outputs in a program even if a version designed for actual assessments but simply not output wads of unnecessary debug information.

It is possible to check for array bound errors so long as you are using your C compiler through its environment (i.e. not at command line):

  • type tpl2cpp -bounds program_name
  • compile the resulting program_name.cpp in the normal manner, ensuring that the safe and not the optimised version is being used (i.e. use the ads32.lib and not the ado32.lib library file when using Visual C and don't use OPT_LIB)
  • the -bounds option adds a function called ad_bound to your program. This function will be called if there is an array bounds error. Put a breakpoint in that function
  • run the program in debug mode
  • if you have an array bound error an error message will appear in the runtime window; the program will have stopped at the breakpoint in ad_bound, and the line that caused ad_bound to be called is the offending line
  • open the stack window (view menu: debug windows : call stack)
  • in the stack window, double click on the first line after the ad_bound call that gives a line number. This will then point to the line after the offending line in your code. Remember to correct errors in program_name.tpl, not in the .cpp file.

Runtime options

  • type program_name -? or program_name -help to view an (almost) full list of runtime options
  • when testing a program ignore the final Hessian calculation stage by using the "-est" option.
  • you can tidy up the output written to screen a lot by using the "-nox" option
  • to use a data file with a name other than program_name.dat or one that is not in the default sub-directory use: program_name -ind path\ name.dat
  • when using "-mcmc", "-mcrb n" where n is a number between 1 and 9 scales the correlation matrix can be used in order to get rid very high numbers, the larger the value of n the more severe the scaling, a value of 3 is recommended
  • to use starting values for mcmc other than the mode (stored in svfile_name.pin):
 program_name -mcmc 500000 -mcpin svfile_name.pin

This still uses the Hessian matrix calculated at the mode, not at the specified starting values.

  • there are three commands that can be issued while the program is executing:
    • q stops execution
    • n skips to the next execution phase
    • c invokes the derivative checker

More efficient code

  • ADMB is frequently used to implement "Integrated Analysis". It is possible to speed up the calculations markedly but removing the catch under the assumption that it is taken in the middle of the year and is known exactly. This means that it is not necessary to estimate a (fully-selected) fishing mortality for each year and fleet (however, negative N's can be a problem, see above).
  • Any variable declared in the DATA_SECTION will be a non-differentiated variable whereas any declared in the PARAMETER_SECTION will have a lot of additional code automatically generated whenever it is used in any calculation. Therefore it is a good idea to define variable as non-differentiated types unless their values are derived from parameters and are used in calculating the objective function value.
  • When calculating a quantity that uses the value of dvariables but does not need to be differentiated (e.g. some output statistic) make sure that this quantity is defined as a constant and use the value() function when calculating it e.g.
dvariable N;
double outstat;
outstat = value(N) / 2.;
  • Don't declare variables in the DATA_ and PARAMETER_ sections unless you need them to be global. Both differentiated and non-differentiated types can be declared locally within other sections or in FUNCTIONs. However their names will differ from those used in the DATA_ and PARAMETER_ sections. Appendix II shows the ADMB types.
  • You can assign a non-differentiated variable to a differentiated variable but not the other way around unless you use value().
  • Many fisheries models involve likelihood functions that include contributions from many sources. In several cases, data are not available for all years (e.g. the assessment starts in 1900 but data are only available from 1990). In such cases, avoid computing any unnecessary model predictions (e.g. predictions of the length-frequency of the catch to fit to observed length-frequencies for years for which actual length-frequency data are not available) as this can substantially reduce run times (AutoDiff keeps track of the derivatives of all model outputs, even if they are not used).
  • It is always good to implement a model using several sub-routines. This is generally easy to do. However, ADMB doesn't like passing integer parameters. The way around this is to include the loop counter as a global variable in the PARAMETER SECTION.
  • If there are calculations that involve modifying the data before they are used in the likelihood function (e.g. turning catch in number into catch proportion at age) do this in the PRELIMINARY CALCULATIONS SECTION as this reduces the amount of (unnecessary) calculations.
  • Always think about using phases when fitting a model; this can substantially reduce the time taken to solve the problem.
  • The values in the TOP_OF_MAIN_SECTION should be considered for every new model. Try different values and see if this affects run times--it can do so substantially.
  • When fitting models to age- and size-composition data, think seriously about "robustifying" the likelihood function. A simple way to do this is to add a small number to the squared residuals before they are included in the likelihood function. However, there are more sophisticated techniques (e.g. Fournier et al.: Can. J. Fish. Aquat. Sci. 47:301-311; 55:2105-2116).

Running tpl2cpp.exe from within MS Visual Studio

If you are using the Microsoft Visual Studio instead of the CL command line compiler you might not want to exit to DOS in order to run TPL2CPP.EXE whenever you alter your .tpl code. To run it from within the Visual Studio:

  • Add program_name.tpl to your project.
  • Click on program_name.tpl then on the Project menu, choose settings, in the Custom Build tab, Commands window, add:
 C:\ADMB\BIN\TPL2CPP.EXE $(InputName)
 move $(InputName).htp $(WkspDir)

In the Output window put:

 $(InputName).cpp
  • If you plan to run program_name.exe from the DOS command line you might want to add a copy command Project/Settings/ Post-build. Note this is the project's settings, not the .tpl file's settings.
 copy debug\program_name.exe
  • Another, unrelated, tip for Microsoft Developer Studio users: in order to have colour-coding applied to your .tpl file, open the .tpl file in the Developer Studio, right click somewhere in the middle of the screen, select Properties, on the General tab select "C/C++" in the Language window.


Here are a few other tips that we have picked up

  • catch is a reserved word in C++!
  • Even if the theory involves maximising the likelihood function, ADMB involves minimising the negative of the logarithm of the likelihood function. It is very embarrassing to find that instead of finding the values for the parameters that correspond to the best fit to the data, you found those that correspond to the worst fit to the data! This error is particularly easy to make when you have multinomial components in the likelihood function.
  • If you get weird error messages when multiplying or dividing real variables; try changing the order of the calculations, that can help! E.g. instead of "2*var" try "var*2".
  • ADMB "signals" an inability to find the minimum of the objective function by the Hessian not being positive definite. If you get the "Hessian not positive definite" message at the end of a run, it is time to (a) rerun the program, and (b) check if any parameters are hitting boundaries.
  • It is often best to impose bounds on all parameters (even if just for numerical stability reasons).
  • When implementing a multinomial likelihood, the term in the model is [[Image:]] where [[Image:]] is the observed proportion falling into category i, and [[Image:]] is the model estimate of the probability falling into category i. I (AP) have found that improved numerical stability arises by adding a constant to the negative log-likelihood so that the term [[Image:]] is minimised (this term equates to zero at its minimum).
  • If you are not sure why the answers don't seem correct, it is a good idea to code the model separately (e.g. in C or FORTRAN) and check that it works (i.e. given values for the model parameters, one gets the correct value for the likelihood function). Getting the model coded wrong is the most frequent cause of ADMB problems!
  • It is not advisable to pass parameters to subroutines. However, this can be done and the attached code (Appendix III) shows how (in a simple case); note the use of the GLOBALS section and the RETURN_ARRAYS statement. Note also that I am not totally sure how this is implemented but it seems to work!

Finally, remember when all else fails curse Dave Fournier! Or bother him with a whingeing email.

Appendix I

Setup options to compile and link ADMB projects using the MSVC v6.0 compiler and the interactive development environment.


Debug version


(a) When creating the new project create a blank 32 bit Windows console application in the folder where the source files are.


(b) In Project/Settings/C++/Code generation, ensure that the runtime library is Debug single-threaded.


(c) In Project/Settings/C++/Preprocessor/Preprocessor definitions add __MSVC32__ to the list so the list is: WIN32,_DEBUG,_CONSOLE,_MBCS,__MSVC32__


(d) In Project/Settings/ C++/Preprocessor/Additional include directories add the folder where the ADMB include files are, as well as the source folder for the current program e.g. c:\admodel\include,c:\admodel\examples\kalman-filter


(e) In Project/Settings/Link/General delete the Object/library models shown and add the ADMB ones: admod32.lib adt32.lib ads32.lib


(f) In Project/Settings/Link/Input add the ADMB library folder, e.g. c:\admodel\lib


You will probably see a warning: c:\admodel\include\admodel.h(3) : warning C4068: unknown pragma on compilation that can be ignored.


You might get the error message: PC.H not found. Find the reference to PC.H in FVAR.HPP and comment this line out.


Release version


All same as above except


(b) The runtime library is Single-threaded


(c) Add OPT_LIB to the default list of definitions as well as __MSVC32__ so that the definitions read WIN32,NDEBUG,_CONSOLE,_MBCS,__MSVC32__,OPT_LIB


(e) Change the library list to read: admod32.lib adt32.lib ado32.lib


Compiling DLLs


Select Win32 Dynamic-Link Library when creating a blank project. Other settings should be the same as above (except of course, that the default definitions are different).


Appendix II

Table showing the ADMB types. A "number" declared in the DATA_SECTION is equivalent to a "double" declared in e.g. the PRELIMINARY_CALCS or PROCEDURE_SECTION. Similarly a "number" in the PARAMETER_SECTION is equivalent to a "dvariable" declared elsewhere.


No differentiation Differentiated
DATA_SECTION PARAMETER_SECTION
number number
vector vector
matrix matrix
3darray 3darray
4darray 4darray
... ...
PROCEDURE_SECTION, etc PROCEDURE_SECTION, etc
double dvariable
dvector dvar_vector
dmatrix dvar_dmatrix
d3_array dvar3_array
d4_array d4_array
... ...

Appendix III

Example code: passing parameters to subroutines.

 GLOBALS_SECTION
 #include <admodel.h>
 dvariable mgexp(dvariable expo)
  {
   RETURN_ARRAYS_INCREMENT();
   dvariable over;
   if (expo > 50.0)
     over = mfexp(50.0);
   else  
    if (expo < -50)
      over = mfexp(-50.0);
    else
      over = mfexp(expo);  
   RETURN_ARRAYS_DECREMENT();
   return(over);
  }
Personal tools