Optimization in library mode

The library mode allows to tailor the evaluation of the objectives and constraints within a specialized executable that calls NOMAD shared object libraries.

For example, it is possible to link your own code with the NOMAD libraries (provided during installation or built) in a light executable that can define and run optimization for your problem. Contrary to the batch mode, this has the disadvantage that a crash within the executable (for example during the evaluation of a point) will end the optimization unless a special treatment of exception is provided by the user. But, as a counterpart, it offers more options and flexibility for blackbox integration and optimization management (display, pre- and post-processing, multiple optimizations, user search, etc.). See examples in Access to solution and optimization data, Multiple runs and fixed variables and Controlling runs with callbacks.

The library mode requires additional coding and compilation before conducting optimization. First, we will briefly review the compilation of source code to obtain NOMAD binaries (executable and shared object libraries) and how to use them. Then, details on how to interface your own code are presented.

Compilation of the source code

NOMAD source code files are located in $NOMAD_HOME/src. Examples are provided in $NOMAD_HOME/examples/basic/library and $NOMAD_HOME/examples/advanced/library.

The compilation procedure uses the provided CMake files along with the source code.

In what follows it is supposed that you have a write access to the source codes directory. If it is not the case, please consider making a copy in a more convenient location.

Using NOMAD libraries

Calling functionalities in NOMAD shared object libraries (so or dll) requires to build a C++ program and link it with the libraries to form an executable (Installation describes how to build the libraries and the examples). This is illustrated on the example located in the directory:

$NOMAD_HOME/examples/basic/library/example1

It is supposed that the environment variable NOMAD_HOME is defined and NOMAD shared object libraries are built. A basic knowledge of object oriented programming with C++ is assumed. For this example, just one C++ source file is used, but there could be a lot more.

Basic example 1

Library mode examples are built during the installation procedure. Let us first test the basic example to check that libraries are working fine and accessible:

> cd $NOMAD_HOME/examples/basic/library/example1
> ls
CMakeLists.txt                example1_lib.cpp        example1_lib.exe
> ./example1_lib.exe
All variables are granular. MAX_EVAL is set to 1000000 to prevent algorithm from circling around best solution indefinetely
BBE OBJ
1 -28247.525326  (Phase One)
5   -398.076167  (Phase One)
47   -413.531262
51   -490.074916
59   -656.349576
60  -1192.679165
65  -1595.921082
A termination criterion is reached: Maximum number of blackbox evaluations (Eval Global) No more points to evaluate 1000

Best feasible solution:     #171 ( 0.9 24.4 2.4 7.8 5.6 10.5 3.8 9.9 2.7 6.5 )        Evaluation OK    f = -1595.9210820000000695      h =   0

Best infeasible solution:   #66734 ( 0 -1.39247e+08 2.57422e+07 -6.45581e+06 -8.23276e+07 -8.42645e+06 7.52545e+07 6.46595e+07 1.91927e+07 3.1608e+07 )       Evaluation OK    f = -1999.9964250000000447      h =   0.5625

Blackbox evaluations:        1000
Total model evaluations:     64042
Cache hits:                  205
Total number of evaluations: 1205

Modify CMake files

As a first task, you can create a CMakeLists.txt for your source code(s) based on the one for the basic example 1.

add_executable(example1_lib.exe example1_lib.cpp )
target_include_directories(example1_lib.exe PRIVATE ${CMAKE_SOURCE_DIR}/src)
set_target_properties(example1_lib.exe PROPERTIES INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")

if(OpenMP_CXX_FOUND)
  target_link_libraries(example1_lib.exe PUBLIC nomadAlgos nomadUtils nomadEval OpenMP::OpenMP_CXX)
else()
  target_link_libraries(example1_lib.exe PUBLIC nomadAlgos nomadUtils nomadEval)
endif()

# installing executables and libraries
install(TARGETS example1_lib.exe  RUNTIME DESTINATION ${CMAKE_CURRENT_SOURCE_DIR} )

# Add a test for this example
if(BUILD_TESTS MATCHES ON)
   message(STATUS "    Add example library test 1")

   # Can run this test after install
   add_test(NAME Example1BasicLib COMMAND ${CMAKE_BINARY_DIR}/examples/runExampleTest.sh ./example1_lib.exe WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} )
endif()

If you include your problem into the $NOMAD_HOME/examples directories, you just need to copy the example CMakeLists.txt into your own problem directory (for example $NOMAD_HOME/examples/basic/library/myPb), change the name example1_lib with your choice and add the subdirectory into $NOMAD_HOME/examples/CMakeLists.txt:

add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/basic/library/myPb)

Modify C++ files

We now describe the other steps required for the creation of the source file (let us use example1.cpp) which is divided into two parts: a class for the description of the problem, and the main function.

The use of standard C++ types for reals and vectors is of course allowed within your code, but it is suggested that you use the NOMAD types as much as possible. For reals, NOMAD uses the class NOMAD::Double, and for vectors, the classes NOMAD::Point or NOMAD::ArrayOfDouble. A lot of functionalities have been coded for theses classes, which are visible in files $NOMAD_HOME/src/Math/*.hpp.

The namespace NOMAD is used for all NOMAD types, and you must type NOMAD:: in front of all types unless you type using namespace NOMAD; at the beginning of your program.

Providing the blackbox evaluation of objective and constraints directly in the code avoids the use of temporary files and system calls by the algorithm. This is achieved by defining a derived class (let us call it My_Evaluator) that inherits from the class NOMAD::Evaluator. The blackbox evaluation is programmed in a user-defined class that will be automatically called by the algorithm.}

/**
 \file   example1_lib.cpp
 \brief  Library example for nomad
 \author Viviane Rochon Montplaisir
 \date   2017
 */

#include "Nomad/nomad.hpp"

/*----------------------------------------*/
/*               The problem              */
/*----------------------------------------*/
class My_Evaluator : public NOMAD::Evaluator
{
public:
    My_Evaluator(const std::shared_ptr<NOMAD::EvalParameters>& evalParams)
    : NOMAD::Evaluator(evalParams, NOMAD::EvalType::BB)
    {}

    ~My_Evaluator() {}

    bool eval_x(NOMAD::EvalPoint &x, const NOMAD::Double &hMax, bool &countEval) const override
    {
        bool eval_ok = false;
        // Based on G2.
        NOMAD::Double f = 1e+20, g1 = 1e+20, g2 = 1e+20;
        NOMAD::Double sum1 = 0.0, sum2 = 0.0, sum3 = 0.0, prod1 = 1.0, prod2 = 1.0;
        size_t n = x.size();

        try
        {
            for (size_t i = 0; i < n ; i++)
            {
                sum1  += pow(cos(x[i].todouble()), 4);
                sum2  += x[i];
                sum3  += (i+1)*x[i]*x[i];
                prod1 *= pow(cos(x[i].todouble()), 2);
                if (prod2 != 0.0)
                {
                    if (x[i] == 0.0)
                    {
                        prod2 = 0.0;
                    }
                    else
                    {
                        prod2 *= x[i];
                    }
                }
            }

            g1 = -prod2 + 0.75;
            g2 = sum2 -7.5 * n;

            f = 10*g1 + 10*g2;
            if (0.0 != sum3)
            {
                f -= ((sum1 -2*prod1) / sum3.sqrt()).abs();
            }
            // Scale
            if (f.isDefined())
            {
                f *= 1e-5;
            }

            NOMAD::Double c2000 = -f-2000;
            auto bbOutputType = _evalParams->getAttributeValue<NOMAD::BBOutputTypeList>("BB_OUTPUT_TYPE");
            std::string bbo = g1.tostring();
            bbo += " " + g2.tostring();
            bbo += " " + f.tostring();
            bbo += " " + c2000.tostring();

            x.setBBO(bbo);

            eval_ok = true;
        }
        catch (std::exception &e)
        {
            std::string err("Exception: ");
            err += e.what();
            throw std::logic_error(err);
        }

        countEval = true;
        return eval_ok;
    }
  };

The argument x (in/out in eval_x()) corresponds to an evaluation point, i.e. a vector containing the coordinates of the point to be evaluated, and also the result of the evaluation. The coordinates are accessed with the operator [] (x[0] for the first coordinate) and outputs are set with x.setBBO(bbo);. The outputs are returned as a string that will be interpreted by NOMAD based on the BB_OUTPUT_TYPE defined by the user. We recall that constraints must be represented by values \(c_j\) for a constraint \(c_j \leq 0\).

The second argument, the real h_max (in), corresponds to the current value of the barrier \(h_{max}\) parameter. It is not used in this example but it may be used to interrupt an expensive evaluation if the constraint violation value \(h\) grows larger than \(h_{max}\). See [AuDe09a] for the definition of \(h\) and \(h_{max}\) and of the Progressive Barrier method for handling constraints.

The third argument, countEval (out), needs to be set to true if the evaluation counts as a blackbox evaluation, and false otherwise (for example, if the user interrupts an evaluation with the \(h_{max}\) criterion before it costs some expensive computations, then set countEval to false).

Finally, note that the call to eval_x() inside the NOMAD code is inserted into a try block. This means that if an error is detected inside the eval_x() function, an exception should be thrown. The choice for the type of this exception is left to the user, but NOMAD::Exception is available. If an exception is thrown by the user-defined function, then the associated evaluation is tagged as a failure and not counted unless the user explicitely set the flag countEval to true.

Setting parameters

Once your problem has been defined, the main function can be written. NOMAD routines may throw C++ exceptions, so it is recommended that you put your code into a try block.

/*------------------------------------------*/
/*            NOMAD main function           */
/*------------------------------------------*/
int main (int argc, char **argv)
{

    NOMAD::MainStep TheMainStep;

    auto params = std::make_shared<NOMAD::AllParameters>();
    initAllParams(params);
    TheMainStep.setAllParameters(params);

    std::unique_ptr<My_Evaluator> ev(new My_Evaluator(params->getEvalParams()));
    TheMainStep.setEvaluator(std::move(ev));

    try
    {
        TheMainStep.start();
        TheMainStep.run();
        TheMainStep.end();
    }

    catch(std::exception &e)
    {
        std::cerr << "\nNOMAD has been interrupted (" << e.what() << ")\n\n";
    }

    return 0;
}

The execution of NOMAD is controlled by the NOMAD::MainStep class using the start, run and end functions. The user defined NOMAD::Evaluator is set into the NOMAD::MainStep.

The base evaluator constructor takes an NOMAD::EvalParameters as input. The evaluation parameters are included into a NOMAD::AllParameters.

Hence, in library mode, the main function must declare a NOMAD::AllParameters object to set all types of parameters. Parameter names are the same as in batch mode but may be defined programmatically.

A parameter PNAME is set with the method AllParameters::setAttributeValue( "PNAME", PNameValue). The PNameValue must be of a type registered for the PNAME parameter.

Warning

If the PNameValue has not the type associated to the PName parameters, the compilation will succeed but execution will be stopped when setting or getting the value.

Note

A brief description (including the NOMAD:: type) of all parameters is given Complete list of parameters. More information on parameters can be obtained by running $NOMAD_HOME/bin/nomad -h KEYWORD.

For the example, the parameters are set in

void initAllParams(std::shared_ptr<NOMAD::AllParameters> allParams)
{
    // Parameters creation
    // Number of variables
    size_t n = 10;
    allParams->setAttributeValue( "DIMENSION", n);
    // The algorithm terminates after
    // this number of black-box evaluations
    allParams->setAttributeValue( "MAX_BB_EVAL", 1000);
    // Starting point
    allParams->setAttributeValue( "X0", NOMAD::Point(n, 7.0) );

    allParams->getPbParams()->setAttributeValue("GRANULARITY", NOMAD::ArrayOfDouble(n, 0.0000001));

    // Constraints and objective
    NOMAD::BBOutputTypeList bbOutputTypes;
    bbOutputTypes.push_back(NOMAD::BBOutputType::PB);     // g1
    bbOutputTypes.push_back(NOMAD::BBOutputType::PB);     // g2
    bbOutputTypes.push_back(NOMAD::BBOutputType::OBJ);    // f
    bbOutputTypes.push_back(NOMAD::BBOutputType::EB);     // c2000
    allParams->setAttributeValue("BB_OUTPUT_TYPE", bbOutputTypes );
    allParams->setAttributeValue("DIRECTION_TYPE", NOMAD::DirectionType::ORTHO_2N);
    allParams->setAttributeValue("DISPLAY_DEGREE", 2);
    allParams->setAttributeValue("DISPLAY_ALL_EVAL", false);
    allParams->setAttributeValue("DISPLAY_UNSUCCESSFUL", false);

    allParams->getRunParams()->setAttributeValue("HOT_RESTART_READ_FILES", false);
    allParams->getRunParams()->setAttributeValue("HOT_RESTART_WRITE_FILES", false);


    // Parameters validation
    allParams->checkAndComply();

}

The checkAndComply function must be called to ensure that parameters are compatible. Otherwise an exception is triggered.

Access to solution and optimization data

In the basic example 1, final information is displayed at the end of an algorithm. More specialized access to solution and optimization data is allowed.

To access the best feasible and infeasible points, use

NOMAD::CacheBase::getInstance()->findBestFeas(bf, NOMAD::Point(n), NOMAD::EvalType::BB, NOMAD::ComputeType::STANDARD);

NOMAD::CacheBase::getInstance()->findBestInf(bi, NOMAD::INF, NOMAD::Point(n), NOMAD::EvalType::BB, NOMAD::ComputeType::STANDARD);

To get the run flag of a run (success or type of fail) use the function

NOMAD::MainStep::getRunFlag()

The run flag is an integer to indicate the optimization termination status. For example, a run flag 0 corresponds to objective target reached OR Mads converged (mesh criterion) to a feasible point (true problem). The different run flags and their meaning are provided in $NOMAD_HOME/src/Algos/MainStep.h.

It is also possible to have access to the termination reason of the run by calling

stopReason = TheMainStep.getAllStopReasons()->getStopReasonAsString();

Where stopReason is a string.

Detailed success stats are also available

auto successStats = TheMainStep.getSuccessStats()

Multiple runs and fixed variables

An example of multiple runs of algorithm while changing the fixed variables is provided in $NOMAD_HOME/examples/advanced/library/FixedVariable.

Controlling runs with callbacks

Various types of callbacks functions are available to control the unfolding of a run. Callback functions must be added either to a MainStep object or the EvaluatorControl.

For example, after each iteration, we want to verify if the algorithm should stop or not based on a user defined criterion. To do that, we first need to add (“register”) the callback to a MainStep

TheMainStep.addCallback(NOMAD::CallbackType::MEGA_ITERATION_END, userIterationCallback);

The first attribute of the function is the type of callback that define when “the calling” must occurs. The second argument is the user defined function

void userIterationCallback(const NOMAD::Step& step, bool &stop)

The stop is set to true to indicate that the MainStep must stop.

This is used in $NOMAD_HOME/examples/basic/library/StopOnFTarget to implement a stop using a user defined objective target criterion. Other examples of user defined criterions are provided in $NOMAD_HOME/examples/advanced/library/.

In the StopOnFTarget example, the callback is called only at the end of an iteration (MEGA_ITERATION_END) and several points can be evaluated after reaching the F target. It is possible to stop the run after any evaluation by adding a callback to the ‘EvaluatorControl’. An example to stop the run when an evaluation fails is provided in $NOMAD_HOME/examples/advanced/library/StopIfBBFails.

Matlab interface

Note

NOMAD solver and Matlab for blackbox evaluation can be used in combination. There are two ways to perform optimization using objective and constraint function evaluated as Matlab code.

The simplest way is to start and run Matlab as a blackbox in batch mode to evaluate each given point. An example is provided in $NOMAD_HOME/examples/basic/batch/MatlabBB. Please note that this strategy is practical only for costly Matlab evaluation because of the important overhead time cost due to Matlab start sequence.

Another way is to build the Matlab MEX interface for NOMAD as described in what follows. Because NOMAD “runs within” a single Matlab, there is no overhead time cost to start Matlab.

Note

Building the Matlab MEX interface requires compatibility of the versions of Matlab and the compiler. Check the compatibility at MathWorks.

The Matlab MEX interface allows to run NOMAD within the command line of Matlab. Some examples and source codes are provided in $NOMAD_HOME/interfaces/Matlab_MEX. To enable the building of the interface, option -DBUILD_INTERFACE_MATLAB=ON must be set when configuring for building NOMAD, as such: cmake -DTEST_OPENMP=OFF -DBUILD_INTERFACE_MATLAB=ON -S . -B build/release.

Warning

In some occasions, CMake cannot find Matlab installation directory. The option -DMatlab_ROOT_DIR=/Path/To/Matlab/Install/Dir must be passed during configuration.

Warning

Building the Matlab MEX interface is disabled when NOMAD uses OpenMP. Hence, the option -DTEST_OPENMP=OFF must be passed during configuration.

The command cmake --build build/release (or cmake --build build/release --config Release for Windows) is used for building the selected configuration. The command cmake --install build/release must be run before using the Matlab nomadOpt function. Also, the Matlab command addpath(strcat(getenv('NOMAD_HOME'),'/build/release/lib')) or addpath(strcat(getenv('NOMAD_HOME'),'/build/release/lib64')) must be executed to have access to the libraries and run the examples.

All functionalities of NOMAD are available in nomadOpt. NOMAD parameters are provided in a Matlab structure with keywords and values using the same syntax as used in the NOMAD parameter files. For example, params = struct('initial_mesh_size','* 10','MAX_BB_EVAL','100');

Help on NOMAD parameters is accessible at the Matlab prompt: nomadOpt('-h param_name').

Note

More details for Windows installation are provided in Guide to build Matlab interface for Windows using MinGW.

PyNomad interface

Note

NOMAD and Python can be used in combination. There are two ways to perform optimization using objective and constraint function evaluated by a Python script.

The simplest way is to run the Python script as a blackbox in batch mode to evaluate each given point. An example is provided in $NOMAD_HOME/examples/basic/batch/PythonBB.

Another way is to obtain the Nomad interface for Python (PyNomadBBO or PyNomad for short). Since version 4.4, the PyNomadBBO package can be install from PyPI:

pip install PyNomadBBO

PyNomadBBO from PyPI relies on Python3 version 3.8+. We recommend to install PyNomadBBO into a virtual environment.

Please note that PyNomad wheels may not be available for your OS and Python version.

PyNomad can also be obtained by building source codes. The source codes and basic tests are provided in $NOMAD_HOME/interfaces/PyNomad. Examples are given in $NOMAD_HOME/examples/advanced/library/PyNomad.

Note

The build procedure relies on Python 3.8+, a recent version of Cython, wheel and setuptools. A simple way to have all packages for PyNomad build is work within an Anaconda <http://www.anaconda.org/> environment or a virtual environment.

To enable the building of the Python interface, option -DBUILD_INTERFACE_PYTHON=ON must be set when configuring for building NOMAD. The configuration command cmake -DBUILD_INTERFACE_PYTHON=ON -S . -B build/release must be performed within a Conda environment with Cython available (conda activate ... or activate ...).

In some situations, the configuration command should be adapted depending on the environment available. For Windows, the default Anaconda is Win64. Visual Studio can support both Win32 and Win64 compilations. The configuration must be forced to use Win64 with a command such as cmake -DBUILD_INTERFACE_PYTHON=ON -S . -B build/release -G"Visual Studio 15 2017 Win64". The Visual Studio version must be adapted.

The command cmake --build build/release (or cmake --build build/release --config Release for Windows) is used for building the selected configuration.

The command cmake --install build/release must be run before using the PyNomad module.

All functionalities of NOMAD are available in PyNomad. NOMAD parameters are provided in a list of strings using the same syntax as used in the NOMAD parameter files. Some basic tests are available in the PyNomad directory to check that everything is up and running.

C interface

A C interface for NOMAD is available. The source codes are provided in $NOMAD_HOME/interfaces/CInterface/. To enable the building of the C interface, option -DBUILD_INTERFACE_C=ON must be set when building NOMAD, as such: cmake -DBUILD_TESTS=ON -S . -B build/release.

The command cmake --build build/release (or cmake --build build/release --config Release for Windows) is used for building the selected configuration.

The command cmake --install build/release must be run before using the library.

All functionalities of NOMAD are available in the C interface. NOMAD parameters are provided via these functions:

bool addNomadParam(NomadProblem nomad_problem, char *keyword_value_pair);
bool addNomadValParam(NomadProblem nomad_problem, char *keyword, int value);
bool addNomadBoolParam(NomadProblem nomad_problem, char *keyword, bool value);
bool addNomadStringParam(NomadProblem nomad_problem, char *keyword, char *param_str);
bool addNomadArrayOfDoubleParam(NomadProblem nomad_problem, char *keyword, double *array_param);

See examples that are proposed in the $NOMAD_HOME/examples/advanced/library/c_api directory.