multiPoint
multiPointSparse
is a helper module for doing multipoint
optimizations. The documentation multiPoint
is given below.
To install, first clone the repository, then go into the root directory and type:
pip install .
For stability we recommend cloning or checking out a tagged release.
Tutorial
The goal of MultiPoint
is to facilitate optimization problems that
contain may different computations, all occurring in parallel, each of
which may be parallel. MultiPoint
effectively hides the required
MPI communication from the user which results in more readable, more
robust and easier to understand optimization scripts.
For our simple example, lets assume we have two parallel codes, A
and B
that we want to run at the same time for an
optimization. The computations of A
and B
do not directly
depend on each other; that is they can be executed in an embarrassingly
parallel fashion. Lets say we need to run 2 copies of code A
, and
one copy of code B
. The analysis path would look like:
Objective
Functions
/------------\ Funcs
/---| Code A |--->----\ User supplied objcon
| \------------/ | /---------------\
Optimizer | /------------\ Funcs | Combine | Combine funcs | Return to
----->---- +---| Code A |--->--- +------>--------+ to get final |----->------
Input | \------------/ | all funcs | obj/con | optimizer
| /------------\ Funcs | \---------------/
\---| Code B |--->----/
\------------/
Lets also assume, that the first copy of code A
requires 3
processors, the second copy of code A
requires 2 processors and
the copy of code B
requires 4 processors. For this case, we would
require 3 + 2 + 4 = 9
total processors. Scripts using
MultiPointSparse
must be called with precisely the correct
number of processors.
>>> from mpi4py import MPI
>>> from multipoint import multiPointSparse
>>> MP = multiPointSparse(MPI.COMM_WORLD)
>>> MP.addProcessorSet('codeA', 2, [3, 2])
>>> MP.addProcessorSet('codeB', 1, 4)
>>> comm, setComm, setFlags, groupFlags, ptID = MP.createCommunicators()
>>> setName = MP.getSetName()
At this point, you should have the following if you executed the code with 9 processors:
>>> print("setName={}, comm.rank={}, comm.size={}, setComm.rank={}, setComm.size={}, setFlags={}, ptID={}".format(setName, comm.rank, comm.size, setComm.rank, setComm.size, setFlags, ptID))
setName=codeA, comm.rank=0, comm.size=3, setComm.rank=0, setComm.size=5, setFlags={'codeA': True, 'codeB': False}, ptID=0
setName=codeA, comm.rank=1, comm.size=3, setComm.rank=1, setComm.size=5, setFlags={'codeA': True, 'codeB': False}, ptID=0
setName=codeA, comm.rank=2, comm.size=3, setComm.rank=2, setComm.size=5, setFlags={'codeA': True, 'codeB': False}, ptID=0
setName=codeA, comm.rank=0, comm.size=2, setComm.rank=3, setComm.size=5, setFlags={'codeA': True, 'codeB': False}, ptID=1
setName=codeA, comm.rank=1, comm.size=2, setComm.rank=4, setComm.size=5, setFlags={'codeA': True, 'codeB': False}, ptID=1
setName=codeB, comm.rank=0, comm.size=4, setComm.rank=0, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, ptID=0
setName=codeB, comm.rank=1, comm.size=4, setComm.rank=1, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, ptID=0
setName=codeB, comm.rank=2, comm.size=4, setComm.rank=2, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, ptID=0
setName=codeB, comm.rank=3, comm.size=4, setComm.rank=3, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, ptID=0
The input to each of the Objective Functions is the (unmodified) dictionary of optimization variables from pyOptSparse. Each code is then required to use the optimization variables as it requires.
The output from each of Objective functions funcs
is a Python
dictionary of computed values. For computed values that are
different for each member in a processorSet or between processorSets
it is necessary to use unique keys. It is therefore necessary for
the user to use an appropriate name mangling scheme.
In the example above we have two copies of Code A. In typical usage, these two instances will produce the same number and type of quantities but at different operating conditions or other similar variation. Since we need these quantities for either the optimization objective or constraints, these values must be given a unique name.
A simple name-mangling scheme is to simply use the ptID
variable that
is returned from the call to createCommunicators:
def objA(x):
funcs['A_%d'%ptID] = function_of_x()
return funcs
A similar thing can be done for B
:
def objB(x):
funcs['B_%d'%ptID] = function_of_x()
return funcs
A processorSet
is characterized by a single “objective” and
“sensitivity” function. For each processorSet
we must supply Python
functions for the objective and sensitivity evaluation.
>>> MP.setProcSetObjFunc('codeA', objA)
>>> MP.setProcSetObjFunc('codeB', objB)
>>> MP.setProcSetSensFunc('codeA', sensA)
>>> MP.setProcSetSensFunc('cdoeB', sensB)
The functions sensA
and sensB
must compute derivatives of the
functionals with respect to the design variables defined in the
optProb
Optimization problem class. Derivatives use the dictionary
sensitivity return format described in pyOptSparse
documentation.
multiPointSparse
will then automatically communicate the values
and call the user supplied objcon
function with the total set of
functions. The purpose of objcon
is to combine functions from the
individual objective functions to form the final objective and
constraint dictionary for pyOptSparse
. A schematic of this
process is given below:
Pass-through keys
/----------->--------------------\
all funcs | | output to pyOptSparse
------->------ + input /--------\ output |------------->----
\-------->--+ objcon | ----------/
keys \--------/ keys
multiPointSparse
analyzes the optimization object and determine if
any of the required constraint keys are already present in all funcs,
these keys are flagged as “pass-through”…that is they “by-pass”
entirely the objcon
function. The purpose therefore of objcon is
to use the remaining functions in all funcs
(the input keys
)
to compute the remainder of the required constraints (output keys
)
and objective. For example:
def objcon(funcs):
fobj = 0.0
for i in range(2):
fobj += funcs['A_%d'%i]
fobj /= funcs[B_0]
fcon['B_con'] = funcs[B_0]/funcs[A_0]
return fobj, fcon
There all three values contribute to the objective, while A_0
and
B_0
combine to form the constraint B_con
. This example has no
pass-though keys
.
Generally speaking, the computations in objcon should be simple and
not overly computationally intensive. The sensitivity of the output
keys
with respect to the input keys
is computed automatically by
multiPointSparse
using the complex step method.
Warning
Pass-through keys cannot be used in objcon.
Warning
Computations in objcon must be able to use complex
number. Generally this will mean if numpy arrays are used, the
dtype=complex
keyword argument is used.
The objcon
function is set using the call:
>>> MP.setObjCon(objCon)
As noted earlier, multiPointSparse
uses the optimization problem
to determine which keys are already constraints and which need to be
combined in objcon
. This is done using:
>>> optProb = Optimization('opt', MP.obj)
>>> # Setup optimization problem
>>> # MP needs the optProb after everything is setup.
>>> MP.setOptProb(optProb)
>>> # Create optimizer and use MP.sens for the sensitivity function on opt call
>>> snopt(optProb, sens=MP.sens, ...)
Examples
Example 1
In this example we show basic operations how to create and access specific sets or processor groups. To start we begin by importing the necessary modules.
# ==============================================================================
# Import modules
# ==============================================================================
from mpi4py import MPI
from multipoint import multiPointSparse
In this example we assume we have two codes or processor sets, codeA
and codeB
.
We assume 3 instances of codeA
where each instance requires 3, 2, and 1 processors.
For set codeB
we assume only 1 instance that requires 4 processors.
Following the creation of the points sets the communicators are created.
# ==============================================================================
# Processor allocation
# ==============================================================================
# Instantiate the multipoint object
MP = multiPointSparse(MPI.COMM_WORLD)
# Add all processor sets and create the communicators
MP.addProcessorSet("codeA", 3, [3, 2, 1])
MP.addProcessorSet("codeB", 1, 4)
comm, setComm, setFlags, groupFlags, ptID = MP.createCommunicators()
# Extract setName on a given processor for convenience
setName = MP.getSetName()
# Create all directories for all groups in all sets
ptDirs = MP.createDirectories("./output")
# For information, print out all values on all processors
print(
f"setName={setName}, comm.rank={comm.rank}, comm.size={comm.size}, setComm.rank={setComm.rank}, setComm.size={setComm.size}, setFlags={setFlags}, groupFlags={groupFlags}, ptID={ptID}"
)
Printing all values gives the following:
setName=codeA, comm.rank=1, comm.size=3, setComm.rank=1, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[ True False False], ptID=0
setName=codeA, comm.rank=2, comm.size=3, setComm.rank=2, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[ True False False], ptID=0
setName=codeA, comm.rank=0, comm.size=2, setComm.rank=3, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[False True False], ptID=1
setName=codeA, comm.rank=1, comm.size=2, setComm.rank=4, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[False True False], ptID=1
setName=codeA, comm.rank=0, comm.size=1, setComm.rank=5, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[False False True], ptID=2
setName=codeB, comm.rank=0, comm.size=4, setComm.rank=0, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, groupFlags=[ True], ptID=0
setName=codeB, comm.rank=1, comm.size=4, setComm.rank=1, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, groupFlags=[ True], ptID=0
setName=codeB, comm.rank=2, comm.size=4, setComm.rank=2, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, groupFlags=[ True], ptID=0
setName=codeB, comm.rank=3, comm.size=4, setComm.rank=3, setComm.size=4, setFlags={'codeA': False, 'codeB': True}, groupFlags=[ True], ptID=0
setName=codeA, comm.rank=0, comm.size=3, setComm.rank=0, setComm.size=6, setFlags={'codeA': True, 'codeB': False}, groupFlags=[ True False False], ptID=0
To perform operations on all processors in a set we can use the setFlags.
This is convenient if we want to perform the same operation on all processors in a given set.
Furthermore, if we want to access only a specific group in a set, the ptID
can be conveniently used as shown.
# ==============================================================================
# Problem setup
# ==============================================================================
# To perform operations on all processors in a set we can use the setFlags
if setFlags["codeA"]: # Alternatively, setName == "codeA" could be used here
# ...
# To access a particular group within the set can be done using the ptID
# Here we access only the processors in the first group
if 0 == ptID:
print(f"setName={setName} comm.rank={comm.rank} ptID={ptID}")
# To access all groups (but still a specific one) we simply loop over the size of the set
for i in range(setComm.size):
if i == ptID:
print(f"setName={setName} comm.rank={comm.rank} ptID={ptID} i={i}")
# Similarly, for the other processor set
if setFlags["codeB"]:
for i in range(setComm.size):
if i == ptID:
print(f"setName={setName} comm.rank={comm.rank} ptID={ptID} i={i}")
API
- class multipoint.multiPointSparse.multiPointSparse(gcomm)[source]
Create the multiPoint class on the provided comm.
- Parameters
- gcommMPI.Intracomm
Global MPI communicator from which all processor groups are created. It is usually MPI_COMM_WORLD but may be another intraCommunicator that has already been created.
Notes
multiPointSparse requires
useGroups=True
(default) when creating the optProb (Optimization instance).Examples
We will setup a multipoint problem with two procSets: a ‘cruise’ set with 3 members and 32 procs each, and a maneuver set with two members with 10 and 20 procs respectively. Our script will have to define 5 python functions:
Evaluate functions for cruise:
def cruiseObj(x): funcs = {} # Fill up with functions ... return funcs
Evaluate functions for maneuver:
def maneuverObj(x): funcs = {} # Fill up with functions ... return funcs
Evaluate function sensitivity for cruise:
def cruiseSens(x, funcs): funcSens = {} ... return funcSens
Evaluate function sensitivity for cruise:
def maneuverSens(x, funcs): funcSens = {} ... return funcSens
Function to compute addition functions:
def objCon(funcs): funcs['new_func'] = combination_of_funcs ... return funcs
>>> MP = multiPointSparse.multiPoint(MPI.COMM_WORLD) >>> MP.addProcessorSet('cruise', 3, 32) >>> MP.addProcessorSet('maneuver', 2, [10, 20]) >>> # Possibly create directories >>> ptDirs = MP.createDirectories('/home/user/output/') >>> # Get the communicators and flags >>> comm, setComm, setFlags, groupFlags, ptID = MP.createCommunicators() >>> # Setup problems and python functions >>> .... >>> MP.setProcSetObjFunc('cruise', cruiseObj) >>> MP.setProcSetObjFunc('maneuver', maneuverObj) >>> MP.setProcSetSensFunc('cruise', cruiseSens) >>> MP.setProcSetSensFunc('maneuver', maneuverSens) >>> MP.setObjCon(objCon) >>> # Create optimization problem using MP.obj >>> optProb = Optimization('opt', MP.obj) >>> # Setup optimization problem >>> # MP needs the optProb after everything is setup. >>> MP.setOptProb(optProb) >>> # Create optimizer and use MP.sens for the sensitivity function on opt call >>> snopt(optProb, sens=MP.sens, ...)
- addConsAsObjConInputs(cons)[source]
This function allows functions to be used both as constraints, as well as inputs to the ObjCon, therefore no longer bypassed.
- Parameters
- consstring or list of strings
The constraint names the user wants to use as ObjCon inputs
- addDVsAsFunctions(dvs)[source]
This function allows you to specify a list of design variables to be explicitly used as functions. Essentially, we just copy the values of the DVs directly into keys in ‘funcs’ and automatically generate an identity Jacobian. This allows the remainder of the objective/sensitivity computations to be proceed as per usual.
- Parameters
- dvsstring or list of strings
The DV names the user wants to use directly as functions
- addProcSetObjFunc(setName, func)[source]
Add an additional python function handle to compute the functionals
- Parameters
- setNamestr
Name of set we are setting the function for
- funcPython function
Python function handle
- addProcSetSensFunc(setName, func)[source]
Add an additional python function handle to compute the derivative of the functionals
- Parameters
- setNamestr
Name of set we are setting the function for
- funcPython function
Python function handle
- addProcessorSet(setName, nMembers, memberSizes)[source]
A Processor set is defined as one or more groups of processors that use the same obj() and sens() routines. Members of processor sets typically, but not necessarily, return the same number of functions. In all cases, the function names must be unique.
- Parameters
- setNamestr
Name of process set. Process set names must be unique.
- nMembersint
Number of members in the set.
- memberSizesint, iteratable
Number of processors on each set. If an integer is supplied all members use the same number of processors. If a list or array is provided, a different number of processors on each member can be specified.
Examples
>>> MP = multiPointSparse.multiPoint(MPI.COMM_WORLD) >>> MP.addProcessorSet('cruise', 3, 32) >>> MP.addProcessorSet('maneuver', 2, [10, 20])
The
cruise
set creates 3 processor groups, each of size 32. and themaneuver
set creates 2 processor groups, of size 10 and 20.
- createCommunicators()[source]
Create the communicators after all the procSets have been added. All procSets MUST be added before this routine is called.
- Returns
- commMPI.Intracomm
This is the communicator for the member of the procSet. Basically, this is the communicator that the (parallel) analysis should be created on.
- setCommMPI.Intracomm
This is the communicator that spans the entire processor set.
- setFlagsdict
This is a dictionary whose entry for
setName
, as specified in addProcessorSet() is True on a processor belonging to that set.- groupFlagslist
This list is used to distinguish between members within a processor set. This list of of length nMembers and the ith entry is true for the ith group.
- ptIDint
This is the index of the group that this processor belongs to.
Examples
>>> MP = multiPointSparse.multiPoint(MPI.COMM_WORLD) >>> MP.addProcessorSet('cruise', 3, 32) >>> MP.addProcessorSet('maneuver', 2, [10, 20]) >>> comm, setComm, setFlags, groupFlags, ptID = MP.createCommunicators()
The following will be true for all processors for the second member of the
cruise
procSet.>>> setFlags['cruise'] and groupFlags[1] == True
- createDirectories(rootDir)[source]
This function can be called only after all the procSets have been added. This can facilitate distinguishing output files when there are a large number of procSets and/or members of procSets.
- Parameters
- rootDirstr
Root path where directories are to be created
- Returns
- ptDirsdict
A dictionary of all the created directories. Each dictionary entry has key defined by ‘setName’ and contains a list of size nMembers, each entry of which is the path to the created directory
Examples
>>> MP = multiPointSparse.multiPoint(MPI.COMM_WORLD) >>> MP.addProcessorSet('cruise', 3, 32) >>> MP.addProcessorSet('maneuver', 2, [10, 20]) >>> ptDirs = MP.createDirectories('/home/user/output/') >>> ptDirs {'cruise': ['/home/user/output/cruise_0', '/home/user/output/cruise_1', '/home/user/output/cruise_2'], 'maneuver': ['/home/user/output/maneuver_0', '/home/user/output/maneuver_1']}
- getSetName()[source]
After MP.createCommunicators is call, this routine may be called to return the name of the set that this processor belongs to. This may result in slightly cleaner script code.
- Returns
- setNamestr
The name of the set that this processor belongs to.
- obj(x)[source]
This is a built-in objective function that is designed to be used directly as an objective function with pyOptSparse. The user should not use this function directly, instead see the class documentation for the intended usage.
- Parameters
- xdict
Dictionary of variables returned from pyOptSparse
- sens(x, funcs)[source]
This is a built-in sensitivity function that is designed to be used directly as a the sensitivity function with pyOptSparse. The user should not use this function directly, instead see the class documentation for the intended usage.
- Parameters
- xdict
Dictionary of variables returned from pyOptSparse
- setObjCon(func)[source]
Set the python function handle to compute the final objective and constraints that are combinations of the functionals.
- Parameters
- funcPython function
Python function handle
- setOptProb(optProb)[source]
Set the optimization problem that this multiPoint object will be used for. This is required for this class to know how to assemble the gradients. If the optProb is not ‘finished’, it will done so here. Therefore, this function is collective on the comm that optProb is built on. multiPoint sparse does not hold a reference to optProb so no additional changes can be made to optProb after this function is called.
- Parameters
- optProbpyOptSparse optimization problem class
The optProb object to use