Optimal Control with Non-Linear Expressions and Non-Matching Grids

Optimal Control with Non-Linear Expressions and Non-Matching Grids#

This demo minimizes the tracking functional:

\[ \min_{u \in Q} J(y, u) = \frac{1}{2} \int_{\Omega_s} (y - d)^2 ~dx + \frac{\alpha}{2} \int_{\Omega_c} u^2 ~dx \]

Subject to the state equation on a fine grid (\Omega_s) and a control on a coarse grid (\Omega_c):

\[ - \Delta y = I_{\Omega_c \to \Omega_s}(u + 0.1 u^3) \quad \text{in } \Omega_s \]

from mpi4py import MPI

import dolfinx
import moola
import numpy as np
import pyadjoint
import ufl
from dolfinx.mesh import create_unit_square
from moola.adaptors import DolfinxPrimalVector

import dolfinx_adjoint
from dolfinx_adjoint import interpolate, interpolate_nonmatching

1. Define Non-Matching Domains#

Coarse grid for the heater control

mesh_control = create_unit_square(MPI.COMM_WORLD, 8, 8)

Fine grid for the physical state

mesh_state = create_unit_square(MPI.COMM_WORLD, 32, 32)

Q = dolfinx.fem.functionspace(mesh_control, ("Lagrange", 1))
V = dolfinx.fem.functionspace(mesh_state, ("Lagrange", 1))

2. Setup Control Variable#

u_c = dolfinx_adjoint.Function(Q, name="Control")
u_c.interpolate(lambda x: np.sin(np.pi * x[0]) * np.sin(np.pi * x[1]))

3. Apply Non-Linear UFL Expression (`ExprInterpolationBlock`)#

The heater output scales non-linearly with the control input

f_expr = u_c + 0.1 * u_c**3

Interpolate the UFL expression onto the control space. PyAdjoint will tape this via ExprInterpolationBlock.

f_c = interpolate(f_expr, Q, ad_block_tag="nonlinear_heater_eval")

4. Cross-Mesh Transfer (NonmatchingInterpolationBlock)#

Transfer the evaluated heater power to the fine physics grid. PyAdjoint will tape this via NonmatchingInterpolationBlock.

f_s = interpolate_nonmatching(f_c, V, ad_block_tag="grid_transfer")

5. Solve the PDE#

y = ufl.TrialFunction(V)
v = ufl.TestFunction(V)

F = ufl.inner(ufl.grad(y), ufl.grad(v)) * ufl.dx - f_s * v * ufl.dx
a, L = ufl.system(F)

mesh_state.topology.create_connectivity(mesh_state.topology.dim - 1, mesh_state.topology.dim)
exterior_facets = dolfinx.mesh.exterior_facet_indices(mesh_state.topology)
exterior_dofs = dolfinx.fem.locate_dofs_topological(V, mesh_state.topology.dim - 1, exterior_facets)
zero = dolfinx.fem.Constant(mesh_state, 0.0)
bc = dolfinx.fem.dirichletbc(zero, exterior_dofs, V)

yh = dolfinx_adjoint.Function(V, name="State")
petsc_opts = {"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type": "mumps"}
problem = dolfinx_adjoint.LinearProblem(
    a,
    L,
    u=yh,
    bcs=[bc],
    petsc_options=petsc_opts,
    adjoint_petsc_options=petsc_opts,
    tlm_petsc_options=petsc_opts,
)
problem.solve()

Coefficient(FunctionSpace(Mesh(blocked element (Basix element (P, triangle, 1, gll_warped, unset, False, float64, []), (2,)), 1), Basix element (P, triangle, 1, gll_warped, unset, False, float64, [])), 7)

6. Define Objective and Optimize#

x = ufl.SpatialCoordinate(mesh_state)
d = ufl.sin(2 * ufl.pi * x[0]) * ufl.sin(2 * ufl.pi * x[1])

alpha = dolfinx.fem.Constant(mesh_control, 1e-4)

Note the integration domains: state tracking on \(\Omega_s\), regularization on \(\Omega_c\)

J = dolfinx_adjoint.assemble_scalar(0.5 * ufl.inner(yh - d, yh - d) * ufl.dx(domain=mesh_state))
J += dolfinx_adjoint.assemble_scalar(0.5 * alpha * ufl.inner(u_c, u_c) * ufl.dx(domain=mesh_control))

Configure PyAdjoint Reduced Functional

control = pyadjoint.Control(u_c)
Jhat = pyadjoint.ReducedFunctional(J, control)

7. Taylor Tests (Gradient and Hessian)#

print("\n--- Running Taylor Tests ---")

--- Running Taylor Tests ---

Define a distinct perturbation direction for the Taylor expansion

h = dolfinx_adjoint.Function(Q, name="Perturbation")
h.interpolate(lambda x: 10 * np.cos(np.pi * x[0]) * np.cos(np.pi * x[1]))

print("\n1st-order Taylor test (Testing Gradient exactness):")

1st-order Taylor test (Testing Gradient exactness):

Test Gradient (Expect convergence rate ~ 2.0)

rate_grad = pyadjoint.taylor_test(Jhat, u_c, h)

Running Taylor test
Computed residuals: [2.6518249078379456e-07, 7.658679563361272e-08, 2.043423694442643e-08, 5.2695759854025955e-09]
Computed convergence rates: [1.7918179387503455, 1.906107306616828, 1.9552295855341846]

print("\n2nd-order Taylor test (Testing Hessian exactness):")

2nd-order Taylor test (Testing Hessian exactness):

To test the Hessian, we evaluate the action of the first and second derivatives in the direction of the perturbation ‘h’

Jhat(u_c)
dJdm = Jhat.derivative()._ad_dot(h)
dHddu = Jhat.hessian(h)._ad_dot(h)

Test Hessian (Expect convergence rate ~ 3.0)

rate_hess = pyadjoint.taylor_test(Jhat, u_c, h, dJdm=dJdm, Hm=dHddu)

Running Taylor test
Computed residuals: [8.237803093790352e-08, 1.0303334796811798e-08, 1.2882956631797008e-09, 1.6105716649893683e-10]
Computed convergence rates: [2.9991482838342076, 2.9995757240740386, 2.99981896689843]

print(f"\nGradient Taylor Test Rate: {rate_grad:.4f}")
print(f"Hessian Taylor Test Rate: {rate_hess:.4f}")

Gradient Taylor Test Rate: 1.7918
Hessian Taylor Test Rate: 2.9991

Solve with Moola

opt_problem = pyadjoint.MoolaOptimizationProblem(Jhat)
u_moola = DolfinxPrimalVector(u_c)
solver = moola.NewtonCG(opt_problem, u_moola, options={"gtol": 1e-6, "maxiter": 50, "display": 3})
sol = solver.solve()

Newton CG method.
------------------------------
Line search:		 strong_wolfe
Maximum iterations:	 50

iteration = 0:	objective = 0.12530785137277056:	grad_norm = 0.006920340749822487:

cg_iter = 0	curve = 8.949851051055951e-08	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = -1.2202035176299235e-06	hesstol = 1.9984014443252818e-15
iteration = 1:	objective = 0.03683684907285076:	grad_norm = 0.12772377172685273:	delta_J = 0.0884710022999198:
cg_iter = 0	curve = 0.006176528410173494	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = -0.0002027354318002919	hesstol = 1.9984014443252818e-15
iteration = 2:	objective = 0.016341706049995474:	grad_norm = 0.04506372924947533:	delta_J = 0.020495143022855284:
cg_iter = 0	curve = 0.00016235895993155572	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 0.0010036125524496128	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = -9.558683428244107e-05	hesstol = 1.9984014443252818e-15
iteration = 3:	objective = 0.0031442415782687717:	grad_norm = 0.016043164278480276:	delta_J = 0.013197464471726702:
cg_iter = 0	curve = 6.779222451464023e-05	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 0.00010340403035695344	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.5859663243827268e-08	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = -3.5577828036387464e-07	hesstol = 1.9984014443252818e-15

iteration = 4:	objective = 0.0022867061785005905:	grad_norm = 0.005469877204784705:	delta_J = 0.0008575353997681812:
cg_iter = 0	curve = 5.539257236483767e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 2.078794935935802e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.0145105092097492e-07	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = -1.284851729081298e-08	hesstol = 1.9984014443252818e-15
iteration = 5:	objective = 0.001979089682625403:	grad_norm = 0.004007386306085043:	delta_J = 0.0003076164958751874:
cg_iter = 0	curve = 2.948956000866032e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.146572619634729e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.0634892764760525e-07	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 9.391562753290822e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.637034760981125e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = -1.4091281478924734e-10	hesstol = 1.9984014443252818e-15
iteration = 6:	objective = 0.0016902519765306886:	grad_norm = 0.005233083105052242:	delta_J = 0.0002888377060947145:
cg_iter = 0	curve = 1.4093539793639737e-05	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 2.177398391742053e-06	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 4.9796050695226625e-08	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 4.989666304454505e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.0491791532840105e-09	hesstol = 1.9984014443252818e-15

cg_iter = 5	curve = 5.181606173979508e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.911267814324969e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 3.2721072581638313e-06	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 2.3118196175407995e-08	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = -1.5208622713050501e-05	hesstol = 1.9984014443252818e-15
iteration = 7:	objective = 0.001651855088716699:	grad_norm = 0.004053770064375925:	delta_J = 3.8396887813989646e-05:
cg_iter = 0	curve = 2.645238745833172e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.2771910565962238e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 6.249064099480026e-08	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 4.786132737080721e-07	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.2737432706533153e-08	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 6.625068544202024e-07	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 2.7400580985968046e-10	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 8.673099705303039e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = -3.4654798116951798e-09	hesstol = 1.9984014443252818e-15
iteration = 8:	objective = 0.001517585565763206:	grad_norm = 0.004163252158485779:	delta_J = 0.00013426952295349302:

cg_iter = 0	curve = 7.898819840968867e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.7342168528440023e-06	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.9622426992676946e-08	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 1.9974717205388256e-07	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 3.550283709594307e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 2.89775416894526e-08	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 3.1772819200003804e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.3812536231235671e-09	hesstol = 1.9984014443252818e-15
iteration = 9:	objective = 0.001454058432878941:	grad_norm = 0.002349857258669414:	delta_J = 6.352713288426486e-05:
cg_iter = 0	curve = 8.067552712935737e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 4.346431281129966e-06	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 9.658326988693956e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 1.1561312522094258e-07	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 3.2155825092812524e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 3.1673133385123304e-08	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 4.634513754979154e-10	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 2.7219562992682985e-10	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 4.111681115196655e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 5.439781521022339e-11	hesstol = 1.9984014443252818e-15

cg_iter = 10	curve = 7.610731323488645e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 1.3762466971095028e-07	hesstol = 1.9984014443252818e-15
iteration = 10:	objective = 0.0013838736986273436:	grad_norm = 0.004804809700544984:	delta_J = 7.01847342515975e-05:
cg_iter = 0	curve = 1.5991786614727236e-05	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.8575833091635473e-07	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.2422876783317987e-08	hesstol = 1.9984014443252818e-15
iteration = 11:	objective = 0.001357001407719976:	grad_norm = 0.0003140009155684028:	delta_J = 2.6872290907367473e-05:
cg_iter = 0	curve = 1.0206264271471062e-08	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 6.405319530807019e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.790072446218268e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 7.758253935470882e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 2.221974846176492e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 7.535641350826022e-09	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 3.3638721933748094e-10	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 7.029336244372428e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.3984572854666503e-10	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.5685790497550544e-10	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 1.1683172211621368e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 1.261121835177222e-08	hesstol = 1.9984014443252818e-15

cg_iter = 12	curve = 2.234048798857149e-10	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 6.315281967885979e-11	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 3.546982728320128e-11	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = -2.4923577456192293e-10	hesstol = 1.9984014443252818e-15
iteration = 12:	objective = 0.001350770146567795:	grad_norm = 0.0008033506792010082:	delta_J = 6.231261152181032e-06:
cg_iter = 0	curve = 4.399627075578802e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.6329397227869054e-09	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 4.3437610211434915e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 1.7154745081595794e-08	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.0999615740323493e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.1466440700645996e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.0646041388096131e-08	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 3.227613347609897e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 6.840294284668328e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 2.7849596986082363e-11	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 6.655658545299126e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 1.8844109863248703e-08	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 3.149091962170225e-10	hesstol = 1.9984014443252818e-15

cg_iter = 13	curve = 1.416918100828168e-10	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = -4.788329134888193e-09	hesstol = 1.9984014443252818e-15
iteration = 13:	objective = 0.0013460716628949303:	grad_norm = 0.0003464081968039303:	delta_J = 4.698483672864809e-06:
cg_iter = 0	curve = 2.566516959429673e-08	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 4.2403343879937444e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.2897792110742387e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 4.9658746343544745e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 5.233383506085363e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.2778885101667084e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 3.820052416457126e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 4.705748665385893e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 9.672977672625046e-12	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.3140149491537761e-11	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 6.817729336317066e-11	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 3.464529639608435e-08	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 2.425439606650316e-11	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 6.269014506091583e-12	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 1.7074615904018338e-11	hesstol = 1.9984014443252818e-15

cg_iter = 15	curve = 4.3467948643362356e-13	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 2.5440392393602364e-12	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 1.2952302989787965e-11	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 4.385541210672642e-11	hesstol = 1.9984014443252818e-15
cg_iter = 19	curve = 1.190852617783572e-09	hesstol = 1.9984014443252818e-15
cg_iter = 20	curve = 5.718496121948874e-11	hesstol = 1.9984014443252818e-15
cg_iter = 21	curve = 1.1744526882559096e-12	hesstol = 1.9984014443252818e-15
cg_iter = 22	curve = 1.4419944372171955e-11	hesstol = 1.9984014443252818e-15
cg_iter = 23	curve = 2.858499425482978e-12	hesstol = 1.9984014443252818e-15
cg_iter = 24	curve = 9.541957526800059e-14	hesstol = 1.9984014443252818e-15
cg_iter = 25	curve = 1.0902393608225945e-13	hesstol = 1.9984014443252818e-15
cg_iter = 26	curve = 6.193223974196409e-14	hesstol = 1.9984014443252818e-15
cg_iter = 27	curve = 2.564861816565974e-11	hesstol = 1.9984014443252818e-15
cg_iter = 28	curve = 5.533324905865259e-13	hesstol = 1.9984014443252818e-15
iteration = 14:	objective = 0.0013404172910725766:	grad_norm = 0.0006845882903743535:	delta_J = 5.6543718223536245e-06:
cg_iter = 0	curve = 2.8501179583825634e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.0061962060420341e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.5297991328028836e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 3.4384976214997137e-10	hesstol = 1.9984014443252818e-15

cg_iter = 4	curve = 2.1313320910091808e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 2.616159050435563e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 4.3836924987622237e-10	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.3966142330308239e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 3.0612234083328734e-12	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.56271483634613e-12	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 4.945258846338954e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 4.566776195251695e-10	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.2148260271716779e-11	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 4.543206570496489e-12	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 4.068049748846112e-12	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 7.631737710326722e-13	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 3.52902149477506e-12	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 9.50206631435427e-10	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 5.371053310844426e-12	hesstol = 1.9984014443252818e-15
cg_iter = 19	curve = 2.7336157419389114e-07	hesstol = 1.9984014443252818e-15
cg_iter = 20	curve = 6.664197040210879e-10	hesstol = 1.9984014443252818e-15
cg_iter = 21	curve = -1.0485729667838706e-10	hesstol = 1.9984014443252818e-15

iteration = 15:	objective = 0.0013396507040132405:	grad_norm = 0.0005055888914302357:	delta_J = 7.66587059336174e-07:
cg_iter = 0	curve = 1.234260653479506e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.5376001700925526e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.387785686468601e-09	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 2.833161559704357e-10	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.9068449803666249e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 3.6312144662486244e-11	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.4601732410580538e-10	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 4.08357910833584e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.704009528081132e-12	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 4.2739918935193654e-13	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 4.3790449577158407e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 8.65551244299244e-11	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 8.048623096117423e-12	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 4.0440538620121966e-13	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 5.670535350637579e-12	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 3.627630303035189e-13	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 5.811876813335218e-13	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 1.6331175562582495e-11	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 1.1283423047992035e-12	hesstol = 1.9984014443252818e-15
cg_iter = 19	curve = 4.4512008684296784e-10	hesstol = 1.9984014443252818e-15

iteration = 16:	objective = 0.001336303973402168:	grad_norm = 7.817650918242465e-05:	delta_J = 3.3467306110725008e-06:
cg_iter = 0	curve = 5.115358220573865e-10	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.2761496873758252e-09	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.914643098121386e-11	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 9.61382358104916e-12	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 6.82765269322832e-12	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 3.770330077629229e-11	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 2.53163252250275e-12	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.4620928439100327e-12	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 7.908826566583915e-13	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 8.501961556194683e-13	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 2.5833275392786966e-13	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 6.199764376842673e-10	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.931705716856616e-12	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 5.293771490705285e-13	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 1.0077991168560494e-13	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 6.720591320795303e-14	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 2.9123708988840415e-14	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 5.010546973892035e-13	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 1.8819810077328445e-14	hesstol = 1.9984014443252818e-15
cg_iter = 19	curve = 1.7998058849420423e-11	hesstol = 1.9984014443252818e-15

cg_iter = 20	curve = 4.1656387519000634e-12	hesstol = 1.9984014443252818e-15
cg_iter = 21	curve = 3.957293499846796e-13	hesstol = 1.9984014443252818e-15
cg_iter = 22	curve = 8.007750188644594e-14	hesstol = 1.9984014443252818e-15
cg_iter = 23	curve = 2.5345191898416328e-15	hesstol = 1.9984014443252818e-15
cg_iter = 24	curve = 5.6130094804547635e-15	hesstol = 1.9984014443252818e-15
cg_iter = 25	curve = 1.2718629416031409e-15	hesstol = 1.9984014443252818e-15
iteration = 17:	objective = 0.0013361354053213264:	grad_norm = 0.000246972799069547:	delta_J = 1.6856808084152533e-07:
cg_iter = 0	curve = 4.483547670183563e-08	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.9080163771403705e-10	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.067609390779685e-11	hesstol = 1.9984014443252818e-15
iteration = 18:	objective = 0.0013360864849532444:	grad_norm = 3.118612747558726e-06:	delta_J = 4.892036808198247e-08:
cg_iter = 0	curve = 1.7760933607672237e-13	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 9.133262240025707e-12	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 6.465462469445103e-12	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 6.708130656950374e-13	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.687164201189123e-13	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.0314869290130043e-13	hesstol = 1.9984014443252818e-15

cg_iter = 6	curve = 1.2419323153338307e-11	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.9280986736571528e-13	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 3.986059158466195e-14	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.8939686060124917e-13	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 4.940549629797446e-14	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 1.4158127529807355e-13	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 5.761145325297017e-11	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 3.835035470113115e-14	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 2.495610782829814e-15	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 1.8316750108152785e-14	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 1.1452300638555583e-15	hesstol = 1.9984014443252818e-15
iteration = 19:	objective = 0.0013360763059096406:	grad_norm = 3.502244885663984e-05:	delta_J = 1.0179043603885177e-08:
cg_iter = 0	curve = 9.030196486919899e-10	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 3.852287031522687e-12	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.530632946982322e-13	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 7.170754871915932e-15	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.905890506620454e-14	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.2816882932793746e-13	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 9.209190894124036e-14	hesstol = 1.9984014443252818e-15

cg_iter = 7	curve = 3.2564251375647667e-15	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 5.981144162561386e-15	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 2.130851757696437e-15	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 9.581238095023931e-13	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 4.533024281366428e-15	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.4763947049773808e-14	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 1.6078082616144057e-15	hesstol = 1.9984014443252818e-15

Tolerance reached: grad_norm < gtol.

iteration = 20:	objective = 0.0013360745504405116:	grad_norm = 7.686173493115221e-07:	delta_J = 1.755469129002124e-09:

print(f"Optimization completed in {sol['iteration']} iterations.")

Optimization completed in 20 iterations.