Optimal Control with Non-Linear Expressions and Non-Matching Grids

Optimal Control with Non-Linear Expressions and Non-Matching Grids#

This demo minimizes the tracking functional:

\[ \min_{u \in Q} J(y, u) = \frac{1}{2} \int_{\Omega_s} (y - d)^2 ~dx + \frac{\alpha}{2} \int_{\Omega_c} u^2 ~dx \]

Subject to the state equation on a fine grid (\Omega_s) and a control on a coarse grid (\Omega_c):

\[ - \Delta y = I_{\Omega_c \to \Omega_s}(u + 0.1 u^3) \quad \text{in } \Omega_s \]

from mpi4py import MPI

import dolfinx
import moola
import numpy as np
import pyadjoint
import ufl
from dolfinx.mesh import create_unit_square
from moola.adaptors import DolfinxPrimalVector

import dolfinx_adjoint
from dolfinx_adjoint import interpolate, interpolate_nonmatching

1. Define Non-Matching Domains#

Coarse grid for the heater control

mesh_control = create_unit_square(MPI.COMM_WORLD, 8, 8)

Fine grid for the physical state

mesh_state = create_unit_square(MPI.COMM_WORLD, 32, 32)

Q = dolfinx.fem.functionspace(mesh_control, ("Lagrange", 1))
V = dolfinx.fem.functionspace(mesh_state, ("Lagrange", 1))

2. Setup Control Variable#

u_c = dolfinx_adjoint.Function(Q, name="Control")
u_c.interpolate(lambda x: np.sin(np.pi * x[0]) * np.sin(np.pi * x[1]))

3. Apply Non-Linear UFL Expression (`ExprInterpolationBlock`)#

The heater output scales non-linearly with the control input

f_expr = u_c + 0.1 * u_c**3

Interpolate the UFL expression onto the control space. PyAdjoint will tape this via ExprInterpolationBlock.

f_c = interpolate(f_expr, Q, ad_block_tag="nonlinear_heater_eval")

4. Cross-Mesh Transfer (NonmatchingInterpolationBlock)#

Transfer the evaluated heater power to the fine physics grid. PyAdjoint will tape this via NonmatchingInterpolationBlock.

f_s = interpolate_nonmatching(f_c, V, ad_block_tag="grid_transfer")

5. Solve the PDE#

y = ufl.TrialFunction(V)
v = ufl.TestFunction(V)

F = ufl.inner(ufl.grad(y), ufl.grad(v)) * ufl.dx - f_s * v * ufl.dx
a, L = ufl.system(F)

mesh_state.topology.create_connectivity(mesh_state.topology.dim - 1, mesh_state.topology.dim)
exterior_facets = dolfinx.mesh.exterior_facet_indices(mesh_state.topology)
exterior_dofs = dolfinx.fem.locate_dofs_topological(V, mesh_state.topology.dim - 1, exterior_facets)
zero = dolfinx.fem.Constant(mesh_state, 0.0)
bc = dolfinx.fem.dirichletbc(zero, exterior_dofs, V)

yh = dolfinx_adjoint.Function(V, name="State")
petsc_opts = {"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type": "mumps"}
problem = dolfinx_adjoint.LinearProblem(
    a,
    L,
    u=yh,
    bcs=[bc],
    petsc_options=petsc_opts,
    adjoint_petsc_options=petsc_opts,
    tlm_petsc_options=petsc_opts,
)
problem.solve()

Coefficient(FunctionSpace(Mesh(blocked element (Basix element (P, triangle, 1, gll_warped, unset, False, float64, []), (2,)), 1), Basix element (P, triangle, 1, gll_warped, unset, False, float64, [])), 7)

6. Define Objective and Optimize#

x = ufl.SpatialCoordinate(mesh_state)
d = ufl.sin(2 * ufl.pi * x[0]) * ufl.sin(2 * ufl.pi * x[1])

alpha = dolfinx.fem.Constant(mesh_control, 1e-4)

Note the integration domains: state tracking on \(\Omega_s\), regularization on \(\Omega_c\)

J = dolfinx_adjoint.assemble_scalar(0.5 * ufl.inner(yh - d, yh - d) * ufl.dx(domain=mesh_state))
J += dolfinx_adjoint.assemble_scalar(0.5 * alpha * ufl.inner(u_c, u_c) * ufl.dx(domain=mesh_control))

Configure PyAdjoint Reduced Functional

control = pyadjoint.Control(u_c)
Jhat = pyadjoint.ReducedFunctional(J, control)

7. Taylor Tests (Gradient and Hessian)#

print("\n--- Running Taylor Tests ---")

--- Running Taylor Tests ---

Define a distinct perturbation direction for the Taylor expansion

h = dolfinx_adjoint.Function(Q, name="Perturbation")
h.interpolate(lambda x: 10 * np.cos(np.pi * x[0]) * np.cos(np.pi * x[1]))

print("\n1st-order Taylor test (Testing Gradient exactness):")

1st-order Taylor test (Testing Gradient exactness):

Test Gradient (Expect convergence rate ~ 2.0)

rate_grad = pyadjoint.taylor_test(Jhat, u_c, h)

Running Taylor test
Computed residuals: [2.6518249078382166e-07, 7.658679566138185e-08, 2.043423691667763e-08, 5.269575985405984e-09]
Computed convergence rates: [1.7918179382273953, 1.9061073090990426, 1.95522958357414]

print("\n2nd-order Taylor test (Testing Hessian exactness):")

2nd-order Taylor test (Testing Hessian exactness):

To test the Hessian, we evaluate the action of the first and second derivatives in the direction of the perturbation ‘h’

Jhat(u_c)
dJdm = Jhat.derivative()._ad_dot(h)
dHddu = Jhat.hessian(h)._ad_dot(h)

Test Hessian (Expect convergence rate ~ 3.0)

rate_hess = pyadjoint.taylor_test(Jhat, u_c, h, dJdm=dJdm, Hm=dHddu)

Running Taylor test
Computed residuals: [8.237803093787657e-08, 1.030333476904271e-08, 1.28829569092851e-09, 1.6105716649555118e-10]
Computed convergence rates: [2.999148287722023, 2.999575689111309, 2.9998189980032]

print(f"\nGradient Taylor Test Rate: {rate_grad:.4f}")
print(f"Hessian Taylor Test Rate: {rate_hess:.4f}")

Gradient Taylor Test Rate: 1.7918
Hessian Taylor Test Rate: 2.9991

Solve with Moola

opt_problem = pyadjoint.MoolaOptimizationProblem(Jhat)
u_moola = DolfinxPrimalVector(u_c)
solver = moola.NewtonCG(opt_problem, u_moola, options={"gtol": 1e-6, "maxiter": 50, "display": 3})
sol = solver.solve()

Newton CG method.
------------------------------
Line search:		 strong_wolfe
Maximum iterations:	 50

iteration = 0:	objective = 0.12530785137277056:	grad_norm = 0.006920340749822488:

cg_iter = 0	curve = 8.949851051055955e-08	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = -1.220203517629922e-06	hesstol = 1.9984014443252818e-15
iteration = 1:	objective = 0.036836849072850814:	grad_norm = 0.12772377172685295:	delta_J = 0.08847100229991975:
cg_iter = 0	curve = 0.0061765284101735225	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = -0.000202735431800293	hesstol = 1.9984014443252818e-15
iteration = 2:	objective = 0.0163417060499955:	grad_norm = 0.045063729249475305:	delta_J = 0.020495143022855315:

cg_iter = 0	curve = 0.0001623589599315539	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 0.0010036125524496265	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = -9.558683428244108e-05	hesstol = 1.9984014443252818e-15
iteration = 3:	objective = 0.003144241578268771:	grad_norm = 0.016043164278480324:	delta_J = 0.013197464471726728:
cg_iter = 0	curve = 6.779222451464175e-05	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 0.0001034040303569515	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.5859663243826967e-08	hesstol = 1.9984014443252818e-15

cg_iter = 3	curve = -3.557782803638977e-07	hesstol = 1.9984014443252818e-15
iteration = 4:	objective = 0.0022867061785005888:	grad_norm = 0.005469877204785219:	delta_J = 0.0008575353997681821:
cg_iter = 0	curve = 5.539257236489324e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 2.0787949359339974e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.0145105092094996e-07	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = -1.2848517290816067e-08	hesstol = 1.9984014443252818e-15
iteration = 5:	objective = 0.001979089682625411:	grad_norm = 0.004007386306085271:	delta_J = 0.00030761649587517786:
cg_iter = 0	curve = 2.9489560008674372e-06	hesstol = 1.9984014443252818e-15

cg_iter = 1	curve = 1.1465726196342028e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.0634892764763627e-07	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 9.391562753290681e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.637034760982656e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = -1.4091281478681515e-10	hesstol = 1.9984014443252818e-15
iteration = 6:	objective = 0.0016902519765404824:	grad_norm = 0.005233083105454037:	delta_J = 0.0002888377060849285:
cg_iter = 0	curve = 1.4093539797396774e-05	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 2.1773983912353836e-06	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 4.979605069911655e-08	hesstol = 1.9984014443252818e-15

cg_iter = 3	curve = 4.98966630422424e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.049179166137473e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 5.182749956447353e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 4.0449712952653e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.2774457444879178e-06	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = -1.4557956312699431e-09	hesstol = 1.9984014443252818e-15
iteration = 7:	objective = 0.001648784257535986:	grad_norm = 0.003765661795610934:	delta_J = 4.146771900449646e-05:
cg_iter = 0	curve = 2.179765199274182e-06	hesstol = 1.9984014443252818e-15

cg_iter = 1	curve = 1.1443104320508226e-05	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 4.831929187652372e-08	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 6.176658173626298e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.2716721260486795e-07	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 2.246027820474205e-09	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 4.124123458187099e-08	hesstol = 1.9984014443252818e-15
iteration = 8:	objective = 0.0014409705715461062:	grad_norm = 0.0026698930903829615:	delta_J = 0.00020781368598987975:
cg_iter = 0	curve = 1.4374252436392956e-06	hesstol = 1.9984014443252818e-15

cg_iter = 1	curve = 4.1383365839072274e-06	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 9.363218780261744e-10	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 2.850322631102562e-08	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.0353257080615407e-10	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 3.335772188101006e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 3.2808783777974364e-08	hesstol = 1.9984014443252818e-15
iteration = 9:	objective = 0.001374729417924078:	grad_norm = 0.001008167320894899:	delta_J = 6.624115362202816e-05:
cg_iter = 0	curve = 4.08640328804731e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.1625746283452926e-07	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 5.137213177575513e-10	hesstol = 1.9984014443252818e-15

cg_iter = 3	curve = 4.8425778434975574e-08	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.4600190360298986e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 2.2175687141576512e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.2364017766253216e-11	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.84930112394761e-08	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.877929560678223e-10	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 7.899940638877803e-11	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 6.5976435876445036e-09	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 3.7750282557390324e-11	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.1102860801868022e-09	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 6.024434052305782e-10	hesstol = 1.9984014443252818e-15

cg_iter = 14	curve = 3.4206864997414954e-10	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 5.578295482101599e-11	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = -1.4511847999676949e-10	hesstol = 1.9984014443252818e-15
iteration = 10:	objective = 0.0013639416468899659:	grad_norm = 0.002665908230090469:	delta_J = 1.0787771034112157e-05:
cg_iter = 0	curve = 5.048169857193829e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 4.316176892396858e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.0156913520427794e-10	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 3.7370631798198264e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 2.176296939479243e-10	hesstol = 1.9984014443252818e-15

cg_iter = 5	curve = 7.603500342379936e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.0628768756620292e-08	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.3877942824672636e-10	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 4.275359125148142e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = -1.7327619488954308e-07	hesstol = 1.9984014443252818e-15
iteration = 11:	objective = 0.0013569107842507173:	grad_norm = 0.0016708137445950407:	delta_J = 7.030862639248545e-06:
cg_iter = 0	curve = 1.6776307346349725e-06	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 7.056255090532736e-08	hesstol = 1.9984014443252818e-15

cg_iter = 2	curve = 4.461842716020023e-10	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 4.237910806071533e-09	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.5870575276698145e-09	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 7.450097199731451e-11	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.8434116259092306e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 6.546234368451102e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.219236613759942e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.2861392923505213e-10	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 9.074602886154217e-10	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 6.05625704729028e-09	hesstol = 1.9984014443252818e-15

cg_iter = 12	curve = 4.0717157191177044e-10	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 1.2696800868520658e-10	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = -4.1371664894015335e-10	hesstol = 1.9984014443252818e-15
iteration = 12:	objective = 0.0013480419560962195:	grad_norm = 0.0008813549362846434:	delta_J = 8.86882815449787e-06:
cg_iter = 0	curve = 1.438738462555493e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 4.5357092100892917e-07	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 4.537319864039564e-10	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 1.1925819348704632e-08	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 3.888612563615652e-10	hesstol = 1.9984014443252818e-15

cg_iter = 5	curve = 1.0375943214933026e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.9402502560163037e-09	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 3.3832482254371835e-11	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.0664948486150536e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 6.693664678856267e-11	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 4.440519094943409e-12	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 2.5292744142990598e-08	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 6.896959623103561e-11	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 1.700910850310938e-11	hesstol = 1.9984014443252818e-15

cg_iter = 14	curve = 1.412725235394926e-12	hesstol = 1.9984014443252818e-15
iteration = 13:	objective = 0.001337932853957597:	grad_norm = 0.0002662200480285135:	delta_J = 1.0109102138622546e-05:
cg_iter = 0	curve = 5.8954721543289356e-09	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.414726496928356e-08	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.4593597555481603e-10	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 3.4031386785000064e-10	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 4.1075391702277966e-11	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 2.519874886013855e-10	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 9.721445076054878e-12	hesstol = 1.9984014443252818e-15

cg_iter = 7	curve = 1.3805802212997653e-12	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 2.0397562058795452e-11	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 9.686053770272503e-13	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 2.1786317007261256e-11	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 8.922679565650103e-09	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.4090848096220333e-12	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 2.019841437050853e-12	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 3.4230410339596602e-12	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 3.309590232948243e-13	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 1.5636555785162937e-13	hesstol = 1.9984014443252818e-15

cg_iter = 17	curve = 9.264141864310123e-11	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 4.294470773020894e-13	hesstol = 1.9984014443252818e-15
cg_iter = 19	curve = 8.501641829113469e-11	hesstol = 1.9984014443252818e-15
cg_iter = 20	curve = 4.058775959372845e-11	hesstol = 1.9984014443252818e-15
iteration = 14:	objective = 0.0013368239602777182:	grad_norm = 0.0008729413446256229:	delta_J = 1.108893679878679e-06:
cg_iter = 0	curve = 5.679492361719343e-07	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.1396626501846956e-09	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 1.725435613053876e-10	hesstol = 1.9984014443252818e-15
iteration = 15:	objective = 0.001336242186866009:	grad_norm = 1.2361797380865928e-05:	delta_J = 5.817734117091744e-07:

cg_iter = 0	curve = 3.873508582212294e-12	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 3.930473221550089e-10	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 9.541687214539641e-11	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 2.4452800486505073e-11	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 6.6069190775298525e-12	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 3.465935283217203e-12	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 5.188250089193244e-11	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 2.335262221098155e-12	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 1.877168153739372e-12	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 1.8371767905835446e-12	hesstol = 1.9984014443252818e-15

cg_iter = 10	curve = 1.2924217111886537e-13	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 7.687964267232537e-13	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 1.5589813481692522e-10	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 2.1291603359378037e-13	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 1.5445182510546634e-13	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 1.0255049327224329e-13	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 7.478082946061314e-14	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 1.7071036126408672e-13	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 2.9487709481177613e-12	hesstol = 1.9984014443252818e-15

cg_iter = 19	curve = 2.5236287209373745e-14	hesstol = 1.9984014443252818e-15
cg_iter = 20	curve = 3.47535139413394e-12	hesstol = 1.9984014443252818e-15
cg_iter = 21	curve = 3.1120446014658374e-11	hesstol = 1.9984014443252818e-15
cg_iter = 22	curve = 5.535679689373126e-14	hesstol = 1.9984014443252818e-15
cg_iter = 23	curve = 8.728170973715312e-13	hesstol = 1.9984014443252818e-15
cg_iter = 24	curve = 1.2776139237010526e-14	hesstol = 1.9984014443252818e-15
cg_iter = 25	curve = 1.881922276269753e-15	hesstol = 1.9984014443252818e-15

iteration = 16:	objective = 0.0013361270551927274:	grad_norm = 6.762365613376619e-05:	delta_J = 1.1513167328162065e-07:
cg_iter = 0	curve = 3.3313440298287556e-09	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.3990212160000812e-11	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 3.653988667762035e-12	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 9.968300556457e-13	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.019028024498697e-12	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 5.7600075780516704e-12	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.1034349090138843e-12	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 5.940963224338722e-13	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 6.12281800578522e-13	hesstol = 1.9984014443252818e-15

cg_iter = 9	curve = 3.559118156563441e-13	hesstol = 1.9984014443252818e-15
cg_iter = 10	curve = 6.217241539655221e-13	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 2.4898333866056263e-11	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 6.03574814818031e-13	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 1.2479207089329795e-13	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 4.023004966533745e-15	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 2.0167187860709346e-14	hesstol = 1.9984014443252818e-15
cg_iter = 16	curve = 2.942264483436885e-14	hesstol = 1.9984014443252818e-15
cg_iter = 17	curve = 1.1316258018415206e-12	hesstol = 1.9984014443252818e-15
cg_iter = 18	curve = 1.2150572703310455e-14	hesstol = 1.9984014443252818e-15

cg_iter = 19	curve = 1.3660226114050377e-10	hesstol = 1.9984014443252818e-15
cg_iter = 20	curve = 1.8990288437510416e-14	hesstol = 1.9984014443252818e-15
cg_iter = 21	curve = 1.0847366971716271e-13	hesstol = 1.9984014443252818e-15
cg_iter = 22	curve = 7.719975316656807e-13	hesstol = 1.9984014443252818e-15
cg_iter = 23	curve = 2.976011592669903e-15	hesstol = 1.9984014443252818e-15
cg_iter = 24	curve = 2.39033880626859e-15	hesstol = 1.9984014443252818e-15
cg_iter = 25	curve = 1.806460814053973e-16	hesstol = 1.9984014443252818e-15
iteration = 17:	objective = 0.0013360780209011845:	grad_norm = 2.3266911735285286e-05:	delta_J = 4.90342915429251e-08:

cg_iter = 0	curve = 3.154550464335823e-10	hesstol = 1.9984014443252818e-15
cg_iter = 1	curve = 1.5460345237484797e-11	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 8.497788868991558e-13	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 4.451098474522833e-15	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 1.1348391191740335e-13	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.2083218070601023e-12	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 1.6268504675137898e-14	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 1.8269445057096588e-14	hesstol = 1.9984014443252818e-15
cg_iter = 8	curve = 8.606089113288199e-15	hesstol = 1.9984014443252818e-15
cg_iter = 9	curve = 2.2306259426476466e-14	hesstol = 1.9984014443252818e-15

cg_iter = 10	curve = 6.153707340494532e-12	hesstol = 1.9984014443252818e-15
cg_iter = 11	curve = 6.554483676095491e-14	hesstol = 1.9984014443252818e-15
cg_iter = 12	curve = 2.7369658734935285e-14	hesstol = 1.9984014443252818e-15
cg_iter = 13	curve = 3.2192590204989016e-15	hesstol = 1.9984014443252818e-15
cg_iter = 14	curve = 5.336445680390039e-15	hesstol = 1.9984014443252818e-15
cg_iter = 15	curve = 6.826016077480001e-16	hesstol = 1.9984014443252818e-15
iteration = 18:	objective = 0.001336074482191157:	grad_norm = 3.084275117277937e-06:	delta_J = 3.5387100274803657e-09:
cg_iter = 0	curve = 6.280720476403652e-12	hesstol = 1.9984014443252818e-15

cg_iter = 1	curve = 1.1959837541750067e-13	hesstol = 1.9984014443252818e-15
cg_iter = 2	curve = 2.4244233127382496e-14	hesstol = 1.9984014443252818e-15
cg_iter = 3	curve = 6.6170549175016345e-15	hesstol = 1.9984014443252818e-15
cg_iter = 4	curve = 2.5926140461689682e-15	hesstol = 1.9984014443252818e-15
cg_iter = 5	curve = 1.6236549143051685e-14	hesstol = 1.9984014443252818e-15
cg_iter = 6	curve = 5.3765979165048455e-15	hesstol = 1.9984014443252818e-15
cg_iter = 7	curve = 4.1865916507478937e-16	hesstol = 1.9984014443252818e-15

Tolerance reached: grad_norm < gtol.

iteration = 19:	objective = 0.0013360744165099665:	grad_norm = 3.775780799157714e-07:	delta_J = 6.568119052113852e-11:

print(f"Optimization completed in {sol['iteration']} iterations.")

Optimization completed in 19 iterations.