Use `cpptrace` for prettier backtraces from exceptions #3225

ZedThree · 2025-12-05T15:11:35Z

We have historically been reluctant to add too many external dependencies, but with CMake and spack making it easier to pull them in and build them, I thought I'd try this library for generating stack traces.

Two useful things:

it's very portable out of the box, so we don't need to rely on external utilities like addr2line and instead can always enable backtraces
we can print snippets:

#3  in checkData(double)
    at /home/peter/Codes/BOUT++/BOUT-dev/include/bout/utils.hxx:493:65
     491: inline void checkData(BoutReal f) {
     492:   if (!std::isfinite(f)) {
   > 493:     throw BoutException("BoutReal: Operation on non-finite data");
                                                                          ^
     494:   }
     495: }
#4  in operator*(Field3D const&, double)
    at /home/peter/Codes/BOUT++/BOUT-dev/src/field/generated_fieldops.cxx:480:12
     478:   Field3D result{emptyFrom(lhs)};
     479:   checkData(lhs);
   > 480:   checkData(rhs);
                     ^
     481: 
     482:   result.setRegion(lhs.getRegionID());
#5  in Blob2D::rhs(double)
    at /home/peter/Codes/BOUT++/BOUT-dev/examples/blob2d/blob2d.cxx:177:41
     176:     ddt(n) = -bracket(phi, n, BRACKET_ARAKAWA) // ExB term
   > 177:              + 2 * DDZ(n) * (rho_s / R_c)      // Curvature term
                                                  ^
     178:              + D_n * Delp2(n);                 // Diffusion term
     179:     if (compressible) {

and can even enable colour:

It also has its own signal handler we could perhaps use instead of ours (though I've not tested this out).

Lastly, this is maybe a complete enough solution that we could also remove MsgStack/TRACE/AUTO_TRACE?

ZedThree · 2025-12-17T14:21:57Z

Lots of CMake pain here

bendudson · 2025-12-29T18:19:30Z

Thanks @ZedThree ! This does look nice, if the CMake pain can be overcome. I think if it works portably then it could replace all the MsgStack / AUTOTRACE code.

Always generating the backtraces can slow the unit tests down by a factor x2-3 An alternative might be to store the `cpptrace:stacktrace` and only convert to string in `BoutException::what()`

- Add paths to `FetchContent` dirs in config file - Switch to custom versions of cpptrace/libdward-lite - Includes some cmake fixes in both projects

Required because cpptrace prevents in-source builds Also switch to using `subprocess.run` so we can better see errors

Seems to cause MPI problems?

ZedThree · 2026-01-06T11:43:27Z

I've removed AUTO_TRACE completely, and TRACE where it was just essentially repeating the function name (or something like intialising PETSc solver in PetscSolver::init).

That leaves uses like TRACE("PetscMatrix setting values at ({}, {})", petscRow, petscCol) which include function arguments or local variables that the stacktrace won't capture. I think they likely have some value for debugging, but quite limited?

Same goes for uses like TRACE("Creating Outer SOL communicators for Single Null operation") in BoutMesh. I've left them in for now, but I'm not sure how much more information they offer over the exception message + stacktrace.

Here's an example with CMAKE_BUILD_TYPE=RelWithDebInfo:

Error encountered: Stack trace (most recent call first):
#0 (filtered)
#1 (filtered)
#2 (filtered)
#3 in BoutMesh::createCommunicators()
   at /home/peter/Codes/BOUT++/BOUT-dev/src/mesh/impls/bout/boutmesh.cxx:709:64
     707:       }
     708:       // Unconditional exception for demo purposes
   > 709:       throw BoutException("Single null outer SOL not correct\n");      
                                                                         ^
     710:       MPI_Group_free(&group);
     711:     }
#4 in BoutMesh::load()
   at /home/peter/Codes/BOUT++/BOUT-dev/src/mesh/impls/bout/boutmesh.cxx:575:22
     573:   /// Communicator
     574: 
   > 575:   createCommunicators();
                               ^
     576:   output_debug << "Got communicators" << endl;
#5 in BoutInitialise(int&, char**&)
   at /home/peter/Codes/BOUT++/BOUT-dev/src/bout++.cxx:201:28
     199:   bout::globals::mesh = Mesh::create();
     200:   // Load from sources. Required for Field initialisation
   > 201:   bout::globals::mesh->load();
                                     ^
     202: 
     203:   // time_report options are used in BoutFinalise, i.e. after we
#6 in main
   at /home/peter/Codes/BOUT++/BOUT-dev/examples/elm-pb/elm_pb.cxx:2161:1
    2159: };
    2160: 
  > 2161: BOUTMAIN(ELMpb);
          ^
#7 (filtered)
#8 (filtered)
#9 (filtered)

====== Exception thrown ======
Single null outer SOL not correct


=== Additional information ===
 -> Creating Outer SOL communicators for Single Null operation on line 687 of '/home/peter/Codes/BOUT++/BOUT-dev/src/mesh/impls/bout/boutmesh.cxx'

And here's what it looks like with CMAKE_BUILD_TYPE=Release (no debug info):

Error encountered: Stack trace (most recent call first):
#0 (filtered)
#1 (filtered)
#2 (filtered)
#3 in BoutMesh::createCommunicators() [clone .cold]
   at /home/peter/Codes/BOUT++/BOUT-dev/build_release/lib/libbout++.so.5.2.0
#4 in BoutMesh::load()
   at /home/peter/Codes/BOUT++/BOUT-dev/build_release/lib/libbout++.so.5.2.0
#5 in BoutInitialise(int&, char**&)
   at /home/peter/Codes/BOUT++/BOUT-dev/build_release/lib/libbout++.so.5.2.0
#6 in main
   at /home/peter/Codes/BOUT++/BOUT-dev/examples/elm-pb/build/elm_pb
#7 (filtered)
#8 (filtered)
#9 (filtered)

====== Exception thrown ======
Single null outer SOL not correct


=== Additional information ===
 -> Creating Outer SOL communicators for Single Null operation on line 687 of '/home/peter/Codes/BOUT++/BOUT-dev/src/mesh/impls/bout/boutmesh.cxx'

Clearly the TRACE call is more useful when we don't have debug info, but if we recommend always building with it, it's probably not that useful?

@bendudson Thoughts on completely removing MsgStack? Maybe in a separate PR so we can more easily revert it if we change our minds?

CUDA tests are failing due to #3233

github-actions

clang-tidy made some suggestions

include/bout/deriv_store.hxx

include/bout/msg_stack.hxx

include/bout/scorepwrapper.hxx

src/bout++.cxx

github-actions · 2026-01-06T12:04:25Z

src/bout++.cxx

-    bout::globals::mesh = Mesh::create();
-    // Load from sources. Required for Field initialisation
-    bout::globals::mesh->load();
+  bout::globals::mpi = new MpiWrapper();


warning: assigning newly created 'gsl::owner<>' to non-owner 'MpiWrapper *' [cppcoreguidelines-owning-memory]

bout::globals::mpi = new MpiWrapper(); ^

src/solver/impls/euler/euler.cxx

src/sys/derivs.cxx

tests/integrated/test-backtrace/boutexcept.cxx

Technically `base.h` is sufficient, but this causes backwards compatibility issues, and the compilation penalty is minor

github-actions

clang-tidy made some suggestions

github-actions · 2026-01-06T15:13:31Z

src/sys/output.cxx


-#include <fmt/base.h>
+#include <fmt/format.h>



warning: included header format.h is not used directly [misc-include-cleaner]

Suggested change

ZedThree requested a review from bendudson December 5, 2025 15:11

ZedThree force-pushed the add-cpptrace branch from a0c128d to 19ae202 Compare December 5, 2025 15:37

ZedThree added 13 commits January 5, 2026 16:39

Switch to cpptrace for stack traces in exceptions

7d3b947

Add some control over BoutException backtraces

189ad96

Always generating the backtraces can slow the unit tests down by a factor x2-3 An alternative might be to store the `cpptrace:stacktrace` and only convert to string in `BoutException::what()`

Try to ensure build_defines is included where it's used

c09ad0a

Try to make backtrace unit test more portable

0965bbe

cmake: Fix some issues with building with cpptrace

408bb26

- Add paths to `FetchContent` dirs in config file - Switch to custom versions of cpptrace/libdward-lite - Includes some cmake fixes in both projects

Prettify symbols in cpptrace output

7facc49

Filter out pre-main functions from stacktrace

561fc8b

Include filtered placeholders

395d013

Remove AUTO_TRACE macro

52500e5

Remove uses of TRACE which are just the function name

767a767

Fix unnecessary includes of msg_stack.hxx

0d81ae5

Don't duplicate exception message when thrown from BoutInitialise

d0cd5f0

Don't print MsgStack dump if it's empty

86dd0c0

ZedThree force-pushed the add-cpptrace branch from a0d6528 to 86dd0c0 Compare January 5, 2026 16:42

ZedThree and others added 3 commits January 5, 2026 16:43

Apply clang-format changes

134c41f

docs: Use build directory for building library

c4ac956

Required because cpptrace prevents in-source builds Also switch to using `subprocess.run` so we can better see errors

Partial revert of d0cd5f0

f50303d

Seems to cause MPI problems?

github-actions bot reviewed Jan 6, 2026

View reviewed changes

ZedThree and others added 7 commits January 6, 2026 14:19

Remove some more unused headers

8fcd68b

tests: Add functions to anon namespace

da38a7c

Don't use char[] for constant

4493e33

Fix use of reserved identifier

662a7d3

Switch use of fmt header

230132a

Technically `base.h` is sufficient, but this causes backwards compatibility issues, and the compilation penalty is minor

docs: Update example backtrace

6e9a84e

Apply clang-format changes

93214da

github-actions bot reviewed Jan 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use `cpptrace` for prettier backtraces from exceptions #3225

Use `cpptrace` for prettier backtraces from exceptions #3225

Uh oh!

ZedThree commented Dec 5, 2025

Uh oh!

ZedThree commented Dec 17, 2025

Uh oh!

bendudson commented Dec 29, 2025

Uh oh!

ZedThree commented Jan 6, 2026 •

edited

Loading

Uh oh!

github-actions bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment

Uh oh!

github-actions bot Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		#include <fmt/base.h>
		#include <fmt/format.h>

Use cpptrace for prettier backtraces from exceptions #3225

Are you sure you want to change the base?

Use cpptrace for prettier backtraces from exceptions #3225

Uh oh!

Conversation

ZedThree commented Dec 5, 2025

Uh oh!

ZedThree commented Dec 17, 2025

Uh oh!

bendudson commented Dec 29, 2025

Uh oh!

ZedThree commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use `cpptrace` for prettier backtraces from exceptions #3225

Use `cpptrace` for prettier backtraces from exceptions #3225

ZedThree commented Jan 6, 2026 •

edited

Loading