ECCE modernization — status (2026-08-26, updated) Standing preference
Andy runs Debian (currently Debian 13 "trixie") — default to Debian, not Ubuntu, for all future environment/package decisions on this and other projects, unless he says otherwise.
Note on this session: the actual build/compile work below was done in an Ubuntu 24.04 sandbox because that's what this cloud session runs on, and the sandbox's network egress only allows Ubuntu's own package mirrors (deb.debian.org is blocked — confirmed directly, debootstrap against it returns 403). Update this session: Andy has since built this (and the .deb) on his own real Debian 13 machine and confirmed both succeed — the Ubuntu-vs- Debian package-name gap noted below is resolved for the C++/wx build. (The JMS gateway work added this session, see below, hasn't specifically been re-verified there yet.)
So none of this had been verified on real Debian 13 as of the previous session. What's been checked instead: the specific package names used below were cross-checked against Debian's own package database and match trixie exactly for the libraries this build depends on — libwxgtk3.2-dev (trixie), libxerces-c3.2/libxerces-c-dev (sid, same 3.2.x line as Ubuntu 22.04/24.04), libgl1-mesa-dev and libglu1-mesa-dev (trixie). So the package list should carry over unchanged, but a real build on Andy's own Debian 13 machine (or a session with unrestricted network access) is the only way to be sure — GCC/glibc version differences between the two distros could still surface new errors the Ubuntu run didn't hit.
Decision
Porting the existing codebase to build on modern toolchains/libraries, not a rewrite. Rationale: a port keeps the old code's behavior as the correctness anchor at every step; a rewrite of domain logic (chemistry/unit conversions/symmetry handling etc.) carries much higher fidelity risk and would need heavy review from Andy to catch subtly-wrong reimplementations, which he does not want to invest time in.
The one deliberate exception so far: ewxGenericFileDialog (see below) had to be substantially rebuilt, not mechanically ported, because its wx2.8 base class was removed from wxWidgets entirely. Andy was asked and chose to have it rebuilt properly rather than stubbed or skipped.
Progress this session: the JMS gateway now actually works, end to end
Previously the .deb deliberately shipped no messaging at all (see "Open question" below, as it stood) -- every ecce- wrapper launched its binary standalone, no broker, no JMSDispatcher, no shared preferences/cache env vars even set. This session closed that gap for the single-machine (client and server on the same box) case, which Andy confirmed is how this is actually run -- the old -remote flow for a separately-hosted server is deliberately still not ported.
Fixed GitHub issue #5 ("ECCE crashes due to too many open files" / java.net.SocketException: Too many open files, filed by Andy in 2017, never fixed). Root cause: MessageHandler.java (java/src/gov/pnnl/emsl/ecce/jms/) opens a DatagramSocket in its constructor on every single JMS subscribe, and nothing ever closed it -- not JMSDispatcher.unsubscribe(), not quit(). Since JMSDispatcher is one long-lived process per login/session, every resubscribe (including every retry eccejobmaster's supervisor loop does around eccejobstore) leaked one fd forever, eventually hitting the process's fd ulimit and throwing exactly the reported SocketException. Fixed: added MessageHandler.close(), wired into unsubscribe() for both the server-subscription path (new p_serverSubscriberHandlers tracking map -- subscribeServer()'s handlers weren't tracked anywhere before) and the local-subscription path (p_localSubscribers, including the resubscribe-without-unsubscribe case). Verified under load: scripted 200 raw UDP subscribe/unsubscribe cycles against a running JMSDispatcher and watched its /proc//fd count -- flat before/after (would have grown by 200 before the fix). The gateway is now built, packaged, and started automatically. java/ (the JMSDispatcher relay) is now built by the main CMake build via the existing java/build/build.xml Ant script (find_program(ant), a custom command producing java/lib/ ecce_jms.jar), installed to /java/lib alongside activemq-all-5.1.0.jar. The vendored ActiveMQ 5.1.0 broker (build/3rdparty-dists/apache-activemq-5.1.0-bin.tar.bz2) is extracted at build time and installed to /server/activemq, using its stock localhost-only config (port 8088, left as the tarball's default -- simpler than patching in a hostname the way install_ecce used to, since client and server are the same machine here). The vendored 32-bit linux-x86-32 Java Service Wrapper binary is excluded from packaging (this build only targets amd64, and dpkg-shlibdeps can't resolve a 32-bit ELF without i386 multiarch libs present at package-build time) -- activemq/bin/activemq picks linux-x86-64 automatically. siteconfig/jndi.properties (JNDI/topic config) is now installed (previously not installed at all), with its provider URL repointed from the hardcoded PNNL host to tcp://localhost:8088; the rest of the file (topic registrations, ~40 of them) is unchanged. New packaging/gateway/ecce-gateway-{start,stop,status} scripts manage the broker + JMSDispatcher as a persistent, per-user background pair -- idempotent start (checked via the broker's port + a JMSDispatcher pidfile in ~/.ECCE), since this package has no single controlling "toolbar" process the way the old scripts/ecce launcher's one-gateway-per-session model assumed. Installed both to /bin and as /usr/bin/ecce-gateway-* wrappers (same pattern as the app wrappers). Every ecce- wrapper (all ECCE_GUI_APPS plus eccejobmaster/ eccejobstore) now calls ecce-gateway-start before exec'ing, unless ECCE_NO_MESSAGING is set (same escape hatch the old launcher used for -admin/-machines). Also fixed while in there: the wrappers never set ECCE_REALUSER/ ECCE_REALUSERHOME, which src/util/genutil/Ecce.C's realUser()/ realUserHome() fatally assert on the first time anything touches preferences or cache paths -- every app would have aborted immediately on real use, this was never just a messaging-only gap. Also exports HOST explicitly: the Java side writes its UDP port file keyed by -DECCE_HOST, the C++ side (DatagramUtil::loadServerPort, src/util/jms/JMSMessage.C) reads it back keyed by the plain HOST env var -- tcsh auto-set that for the old launcher; a bash/sh wrapper has to do it explicitly or the two sides silently never agree on a filename. Verified end to end, twice -- once via a direct classpath run of the repo's own TestJMSPublish/TestJMSSubscribe test harnesses (java/src/.../jms/test/) against a manually-started gateway, and again against the fully packaged, installed .deb: dpkg -i, then a real ecce-eccejobmaster launch auto-starting the broker + JMSDispatcher via ecce-gateway-start, confirmed via ecce-gateway-status, then a fresh publish/subscribe round-trip against the installed jars/broker succeeding. eccejobmaster itself then crashed on an unrelated, pre-existing bug (eccejobmaster.C line 63 unconditionally reads argv[2] into a std::string -- crashes on any invocation without its real job-context arguments, e.g. a plain -h; not something this session introduced, not investigated further since it's outside "get the gateway working"). Known rough edge, not fixed: ActiveMQ's default config writes its own runtime state (KahaDB, logs) inside /server/activemq itself rather than a separate state directory -- dpkg -r reported "directory not empty" on removal because of this. Cosmetic (the package still installs/removes/reinstalls fine), but worth pointing the broker's data/log config outside the package tree if it ever bothers Andy. Progress this session: Python 2 -> 3 port (source-level, not yet packaged)
All 21 .py files in the repo (scripts/prp_gui.py, scripts/pmf_gui.py, scripts/codereg/*.py (17 files), data/admin/basissets/ecpfix.py, src/util/command/test/testCommandManager.py) now parse and compile clean under Python 3 (python3 -m py_compile; 8/21 failed outright before). Fixed via python3 -m lib2to3 -w (print statements, <> -> !=, etc.) plus manual follow-up for what it flagged but couldn't auto-fix or didn't cover: 4 raise "BadRangeFormat"/raise "BadRangeValue" string-exception raises in scripts/codereg/templates.py (Python 3 doesn't support raising strings at all -- converted to raise ValueError(...)), and 6 calls to string.atoi()/string.atof() in the same file (removed from the string module entirely -- converted to plain int()/float()). Also fixed data/admin/basissets/ecpfix.py's ambiguous #!/usr/bin/python shebang to #!/usr/bin/env python3.
This is a syntax/stdlib-API pass only, matching what could actually be verified in this sandbox (no display, no wxPython installed here). The wx-using scripts (prp_gui.py, pmf_gui.py, all of scripts/codereg/) still need their wxPython API calls checked against wxPython Phoenix (the Python 3 wx binding) vs. wxPython Classic (Python 2-only), which they were presumably written against -- this is the same kind of porting work as the C++ wx 2.8->3.2 port earlier this project, just not yet done for the Python side, and not verifiable without a real wxPython install plus a display. Not packaged into the .deb either -- these still live only in the source tree, per the "Andy's forks" note below (packaging them risks shipping the wrong version if Andy's forks have already diverged here).
Codebase shape ~1150 C++ files (.C/.H), plus small peripheral pieces: Java (9 files, JMS gateway now built/packaged, see above), Python (21, syntax-ported to Python 3 this session, see above), Perl (~20, still untouched), Fortran (14). Only a minority of the C++ files include wx headers directly: apps (56/182, now all building and linking), wxgui (29/162, done), wxviz (2/30, done), inv (5/278, done). Everything else (util, tdat, dsm, viz, comm — ~560 files) has zero wx dependency and was already done. The wx 2.8 -> 3.2 port is now functionally complete across the whole codebase. Every module builds, and — for the first time this whole effort — actual executables link successfully, not just static libraries. Build environment used (this session)
Ubuntu 24.04 container with: libwxgtk3.2-dev libxerces-c-dev libgl1-mesa-dev libglu1-mesa-dev libgtk-3-dev libjpeg-dev default-jdk gfortran ant libtool-bin autoconf automake imagemagick libxt-dev csh tcsh. Per above, this same package list is expected to work on Debian 13 but hasn't been run there yet.
Build system: CMake (replaces build_ecce/recursive-make)
CMakeLists.txt at the repo root replaces the old build_ecce + recursive-Make system entirely — now for the whole codebase, including the final executables. One static-library target per old LIBRARY= grouping via an ecce_library() helper, plus a new ecce_app() helper (added this session) for the 22 src/apps executables that mirrors it — find_package(wxWidgets 3.2 REQUIRED COMPONENTS core base gl aui net) for correct wx3.2 auto-detection (AUI is built into core now, no separate ewxaui lib the way Makefile.wx hardcoded for 2.8; net added this session for WxJMSSubscriber's socket usage), find_package(XercesC), find_package(OpenGL), find_package(X11) (added this session — see below), pkg_check_modules(GTK3 ...) for wxplotctrl's GTK dependency. Verified as of this session: cmake -G Ninja .. && ninja from a completely clean build directory compiles all 1128 build targets with zero errors — every static library, every one of the 22 final executables, from a totally clean build directory. (Pre-existing deprecation warnings remain, e.g. non-virtual-dtor deletes and old wxPen/wxBrush constructor overloads — not touched, out of scope.)
Progress this session: the wxWidgets 2.8 -> 3.2 port src/inv/wxinv + src/wxviz (commit 956c1a9) — done, verified building
The Open Inventor/wxWidgets bridge layer and the visualization tools built on top of it. Highlights:
wxGLCanvas no longer implicitly owns a GL context in wx3.x — introduced an explicit wxGLContext member (SoWxRenderArea), matching how every other modern wx OpenGL app has to do it now. wxGLCanvas constructor's attribList parameter moved earlier in the argument list; OnSize() base-class chain-up was removed (now just event.Skip()). wxMouseState::LeftDown()/MiddleDown() renamed to LeftIsDown()/ MiddleIsDown() (only on wxMouseState — wxMouseEvent kept the old names, so this needed care not to over-apply). wxCursor(int) is a private constructor now (ambiguous overload resolution toward wxString(int)) — needs an explicit wxStockCursor-typed variable. wxString::c_str() returns a lazy wxCStrData proxy now, not a raw pointer — implicitly convertible to const char* (still works for C-API call sites) but not to std::string (two chained conversions, disallowed). This was the single most common fix needed throughout the whole wx port — always applied surgically at the exact compiler-flagged file:line, never as a blanket find/replace (plenty of .c_str() calls are still correct as-is). wxADJUST_MINSIZE sizer flag removed entirely (was a no-op since wx2.9) — swept out of 125 files mechanically, safe since it was always OR'd in as a harmless no-op flag. _T()/wxT() misapplied to runtime string expressions instead of string literals — used to silently no-op under old ANSI-mode wx, hard errors (token pastes an invalid identifier) under wx3.x's Unicode-only builds. Found and fixed at each of the handful of sites this occurred. src/wxgui (commits 5b29420, 8f724c6, db0f4ee) — done, verified building
The bulk of ECCE's dialogs and custom widgets — 162 files across 5 CMake targets (eccewxgui, eccewxguicomm, eccewxguimd, eccewxplotctrl, eccewxthings). Went from 158 compile errors down to 0 across several passes. Same .c_str()/wxCStrData pattern as above accounted for most of it, plus a long tail of distinct API changes:
ewxGenericFileDialog rebuilt from scratch. wx2.8's wxGenericFileDialog (a full composite dialog built from a file-listing widget + text/choice controls, meant to be subclassed) was removed from wxWidgets entirely — not renamed, just gone; modern wxFileDialog is backed by native platform dialogs instead. ECCE's ewxGenericFileDialog subclassed it to add a server-choice dropdown (pick the local filesystem or one of the configured EDSI/remote compute servers, then browse it) — genuinely important functionality, not decoration, and three more dialogs subclass that in turn (SaveExperimentAsDialog, ExportTableDialog, OpenCalculationDialog in src/apps/builder, ~1600 lines total depending on it). This was flagged to Andy as a scope discovery — no longer a mechanical port — and he chose to have it rebuilt properly rather than stubbed. Rebuilt as its own wxDialog, built on wxFileDialogBase (still present in wx3.2, still supplies m_dir/m_fileName/m_path/m_wildCard/m_filterIndex/HasFdFlag()/ AppendExtension()) — the same approach wx2.8's own wxGenericFileDialog used internally. All the server-switching business logic (doSetServerChoice, HandleAction, goToHomeDir, saveSettings/restoreSettings) carried over close to verbatim; only widget construction/ownership needed rewriting. Not visually verified — there's no display in this sandbox, so the dialog's on-screen layout needs a look on a real desktop before trusting it completely. The subclasses (SaveExperimentAsDialog etc.) needed no structural changes — they reach the same protected members (m_list/m_choice/m_filterExtension) under the same names. wx2.8's internal file-listing widget wxFileCtrl (in wx/generic/filedlgg.h) was renamed to wxFileListCtrl (in wx/generic/filectrlg.h) — ewxFileCtrl/ewxFileData updated accordingly. (Note: wx3.x also introduced an unrelated new class also called wxFileCtrl — a modern composite file-picker widget, #defined to either wxGtkFileCtrl or wxGenericFileCtrl depending on platform — easy to confuse with the old internal one; ECCE doesn't use this new one.) A vendored TreeListCtrl.C/.H (predates wx's own wxTreeListCtrl, shares its class names) needed: wxTreeItemAttr → wxItemAttr (renamed); wx3.x made wxWindow::ProcessEvent protected in favor of the public ProcessWindowEvent() (deliberately blocks calling ProcessEvent on another window's pointer from outside — reproduced and confirmed against a minimal test case, not a bug in our code); wxScrolledWindow::OnScroll no longer exists as a chainable base method — call HandleOnScroll() directly (that's literally what OnScroll used to do internally); wxString:: Append(int) is ambiguous now, needs an explicit wxUniChar cast. wx/generic/grid.h no longer pulls in the concrete cell renderer/editor classes (wxGridCellStringRenderer, wxGridCellTextEditor, etc.) — they're in wx/generic/gridctrl.h now (4 files affected). wxGrid::SetLabelValue(orientation, text, index) is gone — replaced with the direct SetColLabelValue(index, text)/SetRowLabelValue(index, text) calls it used to dispatch to. A cluster of methods that exist in wx3.2's headers but are compiled out under WXWIN_COMPATIBILITY_2_8 (confirmed off in this build, matching Debian/Ubuntu's standard packaged wx3.2): wxGrid::SetCellTextColour()/ SetCellTextFont() single-arg convenience wrappers (→ SetDefaultCellTextColour()/SetDefaultCellFont()); wxMenuItem:: GetLabel()/SetText() (→ GetItemLabelText()/SetItemLabel()); wxLog's old DoLog(level, const wxChar*, time_t)/DoLogString() sink pair (→ DoLogTextAtLevel(level, const wxString&) sinking through DoLogText() — ewxLogTextCtrl was silently not being invoked as a log target at all under this build until this was fixed, since its override didn't override anything real anymore). wxFileHistory's internal storage moved from a raw pointer array (with an m_fileHistoryN count) to wxArrayString — ContextHistory.C's loops switched to bound on GetCount() instead, and dropped the now-meaningless pointer-truthiness checks. wxURLDataObject on GTK is now itself a wxDataObjectComposite (it internally negotiates both text/uri-list and plain-text formats), so it can no longer be nested inside another wxDataObjectComposite via Add(), which only accepts wxDataObjectSimple*. DnDCalcDrop swapped it for a plain wxTextDataObject — most drag sources offering text/uri-list also offer plain text, so this is a small, deliberate narrowing rather than a hard break (documented in the header for whoever revisits it). wxConfigBase::Write(key, T) has a generic template path for arbitrary T using wxToString(), which isn't specialized for std::string — ewxConfig now converts each vector element to wxString explicitly so it resolves to the direct Write(key, wxString) overload instead. wxPlotCtrl's GTK "fast graphics" path reached into wx-internal wxWindowDCImpl/raw GdkGC/GdkWindow structures to bypass wxDC for speed — GdkGC doesn't exist at all under GTK3 (Cairo replaced it), so this is fundamentally incompatible with the GTK3 backend this build always uses. Forced off (PlotDefs.H), falling back to the portable (slower, correct) wxDC-based drawing path PlotDraw.C already had as a fallback. Misc smaller ones: wxWindowDC::m_owner (GTK internal) → wxDC::GetWindow() (public); wxIcon needing an explicit #include <wx/icon.h> in two files that relied on a transitive include; wxPrintf needing wx/wxcrtvararg.h explicitly; one more _T()-on-runtime-expression bug (WxGridView.C, same class of bug as VizRender.C earlier); a local ButtonSizerFlags identifier in WxMeasurePrompt.C that doesn't exist anywhere in any wx version — replaced with an explicit wxOK|wxCANCEL|wxYES|wxNO|wxHELP|... mask matching its evident intent. src/apps (commit 89f34b8) — done, verified building and linking
The last piece: the 22 top-level executables (builder, organizer, calced, mddynamics, basistool, pertable, etc., plus the three non-wx command-line tools load_tgbs/ecmd/eccejobmaster+eccejobstore; pure Fortran src/apps/symmetry deliberately deferred, not part of wx port scope). CMake gained a new ecce_app(name dir [WX] [GL] LINK ...) helper alongside ecce_library(), mapping each old src/apps//Makefile's PROGRAMS/ECCE_LIBS to a CMake executable target. Same mechanical fix patterns as src/wxgui accounted for the great majority of the ~80 files touched (.c_str() -> .ToStdString(); stray T()/wxT() around non-literal expressions token-pasting garbage identifiers like LShapeData; wxSAVE/wxOPEN/wxOVERWRITE_PROMPT -> wxFD*; wxTR_EXTENDED -> wxTR_MULTIPLE; wxMouseState::LeftDown() -> LeftIsDown(); wxMenuItem::SetText() -> SetItemLabel(); a missing wx/listctrl.h include; one ambiguous GetStringSelection() call).
Two fixes went beyond mechanical translation:
builder's AUI docking (BuilderDockArt) used a locally-patched wx2.8-era AUI fork (wx/ewxaui/*, from build/ewxaui/) that added extra pane-caption buttons (take-focus, pin/add-focus, options) with custom bitmaps/events on top of stock wx AUI — no equivalent exists in wx3.2. Re-pointed at real <wx/aui/aui.h> (wxAuiManager/wxAuiPaneInfo); the extra per-pane mini-buttons are gone (cosmetic feature loss, not functional — docking, show/hide, and save/restore layout all still work through stock wxAUI). New src/apps/builder/EwxAuiCompat.H/.C holds the small compat shim this needed. src/inv/flclient/flfreetype.c: dropped a call to FT_Done_GlyphSlot(), which no longer exists in modern FreeType's public API — it was freeing a face-owned glyph slot it never should have freed, so removing the call is behaviorally a no-op (not a functional change).
Also surfaced two latent gaps that only showed up once real executables started linking (nothing had linked an app against these libraries before this session):
libeccecommxt.a (src/comm/commxt/JobStore.C, used by eccejobstore) calls raw X Toolkit Intrinsics APIs (XtAppAddSignal, XtCreateApplicationContext, etc.) directly — needed find_package(X11) linking X11::Xt/X11::X11 as a PUBLIC dependency on the eccecommxt CMake target so it propagates to every consumer (eccejobstore, eccejobmaster, launcher, mdprepare, organizer). builder/vizthumbnail (which pull in libecceinv, the Open Inventor layer) needed X11::X11, jpeg, and freetype link libs added directly to those two targets.
Verified with a full clean rebuild from scratch (at the time): 1128/1128 targets built with zero errors, covering the entire codebase end to end (src/util through the 22 final src/apps executables).
C++17, src/apps/symmetry, and install() (commit 7a8e091) — done
The last of the previously-deferred cleanup items that were pure C++/build mechanics (as opposed to a fork/scope question — see below):
Removed all old-style dynamic exception specifications (throw(SomeException, ...) on function signatures) — 629 of them across 317 files, a C++11-deprecated / C++17-removed feature this codebase used throughout. Specs naming actual exception types were just dropped (equivalent under C++17's own removal — the function can throw anything, same as no spec); the ~7 bare throw() ("throws nothing") specs became noexcept, the direct modern replacement (mostly EcceException overrides: what()/clone()/report()). Applied via a regex requiring spec contents to be identifier/scope-only — by construction this can never match a real throw statement, since those always contain a ( from constructing the exception object; hand-verified the one close call (throw(InvalidException(msg, WHERE)); in UserEditor.C) was correctly left alone. CMAKE_CXX_STANDARD bumped from 14 to 17. Nothing else needed to change — once the removed-in-C++17 exception-spec syntax was gone, the whole codebase already compiled clean under C++17. src/apps/symmetry (5 standalone Fortran executables — autosym, getfrag, genmol, genmollat, cleansym; testnames excluded, matching the old Makefile's default target) now has CMake targets. Needed per-target Fortran_MODULE_DIRECTORY properties, since several of these independently compile the same spnames.f90/spgen.f90 module sources and would otherwise collide on a shared .mod output. Also deleted src/apps/symmetry/spnames.mod — a stale compiled module file the old build system had committed straight into the source tree, which was shadowing the freshly-compiled per-target one on the compiler's include path and failing with "created by a different version of GNU Fortran". Basic install() rules — all 29 executables to bin/, the data/ tree alongside. Deliberately not a port of build_ecce/ install_ecce/create_ecce_bin (see "Open question" below) — just enough for cmake --install --prefix
to produce a usable layout on its own. Verified working end to end.Verified with a full clean rebuild from scratch: 1210/1210 CMake targets build with zero errors under C++17, and cmake --install correctly deploys everything to a test prefix.
.deb packaging via CPack (commit a489598) — done, replaces install_ecce
Andy's call: instead of porting the old install_ecce/create_ecce_bin self-extracting installer (see below), just build a plain .deb. CMake's CPack does this directly off the existing install() rules:
cd build-cmake && cpack -G DEB
produces ecce__amd64.deb (version pulled from data/client/config/Version, v7.3.4-beta → the Debian-legal 7.3.4~beta). Everything installs under /opt/ecce (bin/ + data/ as siblings — matches what the compiled apps expect via $ECCE_HOME, see Ecce.C/SFile.C's $ECCE_HOME path expansion; a split FHS layout would need path-handling surgery this session hasn't done). New thin /usr/bin/ecce- wrapper scripts (ecce-organizer, ecce-builder, etc. — 19 of them, one per GUI app) set ECCE_HOME=/opt/ecce and exec the real binary, so apps land on PATH with nothing to source into .bashrc/.cshrc. These are newly written for this build's flat single-platform layout, not a port of the old scripts/ecce, ebuilder, eviewer etc, which assume a vendored 3rdparty Python interpreter, multi-platform ECCE_SYSDIR subdirectories, and (for the top-level ecce launcher specifically) the Java/ActiveMQ JMS messaging server. Those old scripts, plus the Perl/Python helper tools (gbsDAVConverter/gbsDescriber/gbsNWChemConverter, eccejobmonitor, processmachine, pmf_gui.py/prp_gui.py), are deliberately excluded from the package — they're exactly the peripheral Perl/Python territory Andy's own forks have diverged on (see "Andy's forks" below), so packaging this repo's copies would ship the wrong version. The internal command-line tools (load_tgbs/ecmd/eccejobmaster/eccejobstore, the 5 symmetry Fortran tools) install to /opt/ecce/bin but get no /usr/bin wrapper, matching how the old build never gave them a top-level e* script either.
Runtime deps are auto-detected via dpkg-shlibdeps (CPACK_DEBIAN_PACKAGE_SHLIBDEPS) rather than hand-listed, so it stays correct as wx/Xerces/Mesa versions change — the built package's actual Depends: line came out as libc6, libfreetype6, libgcc-s1, libgfortran5, libglu1-mesa|libglu1, libglx0, libjpeg8, libopengl0, libstdc++6, libwxbase3.2-1t64, libwxgtk-gl3.2-1t64, libwxgtk3.2-1t64, libx11-6, libxerces-c3.2t64, libxt6t64. CPACK_STRIP_FILES strips debug symbols from the packaged copies only (the RelWithDebInfo build tree itself keeps them, useful while still actively porting) — brought the package from 700MB down to 57MB.
Verified end to end in this sandbox: dpkg -i installs cleanly with every dependency already satisfiable, ldd shows no missing shared libraries on the installed binaries, the wrapper scripts land on PATH and point at /opt/ecce correctly, and dpkg -r removes cleanly. Not yet verified: actually launching a GUI app (no display in this sandbox — see "Remaining work"), and building/installing on real Debian 13 rather than this Ubuntu sandbox.
Open question: build_ecce/install_ecce/create_ecce_bin
Resolved for the JMS/gateway piece this session -- see "Progress this session: the JMS gateway" above. Andy confirmed client+server-same-machine is the normal case; the single-machine setup install_ecce used to do interactively (extract the broker, patch siteconfig, write start/stop scripts) is now handled automatically by the .deb + ecce-gateway-start. The old -remote / separately-hosted-server flow those scripts also supported is still not ported (deliberately out of scope). The rest of what those scripts did (general packaging/distribution-building) remains superseded by CPack, as before.
Remaining work Decide what to do about the two personal forks with your Perl/Python changes (see "Andy's forks" below) -- this determines what "finish the peripheral pieces" means for the Perl scripts specifically (Python now has a syntax-level Python 3 port done regardless of the fork question, see above). Verify this whole build (and the .deb) on real Debian 13 -- done, confirmed by Andy this session. The JMS gateway additions haven't specifically been re-verified there yet (built/tested only in this session's Ubuntu sandbox) -- worth a quick confirm next time Andy rebuilds. Look at ewxGenericFileDialog's on-screen layout on a real desktop, and now also builder's AUI docking layout (the dropped mini-buttons, confirm docking/undocking/layout persistence look and behave right) -- neither could be visually verified at all in this sandbox (no display). Runtime smoke testing of the GUI apps themselves (opening a molecule file, running a calculation setup, etc.) -- still zero done, still needs a real display. The non-GUI messaging path (broker/dispatcher/pub-sub) is now verified this session (see above) -- this item is specifically about the GUI apps' own behavior once actually running. Blocked in practice by the gateway crash investigation below — gateway is one of the ECCE_GUI_APPS and currently can't complete startup. wxPython Classic -> Phoenix API port for the Python wx scripts (prp_gui.py, pmf_gui.py, scripts/codereg/*.py) -- the Python 3 syntax port is done (see above), but their actual wx API calls are unverified; needs wxPython installed plus a display to check. Also still open: the Perl peripheral pieces (~20 files, untouched), which is where item 1's fork decision matters. Decide on and, if wanted, port build_ecce/install_ecce/ create_ecce_bin -- done for the gateway/messaging piece this session, see "Open question" above.
The throw(...)/C++17 cleanup and the src/apps/symmetry CMake target — done, see above.
Andy's forks — open question
FriendsofECCE/ECCE (this repo, develop branch) is believed to be Gary's/PNNL's upstream version — everything ported and modernized this session lives here. Andy separately maintains two of his own forks with changes concentrated in the Perl/Python layer (not touched by this session at all): one with "moderate" fixes/nagging-issue cleanup, one with "adventurous" functional changes. Their relationship to FriendsofECCE/ECCE (ahead of it, diverged from a common ancestor, cherry-pick-able, or something else entirely) is not yet known — Andy wasn't sure how to locate them via GitHub's fork UI, which only surfaces repos created with GitHub's own "Fork" button; given ECCE's long history (SourceForge era, institutional copies), they may be separate repos never forked through GitHub at all. Once Andy provides their locations (repo URL, local path, or export), next steps are: figure out how they relate to this codebase, and decide whether to port his Perl/Python changes onto this modernized C++ base, modernize the C++ directly in whichever of his forks is the real target, or something else.
Where things live Local clone + branch modernize-build, nine commits on top of develop's current head (f98cecd): 623952f, d0cf2d6, 956c1a9, 5b29420, 8f724c6, db0f4ee, 89f34b8, 7a8e091, a489598. Not pushed anywhere — this sandbox has no GitHub push access (confirmed: the GitHub proxy this environment routes through reports repo access isn't enabled for this session, and there's no GitHub connector configured for Andy's account either). Delivered as a patch file (ecce-modernize- build-05.patch as of the last send) for Andy to apply and push himself; git am on a clean checkout of develop at f98cecd applies all commits in order. Andy has SSH access to GitHub in the past (unconfirmed whether it's still live) separately from the website login, which is currently blocked behind an MFA prompt he doesn't have set up. ssh -T git@github.com from his machine will confirm whether a key is still registered and working, independent of the web-login MFA issue. The active gateway crash investigation (below) is happening directly on niobium, in ~/tmp/ecce/ECCE (a real Debian 13 machine with a display) — a separate local branch modernize-build-fixes, also not pushed. How to compile (once the patch is applied)
Packages needed (Ubuntu 24.04 names used/verified this session — see "Standing preference" above for the Debian 13 equivalents, expected to match): libwxgtk3.2-dev libxerces-c-dev libgl1-mesa-dev libglu1-mesa-dev libgtk-3-dev libjpeg-dev libxt-dev default-jdk gfortran cmake ninja-build. Then, from the repo root:
mkdir build-cmake && cd build-cmake cmake -G Ninja .. ninja
That builds all 1215 targets (every static library plus all 29 executables — the 22 src/apps apps, 2 jobstore tools, load_tgbs/ ecmd, and the 5 symmetry Fortran tools). To install them somewhere: cmake --install . --prefix /wherever (drops executables into /bin and the data/ tree alongside). A Makefile generator also works (cmake .. instead of -G Ninja, then make -j$(nproc)) if Ninja isn't installed — Ninja was just what this session used for its faster incremental rebuilds.
Or build a .deb directly (see the CPack section above):
cd build-cmake && cpack -G DEB sudo dpkg -i ecce_*.deb
Installs everything to /opt/ecce, adds ecce-builder/ecce-organizer/ etc. wrapper commands to PATH. sudo apt-get install -f after dpkg -i if it complains about missing dependencies (shouldn't, on a system with the dev packages above already installed, since those pull in the same runtime libs).
gateway OOM/crash — RESOLVED, both fixes committed (2026-08-28)
Separate from the porting work above. Started because gateway (one of the ECCE_GUI_APPS, wxWidgets toolbar app in src/apps/gateway/) was consuming multi-GB RAM and freezing whole machines (Beryllium, niobium) before being OOM-killed. This was picked up by a claude-code session running directly on niobium (real local shell, not the earlier file-bridge relay — see "A note on tooling" below, which turned out to matter a lot) and closed out the same day. Two distinct bugs were found and fixed, both the same root-cause class (a wx3.2/GTK3 resize/layout reentrancy storm) but with different, non-obvious triggers. Commits on branch modernize-build (not a separate modernize-build-fixes branch — that name from earlier notes either never existed on this checkout or the work just landed directly on modernize-build): aeb332f (construction-time fix), d709f66 (unrelated EcceDAVClient fix, see below), 94ed704 (a new ECCE_SYSDIR bug found while verifying the first fix), 3c50f03 (the second, Show()-triggered fix). CLAUDE.md in the repo root has the full blow-by-blow if any of the summary below needs expanding.
Bug #1: construction-time crash, fixed in aeb332f
Confirmed root cause chain
GatewayApp::OnInit() (GatewayApp.C:108) → Gateway::Gateway() (Gateway.C:97, new GatewayPrefs(NULL)) → GatewayPrefs::GatewayPrefs() constructor (GatewayPrefs.C, originally line 125's Fit() call, see patches below) → wxWidgets/GTK3 enters a resize/Layout reentrancy storm: wxWindow::DoSetSize fires a wxEVT_SIZE event, whose handler (wxWindowBase::InternalOnSize) calls Layout(), which repositions children, one of which resizing fires another size event, recursing without ever converging. Confirmed via gdb backtraces reaching ~12,470 repeating iterations / 100,000+ stack frames. With an unbounded stack (ulimit -s unlimited, the default) this consumes ~2GB RSS over ~20 seconds before the kernel OOM-kills the process — that's what was freezing the machines. With ulimit -s 8192 (8MB) it instead fails fast as SIGSEGV in wxWindow::DoSetSize, which is the repro used for all debugging below (much faster iteration than waiting for OOM).
Confirmed via Valgrind (--tool=memcheck --track-origins=yes) that this really is a stack overflow (Stack overflow in thread #1... at wxWindow::DoSetSize), not heap corruption — an earlier hypothesis along those lines (crash location/depth varying between runs) was tested and ruled out. (Valgrind also surfaced 3 unrelated "mismatched free() / delete[]" bugs in EDSIServerCentral's C++ static initializers doing Xerces DOM parsing — real bugs, but run before main(), nowhere near this crash; not investigated further, worth a separate look someday.)
Confirmed single-threaded (all recursion on the GUI thread, "gateway" — ruled out a cross-thread race with the JMS listener threads as a cause).
Updated root cause detail: the GatewayPrefs.C:125/190 Fit() call named above turned out not to be the only relevant trigger. The real first Fit() call in the object's lifetime is earlier and was originally unguarded: GatewayPrefsGUI::Create() (GatewayPrefsGUI.C:154, GetSizer()->Fit(this)), invoked from the GatewayPrefsGUI base-class constructor, which — by ordinary C++ construction order — fully runs and completes before GatewayPrefs's own constructor body (where GatewayPrefs.C's Fit() and its guard live) ever starts. Earlier attempts targeted only the later, GatewayPrefs.C call because that's the one visible in GatewayPrefs's own source, but a plain wxWindowBase::Fit()/Layout() backtrace can't distinguish which call site triggered it, so the earlier one was missed until the source was read line-by-line. Both call sites needed the same fix, see below.
Ruled out along the way (kept for context, not re-investigated): EcceDAVClient::getBody() buffer overread (src/dsm/dav/EcceDAVClient.C, was (*mem) << buff treating a non-null-terminated 1500-byte scratch buffer as a C-string) — real bug, fixed (mem->write(buff, nRead), committed in d709f66), but a diagnostic trace proved it's not on this crash's code path. Heap corruption — tested directly via Valgrind. It's a genuine stack overflow. Five earlier fix attempts (Freeze()/Thaw() around SetSize(); a depth-capped Layout() override on ewxFrame/ewxDialog; moving SetSizer() to the end of CreateControls(); and two rounds of a wxEventFilter-based suppressor at different Fit() call sites) all failed to stop the recursion. The full detail of why each failed is preserved in this repo's CLAUDE.md if it's ever useful precedent — the short version is that none of them were wrong in mechanism, just not armed for the actual call that mattered (see the real fix below).
The actual fix (committed aeb332f): the wxEventFilter mechanism from the last two attempts above was correct all along — proven by adding an unconditional diagnostic trace (no suppress-flag gating, log every event FilterEvent sees) exactly as this doc had already flagged as the needed next step. The trace showed FilterEvent being called correctly, and correctly suppressing wxEVT_SIZE during the guarded window — but both GatewayPrefs.C and GatewayPrefsGUI.C re-enabled event processing (g_suppress... = false) before the follow-up Layout() call each made afterward to "fix up final child positions." That unguarded Layout() — not Fit() itself — was the actual trigger. Moving the re-enable line to after Layout() in both files closed the gap. Verified crash-free across 8+ repro runs.
Also committed while verifying this fix: 94ed704, guarding getenv("ECCE_SYSDIR") against NULL at 5 call sites (ResourceDescriptor.C and 4 others). This build's flat single-platform CPack packaging (see above) deliberately no longer sets ECCE_SYSDIR — old multi-platform subdir routing, not applicable to this layout — so any of these call sites null-derefed the first time gateway got far enough to reach them, which it only could once the stack-overflow crash above was out of the way.
Two of the five earlier attempts were reverted rather than kept once the real fix was found: the Freeze()/Thaw() wrapper (confirmed to have zero effect — the whole recursion happened inside its wrapped call) and the depth-capped Layout() override on ewxFrame/ewxDialog. The latter was worse than just ineffective — it silently returns without laying out anything past a hardcoded depth, which could mask a genuinely different future bug as a subtly-broken UI instead of a loud crash, and it was never proven necessary once the real trigger was understood.
Bug #2: a second, separate crash found while verifying bug #1's fix, fixed in 3c50f03
Found the same evening, while trying to visually confirm the construction-time fix on a real display (gateway has no way to actually show a window in this environment — see "DataServers / EDSI data server" note below — so a small standalone test harness was written to construct GatewayPrefs directly and Show() it, bypassing GatewayApp::OnInit()'s server checks entirely). That harness's very first run had no memory cap and grew to 40GB+ RSS before being killed by hand — the exact OOM failure mode this whole investigation started from, and entirely avoidable: this doc already documented that risk, and it should have been guarded against from the very first live test. Every run after that used a hard, kernel-enforced cap (systemd-run --user --scope -p MemoryMax=... on top of ulimit -s, since ulimit alone only bounds the stack, not an unbounded heap-growth variant of this same bug class) — worth remembering as standing practice for any future live testing of a fresh, unverified hypothesis against this codebase.
Root cause: Gateway.C's real "open Preferences" handler (p_prefsDlg->Show(true), not a test artifact) triggers the same DoSetSize -> wxEVT_SIZE -> InternalOnSize -> Layout() -> RepositionChildren -> DoSetSize cycle as bug #1, but starting asynchronously, well after Show() (and wxEVT_SHOW) have already returned/fired. Confirmed via a gdb capture that the storm begins inside GTK's own recursive size-allocate cascade (wxPizza::size_allocate_child, wx's internal GTK container class), not synchronously inside any call this code controls.
Four release-timing strategies were tried and rejected before the actual fix: a synchronous Show() wrapper (released before the trigger even started); a single CallAfter() (one event-loop round wasn't enough); binding wxEVT_SHOW plus a 2s safety timer (based on reading wxWidgets' GTK3 backend source — theory was that Show() defers the real GTK realize while it round-trips with the window manager for frame extents, and wxEVT_SHOW should bracket that; fired too early anyway); and a fixed 1.5s timer, which is the most important negative result of the four — non-deterministic (10/13 survived in one batch, 5/8 in another), proving no fixed duration can be trusted since the real trigger's timing genuinely varies run to run.
What actually broke the investigation open: rather than keep guessing release points, a gdb Python breakpoint on wxWindowBase::Layout whose stop() condition walks the live call stack and only actually stops if "Layout" appears 2+ times (genuine self-reentrancy, not just any call) — this catches the first reentrant call at a shallow depth (tens of frames), not the eventual 100,000+ frame crash, so gdb can unwind and print it instantly with no memory pressure (a plain "capture the crash and unwind it" approach hit a real wall here: gdb itself needs more memory/time than the crashing process to unwind that deep, directly in tension with the safety caps the bug requires).
The fix: SizeEventSuppressor::FilterEvent (the same class used for bug #1) now also swallows a wxEVT_SIZE whenever a call to Layout() is already on the stack when it arrives — detected via glibc's raw backtrace()/backtrace_symbols() (<execinfo.h>). An earlier attempt at this same idea using wxStackWalker crashed with SIGBUS from this call context (not further diagnosed why, just confirmed unsafe here); raw backtrace() bypasses wx's own wrapper and worked cleanly. Symbol names come back mangled (e.g. _ZN12wxWindowBase6LayoutEv), but a mangled name still contains the original identifier as a substring, so a plain strstr() for "Layout" matches correctly with no demangling needed. This needed no changes to GatewayPrefs.H at all — no Show() override, no timer, no new state — just an addition inside the existing global filter, always active, reacting to real reentrancy instead of any guessed time window.
Verified: 20/20 safely-capped runs of the test harness (including three 30-second runs) driving the real Show(true) path, plus 12/12 runs of the actual gateway binary confirming bug #1's fix still holds with no regression (important since the new check runs on every wxEVT_SIZE process-wide, not just within GatewayPrefs).
DataServers / EDSI data server — separate, unported piece of infra, not fixed, not blocking
gateway fully constructs its GUI (both crashes above) before GatewayApp::OnInit() checks connectivity to the EDSI/WebDAV "ECCE Server" listed in /opt/ecce/siteconfig/DataServers. That config's default entry is still the real PNNL production URL, unreachable from a dev sandbox, so EDSIServerCentral::checkServerSetup() (untouched legacy code, not part of this session's changes) hard exit(1)s with a cerr message — old, pre-existing behavior, not a regression from anything here. This is why neither crash fix could be confirmed by actually looking at the window on screen — gateway never gets far enough to show one in this environment.
Turns out old ECCE's "server" was actually two independent pieces, both started by the old build/server_admin/start_ecce_server.ecce script: an Apache 2.2 + mod_dav data server (a vendored httpd-2.2.25.tar.bz2 sits unused in build/3rdparty-dists/, alongside account/htaccess CGI scripts in build/server_admin/) and the ActiveMQ message server. This session's earlier JMS gateway work (see above) ported and packaged the message-server half only — the data-server half has never been touched, and standing it up for real (building Apache 2.2.25 from source on modern Debian, wiring mod_dav, porting the account/htaccess scripts) is a separate task comparable in scope to that JMS porting work, deliberately not folded into this crash investigation. EDSIFactory does support a file:// (or empty-protocol) scheme via FileEDSI that just stat()s a local directory, no network — plausibly how old ECCE could run without a real server — but DataServers is root-owned on this checkout, so redirecting it needs explicit sign-off, not yet given.
A note on tooling: this is why niobium shell access mattered
Earlier notes on this investigation (before this session) were relayed through a Cowork session with file read/write access to niobium but no shell there — every rebuild and gdb run had to be done by hand and pasted back, which is exactly the kind of tight build-run-inspect loop that's slow and error-prone to relay. This session ran claude-code directly on niobium instead, and closed out in one sitting what had stalled for a while under the relay model. Worth defaulting to for any future niobium-side debugging with a similar shape (many short, iterative native rebuild/run/inspect cycles).
Still open
Manual interactive resize testing — still not done. Both fixes are verified via safely-capped automated runs (no crash across many repetitions), not by an actual human resizing the real toolbar and Preferences dialog on screen, per this investigation's original goal. Blocked on either the DataServers/EDSI question above, or reusing/rebuilding the standalone test harness (not committed, lived only in that session's scratchpad) in a way that actually confirms visual correctness, not just non-crashing. FilterEvent now runs backtrace()/backtrace_symbols() on every wxEVT_SIZE process-wide (outside the manually-suppressed construction window) — correctness is verified, but overhead during rapid real interactive resizing (many wxEVT_SIZE events per second while a user drags a window edge) hasn't been measured. backtrace_symbols() calls malloc and does real symbol-table work each time; worth profiling once there's a real display session to check on. Whether the same Fit()/SetSizeHints() structure needs the same fix elsewhere: confirmed via grep to exist across dozens of *GUI.C files in ~15 other ECCE_GUI_APPS (same wx designer tool generated all of them), but none of them have ever been reported crashing, and both bugs found here were specific to GatewayPrefs's particular widget tree and Show() timing — not proof that Fit()/SetSizeHints() is inherently dangerous elsewhere. Treat as a watch-list item if another app is ever reported freezing/crashing, not something to preemptively patch without evidence.