Add Playwright coverage for Sketcher More Actions menu

[Sean-Seekins]

Summary

• add the opt-in WASM Playwright test bridge and Squish-parity Sketcher wrappers
• port the More Actions menu workflow to real Playwright mouse and keyboard input
• grant browser clipboard permissions so Copy, Cut, and Paste use the real clipboard
• add visual baselines for the converted workflow

Validation

• SKETCHER_WASM_BUILD_DIR=wasm-fast /home/linuxbrew/.linuxbrew/opt/node@22/bin/npx playwright test e2e/from_squish/test/tst_more_actions_menu.test.js --reporter=line
  • 1 passed (21.6s)

The test-only bridge is compiled only when SKETCHER_ENABLE_PLAYWRIGHT_TEST_BRIDGE is enabled.

[Sean-Seekins]

I don't know about format checks yet outside of QA, so I'm not surprised that tests failed. This is just for visibility to look at the code that I have so far.

[cdvonbargen]

these should be doable with playwright natively, I dont think you need these APIs

[Sean-Seekins]

Probably. Would it be better to do something like 'await page.evaluate(() => navigator.clipboard.readText());' rather than 'QApplication::clipboard()->text().toStdString();'

I think squish just wraps the qt command in the backend. Is there an advantage to one method over the other?

[Sean-Seekins]

I gave the command to get text rather than set text, but it is the same idea

[cdvonbargen]

I dont see the need for this? Is this used somewhere else?

I think this can be done with getWidgetRect → page.mouse.click → page.keyboard.type()

[Sean-Seekins]

It might be an intermediate function that ended up not being used in the final test. I didn't clean up yet. This PR is for proof of concept and initial discussion, not detailed review for production code.

[Sean-Seekins]

It also wraps several text functions into one. line_edit, text_edit, combo_box, spin_box

[Sean-Seekins]

It was an intermediate function that was replaced by a more human style interaction at some point

This is what is currently used.
/** Click a visible text control and replace its value through keyboard input. */
export async function setWidgetText(page, objectName, text) {
await clickWidget(page, objectName);
await page.keyboard.press('ControlOrMeta+a');
await page.keyboard.type(String(text), { delay: 10 });
}

[cdvonbargen]

I think we should redo these as get_item_rect((kind, index) so that you can grab atoms, bonds, monomers, S-groups.

[Sean-Seekins]

I can probably combine them if we want to. They both call scene_item_rect. monomers, S-groups... etc are accessible from atom coordinates, at least in squish. It might end up being a bit different here. Not sure yet.

[Sean-Seekins]

I haven't done refactoring at this point. It doesn't make sense to me to do that until we decide that this is the path we want to take with testing. This is just proof of concept. I don't want to spend time on details yet. This would be a very easy fix.

[cdvonbargen]

Ok so I talked to claude about this a bit, and think we could reduce this infra down to 3 functions (below). Curious for @JarrettSJohnson @juzerzarif thoughts. @Sean-Seekins would this cover everything you need?

  /**
   * Return position, size, and interaction state of a child widget found by its
   * Qt objectName, as a JSON object:
   *
   *   {"x":…, "y":…, "width":…, "height":…,
   *    "visible":…, "enabled":…, "checked":…}
   *
   * Coordinates are relative to the sketcher widget's top-left corner, which is
   * also the top-left of the WASM canvas, so e2e tests can use them directly as
   * page coordinates.
   *
   * When several widgets share an objectName, a visible one is preferred.
   * "checked" is false for non-checkable widgets. A hidden widget still returns
   * a full object with "visible":false, so tests can distinguish "hidden" from
   * "absent"; only a genuinely missing objectName returns "{}".
   *
   * Replaces sketcher_get_widget_rect, which returned the rect alone and
   * reported hidden widgets as missing.
   */
  std::string sketcher_get_widget_info(const std::string& object_name);

  /**
   * Return the position and size of a Scene graphics item addressed by model
   * index, as {"x":…, "y":…, "width":…, "height":…} in the same coordinate space
   * as sketcher_get_widget_info.
   *
   * Scene items are QGraphicsItems rather than QWidgets, so they cannot be found
   * by objectName; this maps the item through the View transform instead.
   *
   * The returned rect is always centered on the item's scene position. This
   * matters for atoms whose label isn't painted — an unlabeled carbon has an
   * empty bounding rect, and would otherwise yield a rect that can't be clicked.
   * Such items return a zero-size rect at the atom's center, so the rect center
   * is a valid click target for every item this returns.
   *
   * @param kind "atom", "bond", or "sgroup". Monomers are addressed as "atom",
   * since they are atoms of a monomeric molecule.
   * @param index Index into the MolModel molecule's atoms or bonds, or into its
   * substance groups when kind is "sgroup".
   *
   * Returns "{}" if index is out of range. Throws std::runtime_error for an
   * unrecognized kind, which indicates the test needs updating.
   */
  std::string sketcher_get_item_rect(const std::string& kind, int index);

  /**
   * Activate a named control: clicks a QAbstractButton, or triggers a QAction if
   * no button matches. Actions are matched on objectName first, then on display
   * text, since most menu actions are created without an objectName.
   *
   * Used by e2e tests for controls inside Qt::Popup windows — popup menus and
   * popup widgets get their own WASM canvas, whose event listeners Playwright
   * cannot target. Controls in the main canvas should be clicked with real mouse
   * events via sketcher_get_widget_info instead, so that Qt's own hit-testing
   * and enabled-state handling are exercised.
   *
   * Throws std::runtime_error if nothing matches, or if the target is disabled.
   * Programmatic activation bypasses the enabled check a real mouse event goes
   * through, so refusing here keeps a test from passing against a control the
   * user could not actually have activated. To assert that something is
   * unavailable, check "enabled" from sketcher_get_widget_info rather than
   * activating it and observing that nothing happened.
   *
   * Replaces sketcher_click_button.
   */
  void sketcher_activate(const std::string& name_or_text);