fix: stop calls to IO and other writes to stdout from crashing Expert.

[Moosieus]

The Problem

Expert uses stdio as the communication bus for LSP as is customary for language servers. A problem is that any writes to stdout that don't strictly follow the protocol will break it. This error case manifests as an explicit server shutdown in the best case scenario (client shuts down the server), and in the worst case, obfuscated errors like Header must provide a Content-Length property.

This problem persists today in Expert where the manager node makes (e)rpc calls to the project node, which redirects the project node's standard output to the manager node, which can corrupt the LSP connection. This problem is ephemeral and hard to replicate, and I suspect is the cause of several issues, including but not limited to #366 (comment), #539, #744, and #781.

The Solution

1. Create a thin I/O device that forwards to :standard_error and register that as the :user I/O device.

• Give GenLSP the PID of the original :stdout I/O device.

This prevents IO module calls from corrupting the :stdout connection by routing their output to :standard_error which language clients handle. In the case of Visual Studio Code, the error writes appears as messages in the LSP output, which is especially useful for debugging, since the output is colocated with the LSP message logs.

This effectively closes this error path by isolating stdio to the language server output, so we don't have to play whack-a-mole with every potential write to :stdout going forward.

[mhanberg]

Is there any cases where the expert node is printing to stdout on accident? or is this mostly about the untrusted engine code from the user's project and dependencies?

On Next LS, all I did was make sure the engine node used its own stdout instead of redirecting to the server node: https://github.com/elixir-tools/next-ls/blob/eb47c98eef92ffe2b369c7c2bf56ce63b837f949/priv/monkey/_next_ls_private_compiler.ex#L1072, which I think would solve the issue here.

[Moosieus]

3 outstanding items here:

1. I haven't fully considered the implications on supervision and recovery. I need to do a double-take on supervision and restart here, as the base :stdout I/O device is being passed as a PID to GenLSP.
   • PR updated per Correctly shutdown in case we shut down the buffer #428. The best recourse for losing the buffer is simply for the server to exit and restart with a fresh connection.
2. deps currently points to my fork of :gen_lsp, awaiting this PR here:

   feat: allow GenLSP.Communication.Stdio to use devices other than :stdio elixir-tools/gen_lsp#88

3. CI seems to be having a cow where things were fine yesterday:

   https://github.com/expert-lsp/expert/actions/runs/29839733158/job/88665221272?pr=808

   Run extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3
   Run extractions/setup-crate@0551596312d4008a6dfbc4d3c38fa20eaef46d2d
   Error: no release for just matching version specifier
   

   • CI is being funky here but passes here on my fork, spare the Nix test not liking the git-based package dependency.
   • fix: stop calls to IO and other writes to stdout from crashing Expert. Moosieus/expert#4

[Moosieus]

Is there any cases where the expert node is printing to stdout on accident? or is this mostly about the untrusted engine code from the user's project and dependencies?

On Next LS, all I did was make sure the engine node used its own stdout instead of redirecting to the server node: https://github.com/elixir-tools/next-ls/blob/eb47c98eef92ffe2b369c7c2bf56ce63b837f949/priv/monkey/_next_ls_private_compiler.ex#L1072, which I think would solve the issue here.

My acute issue was due to compilation messages coming back to the manager node via (e)rpc. I don't know if there's any cases today where the expert node is printing to stdout on accident, but it's been a historically recurring issue.

My motivation here is to factor out the vector entirely and end the whack-a-mole of this problem. Another practical upshot is that this lets us use the IO module for debugging. Most language clients log stderr inline with other LSP messages, as seen below in VSCode:

[doorgan]

This kinda sems to work, as in it does not crash expert due to a context-length error or stuff leaking to stdout, but the error message shows as a notification

I can reliably reproduce this issue in my local fork of Spitfire, I think there's a different bug that happens if you have a corrupted hex store(probably due to me constantly switching elixir versions)

[Moosieus]

I've refined this draft further into #824, which takes a different approach. In retrospect: group_leader is used for more than just strictly I/O routing, so manipulating it is cause for uncertainty. The problem was that the proper API for this sort of thing is undocumented (discussed in the new PR), so I didn't find it right away. It's much more the correct path though.