[ralgar]

Summary

Adds uniform controls to the Effects panel in the GUI.

Closes #62

Preview

Features

Added support for all ReShade FX UI widget types:

• Input
• Drag
• Slider
• Combo
• Radio
• Color
• Button

Added support for the following uniform annotations:

• ui_min
• ui_max
• ui_step
• ui_items:
• ui_label
• ui_tooltip
• ui_category
• ui_category_closed
• ui_category_toggle
• ui_text
• ui_spacing
• ui_units
• hidden
• noedit
• nosave (unrelated to this PR, will be merged separately)
• noreset

Testing

• Tested each uniform/widget type
• Tested step and reset controls
• Tested ordering and category behavior
• Tested failure paths

[ralgar]

@kakra Any chance you want to try this out? :D

The widgets and controls should be fully functional. The layout is more of a draft and is subject to change still.

[kakra]

Looks great, yeah I'll try. Currently I'm working on training a CNN to remove bad anti-aliasing artifacts from Elite Dangerous by reconstructing smooth edges, then convert it into a shader. Does look promising currently, but unfortunately not too promising, partially due to Elite Dangerous not applying SSAO to high res screenshots for training data which kills the learning masks. :-( Also, it looks like vkShade will need to support compute shaders some time in the future to get decent performance from such a shader, currently I can only do full screen passes with vkShade which eats a lot of texture bandwidth, at least if I go beyond 7x7 or 11x11 context masks.

But for ETS2, judging by the screenshot, this will certainly improve my workflow a lot! <3

BTW: If this works well, it could become a contribution to the community shaders repo you're planning. :-)

[ralgar]

That sounds interesting!

partially due to Elite Dangerous not applying SSAO to high res screenshots for training data which kills the learning masks

Strange that it would take a different render path for this. It would be pretty easy for vkShade to dump the presentation image out to a file. I've actually been tinkering with this a little bit already, but there would need to be a bit of GUI and input work first to facilitate that as cleanly as I would like. Compute shaders will be a near-future thing as well.

BTW: If this works well, it could become a contribution to the community shaders repo you're planning. :-)

Sure, that'd be great! Let me know when you finish it and I'll look at creating the repo. I might do a preset repository as well, or maybe just a monorepo for both.

But for ETS2, judging by the screenshot, this will certainly improve my workflow a lot! <3

Let me know how it goes! Not sure if you noticed but there is a file chooser now as well, so you can have a collection of presets and switch them up at runtime.

I'll probably align the layout more with ReShade - a single "Reset all" button, per-effect rather than per-uniform, then move the step controls and labels to the right, and move the slider/drag vector widgets all onto the same line. We lose the per-uniform reset button, but I think it'll be an improvement overall, and it's probably necessary to facilitate the spacing and text annotations cleanly.

[kakra]

Strange that it would take a different render path for this.

SSAO requires the full screen space, but the high res screenshot tool in ED (Alt+F10) renders the frozen scene in multiple tiles to stitch together a 16k image. Due to this tiling, SSAO cannot be used. It seems to be the only post-processing effect that is missing, tho - except that blooming is also resolution dependent and I need to turn that off for the training. For my CNN, it could be helpful to dump the HDR buffer of the scene, it seems to have multiple candidates: VK_FORMAT_A2B10G10R10_UNORM_PACK32 and VK_FORMAT_B10G11R11_UFLOAT_PACK32. Developing a game engine, you probably know more about it, this is just the result of my research so far.

Currently, for the CNN, my biggest problem is selecting the training data masks automatically from the training data - I cannot (or rather don't want to) draw mask lines manually for potentially hundreds of screenshots. I've come to the conclusion that anti aliasing is not broken in ED per-se, actually it is quite good, at least texture AA works well. The real problem is specular/shading aliasing. Odyssey uses much higher detailed models, often using geometry instead of tessellation, and thus it made this problem much more prominent: On model edges, you get a lot of light or dark pixels wandering around during camera movement due to lighting changes of the angled and narrow surfaces, and even when the camera is not moving, there are dotted lines or lines with repeating luma gradients that don't merge with the model edge properly. Since the Cobra engine of that generation does not support TAA, the ED AA has no chance of properly distributing the render error over time. Instead of smoothing AA - which has been the wrong idea in the beginning - I'm now trying to train the CNN to reconstruct exactly that error back into a properly anti-aliased line according to the 16k target image (properly downsampled to the input image).

My first version of the CNN instead learned to reconstruct the overall smoothness of the lanczos downsampled 16k image instead of repairing the edges.

It's still a curiosity experiment, tho. No shader implementation exists yet. My current 11x11 context area is probably the biggest thing I should use before a simple full screen path shader causes texture bandwidth to become a bottleneck. Beyond that size, a compute shader probably becomes cheaper.

And yes, I noticed the file chooser. There are a few bugs or missing features I found around it. Not in the file chooser itself (it works well), but how vkShade behaves. I found those earlier but only took notes so far, did not post an issue yet. The only non-intuitive thing I've noticed so far is: The "Cancel" button is on the left, not on the right. But that's probably personal preference. ;-)

[ralgar]

SSAO requires the full screen space, but the high res screenshot tool in ED (Alt+F10) renders the frozen scene in multiple tiles to stitch together a 16k image. Due to this tiling, SSAO cannot be used.

That makes a lot more sense.

For my CNN, it could be helpful to dump the HDR buffer of the scene

That would be possible as well.

And yes, I noticed the file chooser. There are a few bugs or missing features I found around it. Not in the file chooser itself (it works well), but how vkShade behaves. I found those earlier but only took notes so far, did not post an issue yet. The only non-intuitive thing I've noticed so far is: The "Cancel" button is on the left, not on the right. But that's probably personal preference. ;-)

I'll be curious to see the issue. As for the cancel button, I'd actually prefer it on the right since that's the standard place for it, but the file chooser library seems to put it on the left by default. I wasn't too concerned about it at the time but, looking at it now, it does appear to be customizable, so that can definitely be changed.

[kakra]

So I think I found a way to automatically find the aliasing error mask for my CNN. I reactivated my rusty almost 25 years old algorithmic computer vision skills from university. The result looks promising: The filters now detect the 3x3 pixel core of shading errors properly, even follow rounded edges, smooth out the weights to the 11x11 mask, only select a few irrelevant edges, and throw away the rest (light and fog changes between two screenshots which the CNN should not learn). \o/

[kakra]

This evening, there's time to put these GUI changes to a test: My CNN now has learned to properly select and work only on the edges instead of just smoothing out the complete images or only smooth the textures. However, it has not learned to properly restore a smooth edge yet, instead it just smooths the pixels without removing the discrete steps and specular pixel errors, so it hasn't learned to detect the edge direction, and it needs a gain of 4 to even see a visible change. But this is good enough to create a shader from it, and then play around with uniform parameters live - thanks to your PR. I hope the shader will be fast enough. If performance suffers too much, I think I'm going to need a completely different idea. And I may have just not enough training data: Currently I'm only using 10 screenshots (real render plus an identical high res screenshot downscaled via lanczos), and additional 10 screenshots for learning evaluation.

[kakra]

I tested this a little bit but had less time than expected. I think, two ideas could be useful:

• A slider should have a value input, too. Otherwise, if the window is narrow, it's hard to hit an exact value. This feature probably needs keyboard interception to prevent the game from seeing the same keypresses.
• A useful addition would be to have a per-effect on/off toggle. This does not need to be stored or exist in the ini file.

[ralgar]

This evening, there's time to put these GUI changes to a test: My CNN now has learned to properly select and work only on the edges instead of just smoothing out the complete images or only smooth the textures. However, it has not learned to properly restore a smooth edge yet, instead it just smooths the pixels without removing the discrete steps and specular pixel errors, so it hasn't learned to detect the edge direction, and it needs a gain of 4 to even see a visible change. But this is good enough to create a shader from it, and then play around with uniform parameters live - thanks to your PR. I hope the shader will be fast enough. If performance suffers too much, I think I'm going to need a completely different idea. And I may have just not enough training data: Currently I'm only using 10 screenshots (real render plus an identical high res screenshot downscaled via lanczos), and additional 10 screenshots for learning evaluation.

Sounds very promising! From what I know about training, you're likely correct that more data points would be necessary. Most sources I've read seem to quote low-mid hundreds as a minimum. If this is true, you have my sympathies. I would not have the attention span for this. XD

A slider should have a value input, too. Otherwise, if the window is narrow, it's hard to hit an exact value. This feature probably needs keyboard interception to prevent the game
from seeing the same keypresses.

This should work with Ctrl + click, but it looks like there's an issue with how the input is handled on the ImGui side. Mod keys need to be reported to ImGui twice, once as a regular key, and again as a mod key. This is easy enough to fix.

A useful addition would be to have a per-effect on/off toggle. This does not need to be stored or exist in the ini file.

I agree, but it's out of scope for this PR and release. It actually should be stored in the INI, like all other state. This is basically what the delta between Techniques and TechniqueSorting allows ReShade to achieve, to maintain the effect order while being able to toggle effects freely. Very doable, but it requires some broad changes in effect handling, and should come after technique support so vkShade can just standardize on the ReShade preset syntax directly without going through multiple breaking changes in a row. All of this is in scope for the near future, though, just not right this second.

[kakra]

Sounds very promising! From what I know about training, you're likely correct that more data points would be necessary. Most sources I've read seem to quote low-mid hundreds as a minimum. If this is true, you have my sympathies. I would not have the attention span for this. XD

I put it to a real test to know how the complete pipeline behaves. Good thing: As a shader, it only takes around 1ms from the frame time budget at 4k, which is a lot less than letting ED render at supersampling resolutions. Bad thing: The CNN learned something but not reconstructing the edges. Instead it started to take shortcuts again, started to hate blue and white but left orange alone, and it ignored most edges except very special ones. I still have one or another idea to get around that.

This should work with Ctrl + click, but it looks like there's an issue with how the input is handled on the ImGui side. Mod keys need to be reported to ImGui twice, once as a regular key, and again as a mod key. This is easy enough to fix.

Let me know when to test again. Didn't know about ctrl+click.

I agree, but it's out of scope for this PR and release.

Sounds fine. I didn't expect that the state would have to be stored in the ini at some stage later anyways. Good to know, the scope decision looks right to me.