Formalize milestones for JOSS paper

[dannys4]

in the 7/15 meeting, @bonh brought up that it's a little misleading (and I agree) that we advertise both Python bindings and GPU support, but not GPU support on MParT in Python. This became an interesting question of feature creep-- what do we want to be done for the JOSS paper submission?

• Comprehensive bindings (how comprehensive?)
• GPU support
• GPU support in bindings
• Optimization framework
• Adaptive Transport Maps
• Adding examples highly driven by application to show in the paper (thanks @bonh !)
• other things?

[dannys4]

• I'm supportive of having bindings comprehensive enough so that you can easily make, optimize, and sample from a transport map in each binding. I'm a little skeptical of needing all the bindings (in, e.g., MultiIndex), at least at the JOSS stage working. Just enough to be able to reorder, etc.
• Obviously we already have GPU support
• I'm satisfied without GPU support in bindings in the first submission. I think we can always make a second submission somewhere else once we start using MParT for a sufficiently large problem to actually rigorously benchmark the GPU usage
• I don't believe that we should have the optimization framework in the submission
• I have no comment on ATM.
• I do not know what examples would be best for the submission, but I liked @michael-c-brennan 's idea of using it for transport-powered MCMC or something similar.

[rubiop]

• I am also supportive of having bindings that are just required to run examples to provide (pushing that point to another one)
• I trust you guys blindly for GPU stuff, I would say: if it is easy enough to have the compatibility let's put it in otherwise aim for a second paper
• I don't think we want optimization and ATM (which is tightly related) in the core library. Maybe we would want a tutorial example using ATM in the host language as we will most likely have examples with optimization.
• For examples we want, that's an open question to me and it is up to what kind of examples are usually provided in JOSS papers. I think we want low level examples to show how to manipulate main objects in every languages but maybe we also want some more targeted applications to show 'real' use. Maybe do the same type of things that has be done with MUQ? We might also want to provide examples from our one research but I am not sure if we want to do this before or after the JOSS submission.

[mparno]

Thinking more about the examples... I think it would also be good to include at least one example showing how known conditional independence structure can be used to define a multiindexset and ultimately a map. This doesn't need to be ATM, but some other graphical model. This would also be a chance for us to show a "high" dimensional example.

[rubiop]

Yes maybe trying to minimize custom functions in host languages is best for tutorials. For this kind of structure specific examples I bet @baptistar have some under his sleeves.

[mparno]

GPU support with the bindings is something I've been thinking about quite a bit. Currently, the issue is that the ParameterizedFunctionBase<MemorySpace> and ConditionalMapBase<MemorySpace> classes can only be given Kokkos::View objects with memory on the same Kokkos memory space. Because the bindings are defined to work with arrays defined on Host Memory (e.g., on the CPU), we don't currently have the ability to leverage GPU acceleration from the bindings. But, I think we should change that.

I propose that we remove the class-level template argument and instead overload the Evaluate, LogDeterminant, SetCoeffs, and gradient functions to accept views that either on the Host or the Device (assuming MParT was compiled with GPU support). If a host view is given, we can decide if it's worth copying to the device, copy it if necessary, and then run either the host function or device function. If a device view is given to the function, then it is always evaluated on the device. What do you guys think?

[dannys4]

I was chatting with michael about this and I thought almost the exact opposite thing. I was thinking we could just create a python function like toDevice that took in an eigen vector/matrix and returned a kokkos view into the device (or some wrapped object that's basically that), then you could template all the wrapping functions (e.g. MapFactoryWrapper) on the memory space and pretty easily instantiate both for device and host.

[dannys4]

I will also say that removing class-level templating would make the julia bindings significantly easier to write and support, so I support your idea with ulterior motives. Just throwing out an idea that I believe would be less work for us now.

[mparno]

We already have a ToDevice function in ArrayConversions.h, so that should be pretty easy. I'd be fine with that for now. If you were to wrap anything, it could by the StridedMatrix and StridedVector aliases defined in ArrayConversions.h, but I'm not sure you even need that. You could just add a lambda in the wrapper that does the copy and only requires the user to work with the language-native array object (e.g., numpy array).

[mparno]

Here are some more thoughts on the JOSS submission:

Comprehensive bindings (how comprehensive?)

I think just enough to easily solve our examples. To me, this involves MultiIndexSet, ParameterizedMapBase, ConditionalMapBase, TriangularMap, and MapFactory.

GPU support

Done

GPU support in bindings

I think we need this and the idea prposed by @dannys4 above should be relatively simple.

Optimization framework

Out of scope.

Adaptive Transport Maps

Out of scope. Also requires optimization unless we write some pretty extensive examples using language-specific optimizers.

Adding examples highly driven by application to show in the paper (thanks @bonh !)

Yes! I think the BOD example we have for Bayesian inference is great. Some sort of real observational data would be great for maps-from-samples. I also have some stress-strain data from the Sandia Fracture Challenge that could be a good option for monotone regression.

transport-powered MCMC

Obviously I would love to see this, but I'm not sure it's the most intuitive example for users to learn MParT. It would require a bit of infrastructure that's not really MParT-specific. Maybe a better option would be correcting samples from an approximate sample using importance sampling. This could be a small extension to the likelihood-free version of the BOD problem that @rubiop already put together.

[dannys4]

Examples for JOSS paper--
0 through 3 from SIAM annual (including BOD) + Stochastic volatility model

[bonh]

See MeasureTransport/MParT-examples#14 and MeasureTransport/MParT-examples#13

[bonh]

And MeasureTransport/MParT-examples#15, MeasureTransport/MParT-examples#16.