Ensure diversity sampling instead of random (proportionate) sampling

[mlopatka]

python_mozetl/mozetl/taar/taar_similarity.py

Line 131 in 32d78c3

donor_df = clusters_df.sampleBy("prediction",

The diversity of the donor pool is only ensured by the assumption that higher level clustering is substantially diverse. This could be improved by verification of cross-cluster diversity in the addons space.

[mlopatka]

This also comes back when we specify a proportionate sampling strategy here:

python_mozetl/mozetl/taar/taar_similarity.py

Line 168 in 491fbda

donor_df = clusters_df.sampleBy(

A solution here would be to specify weights that emphasize specific (niche) cluster representation in the final sample without compromising the non-addon diversity of "large" cluster sampling.

Even an inverse of the current strategy could be evaluated.

[mlopatka]

@Dexterp37 can you assign this issue to me please? I have insufficient privileges to grab it :|