Limiting test data distributed within astroquery

[bmorris3]

I'm working with JupyterLite, where I'd like to use astroquery. Performance for starting a JupyterLite kernel depends on the memory footprint of the dependencies.

The uncompressed size of astroquery is 28 MB. That’s smaller than astropy (45 MB), but similar to numpy (32 MB), botocore (25M), and jupyterlab (22 MB). I wonder if we could bring that number down.

I'm looking within lib/python3.13/site-packages/astroquery and running du -hs * | sort -hr to check the sizes of each module. The top three, accounting for more than half of the size of astroquery, are:

8.4M	esa
3.5M	ipac
2.8M	mast

Breakdown by submodule

ESA

5.9M	esa/integral/tests
1.0M	esa/hubble/tests
252K	esa/euclid/tests

70% of ESA module’s weight is in astroquery/esa/integral/tests/data, which has two test files, totaling 5.8 MB.

There are many subdirs within esa with much smaller sizes; note the big difference between the sizes of the last two.

ipac

952K	ipac/nexsci/nasa_exoplanet_archive/tests
880K	ipac/irsa/tests/data
408K	ipac/irsa/irsa_dust/tests

mast

2.3M	mast/tests

What can we do to limit astroquery's footprint?

[ManonMarchand]

I'd also find this extremely useful

[keflavich]

The INTEGRAL tests are the obvious first target; the tarballs can be shrunk to have placeholder files because the tests don't use them. Maybe Hubble and Euclid have similar easy fixes?

@jespinosaar Do you want to take the lead on that? (you mentioned on Slack that you'd look into it).

The MAST files are less clear to me, they are giant metadata files.

[orionlee]

Are there requirements that tests / test data need to included in the wheel ?

If not, there should be ways to exclude them (tweaking pyproject.toml, or move the tests files into a parallel directory hierarchy).

[jespinosaar]

The INTEGRAL tests are the obvious first target; the tarballs can be shrunk to have placeholder files because the tests don't use them. Maybe Hubble and Euclid have similar easy fixes?

@jespinosaar Do you want to take the lead on that? (you mentioned on Slack that you'd look into it).

The MAST files are less clear to me, they are giant metadata files.

Hi all! And yes, I will take care of this. We will try to reduce the test samples.