Test more packages' latest builds in upstream ci

[Sevans711]

Closes #1785

Overview

Adds geoviews (and more other packages, too) to the upstream ci job (via install-upstream.sh). Using the latest geoviews should be sufficient to fix #1785; the upstream ci is failing because it is testing the latest geoviews release (which is not compatible with the latest cartopy; see #1780) when it probably should be testing the latest version of geoviews instead (where a fix has already been merged; see holoviz/geoviews#884).

Closing the original issue only requires adding geoviews to the upstream ci job, but this PR adds more packages there, too. There have recently been other bugs related to the latest versions of packages including breaking changes (see, e.g., #1542, where some cartopy<0.26 plots don't work properly with matplotlib>=3.11). Including more packages in upstream ci may help spot this before the relevant releases actually occur.

Minor sidenote: moved the pip uninstall packages at start of install-upstream.sh into the conda remove command, because they were both being installed by conda. This comes from upstream-dev-ci.yml which uses ci/environment.yml which does not have a pip: ... block; everything there is installed via conda; confirmed by looking at the conda list output from an actual CI Upstream run (e.g.: https://github.com/UXARRAY/uxarray/actions/runs/35949787277/job/107475635076).

(Tiny sidenote: woops, misspelled the branch name! Fixing it means deleting the PR, though, so that typo is here to stay....)

PR Checklist

General

• An issue is created and linked
• Added appropriate labels (if your uxarray repo permissions allow it)
• Filled out Overview and Expected Usage (if applicable) sections

Testing & Benchmarking

• There is adequate test coverage of changes from this PR (add new tests if needed)
  • (Upstream CI workflow shows tests pass when run on this branch, see: https://github.com/UXARRAY/uxarray/actions/runs/36005027889/job/107650878616)
• [N/A] If this PR could affect performance, ran ASV benchmarks and confirmed they show expected behavior (add a new benchmark if necessary)

Documentation and Examples

• [N/A] Docstrings updated with any function changes, and included in all new functions
• [N/A] User (public) functions added to docs/api.rst; internal (private) function names start with an underscore (_)
• [N/A] If touched any notebook files, cleared the output of all cells before committing
• [N/A] If added new notebook files, put into appropriate directories and referenced in appropriate files

AI Disclosure

AI Usage: GitHub Copilot's inline code suggestions, plus asked claude about how to decide whether packages belong in the conda remove or pip uninstall block.

• I have tested and take responsibility for all AI-generated content in my PR.

[dylannelson]

Hey I tried to follow the order of these commands and may see a potential problem. Breaking some of this file into pieces and testing each line individually can show an error where matplotlib is installed

PackagesNotFoundError: The following packages are missing from the target environment:
  - matplotlib

but I think the reason this doesn't come up during the CI run is because this isn't an error, and just a warning. I think this causes some/all of the rest of the code to not run at all, and fail silently. I think if it's changed to matplotlib-base it works as intended though. Or if you want to test this, I believe if you add set -e to the top of the script, it will result in the error above, and can warn you of future ones.

The reason it seems to work with matplotlib-base is because /ci/environment.yml requires matplotlib-base<3.11 and calling conda remove matplotlib doesn't find matplotlib-base

I think you can reproduce with the set -e or by making the env yourself and trying to run the commands:

conda env create -n upstream-check -f ci/environment.yml
conda activate upstream-check
conda remove -y --force matplotlib

or even the full

conda remove -y --force antimeridian cartopy dask datashader distributed matplotlib holoviews hvplot geoviews pandas pyarrow requests scikit-learn scipy shapely spatialpandas xarray

assuming your terminal is at the location of this branch's environment.yml

I'm also not seeing an issue with uninstalling pooch as part of the conda list. If I create an env and run
conda remove -y --force pooch it seems to work. Though I'm not as sure what you saw while running that warranted the change in the first place, so I'm not as certain about this one being a concern. For me though it does appear on conda list:

pooch                               1.9.0                  pyhd8ed1ab_0             conda-forge

and it works fine uninstalling for me

conda remove -y --force pooch
3 channel Terms of Service accepted

## Package Plan ##

  environment location: ...

  removed specs:
    - pooch


The following packages will be REMOVED:

  pooch-1.9.0-pyhd8ed1ab_0



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Hoping these aren't just a local quirk/windows machine bug too, if so, then carry on, haha.

[Sevans711]

Good finds, I fixed those two issues (I think) by:

1. swapping the spelling to matplotlib-base and matplotlib-inline instead of just matplotlib, during the conda remove command
2. adding pooch into the conda remove command. I originally put pooch separately because I was mistakenly looking at the second conda list during the job, instead of the first one (which happens right before the install-upstream.sh script runs). It does indeed appear in the first one.

Now trying to debug what appears to be a completely separate issue, occurring while collecting the tests:

__________________ ERROR collecting test/io/test_geopandas.py __________________
ImportError while importing test module '/home/runner/work/uxarray/uxarray/test/io/test_geopandas.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../micromamba/envs/uxarray_build/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
test/io/test_geopandas.py:1: in <module>
    import geopandas as gpd
../../../micromamba/envs/uxarray_build/lib/python3.13/site-packages/geopandas/__init__.py:3: in <module>
    from geopandas.geoseries import GeoSeries
../../../micromamba/envs/uxarray_build/lib/python3.13/site-packages/geopandas/geoseries.py:12: in <module>
    import shapely
../../../micromamba/envs/uxarray_build/lib/python3.13/site-packages/shapely/__init__.py:20: in <module>
    from shapely.lib import GEOSException
E   ImportError: libgeos_c.so.1: cannot open shared object file: No such file or directory

This is also occurring on main, so at least it wasn't introduced here. But, it wasn't occurring on main when #1785 (and this PR) were first created. Summarizing latest upstream CI runs here:

• Night of Sept 22 (and previous nights before that): passed.
• Night of Sept 23: failed due to "lon_0" KeyError from geoviews
• Night of Sept 24: failed due to "lon_0" KeyError from geoviews
• Night of Sept 25: failed due to "lon_0" KeyError from geoviews
• Night of Sept 26: failed while collecting tests; ImportError: libgeos_c.so.1: cannot open shared object file
• Night of Sept 27: failed while collecting tests; ImportError: libgeos_c.so.1: cannot open shared object file
• Night of Sept 27 (i.e. last night): failed while collecting tests; ImportError: libgeos_c.so.1: cannot open shared object file

Hoping to get a chance to look into this further later today; will follow up here with any progress.