Skip to content

xnetcdf and 2-d lat/lon coordinates - #963

Merged
davidhassell merged 47 commits into
NCAS-CMS:mainfrom
davidhassell:rotated-pole
Aug 28, 2026
Merged

xnetcdf and 2-d lat/lon coordinates#963
davidhassell merged 47 commits into
NCAS-CMS:mainfrom
davidhassell:rotated-pole

Conversation

@davidhassell

Copy link
Copy Markdown
Collaborator

Fixes #961 and #962

@davidhassell davidhassell added dataset read Relating to reading datasets zarr Related to Zarr datasets xarray Related to xarray datasets regridding Relating to regridding operations labels Aug 26, 2026

@sadielbartholomew sadielbartholomew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great overall. Some minor comments as usual - see in-line and the more general comments below. There are some merge confilcts to address also, but once all has been considered, happy for you to merge.

  • Though the umread C library files & headers have been deleted, the umread_lib directory still remains on this branch and it would be good to tidy by removing it. I think to do that we'd have to remove reference to it (e.g. to the built .o object files locations) from the .gitignore, commit that, and then clean and rmdir or similar. Could do this as follow-on work, but if you're up to include it in the PR that would sort it to nicely blitz umread in this PR.
    (Possibly the directory remains locally for me due to my having the built object files in the sub-directory c-lib, but I think the .gitignore would also prevent it from going.

  • There is a pyproj warning emerging from the test_Field_2d_create_latlon_coordinates_albers_equal_area which could be important:

    test_Field_2d_create_latlon_coordinates_albers_equal_area (__main__.LatLon2dTest.test_Field_2d_create_latlon_coordinates_albers_equal_area)
    Test albers_equal_area. ... /home/slb93/miniconda3/envs/cf-env-314/lib/python3.14/site-packages/pyproj/crs/crs.py:1295: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
      proj = self._crs.to_proj4(version=version)
  • pyproj should be listed in the package files and docs as an optional dependency, at least - but (thinking about the imminent release) can you confirm it isn't intended as a compulsory one (it isn't fully clear without some investigation how widespread the grid mapping logic is to the codebase)?

Comment thread cf/mixin/fielddomain.py Outdated
Comment thread cf/test/test_kerchunk.py Outdated
Comment thread cf/test/test_kerchunk.py Outdated
Comment thread Changelog.rst Outdated
(https://github.com/NCAS-CMS/cf-python/issues/961)
* Deprecated keyword to `cfdm.read`: ``netcdf_backend``
(https://github.com/NCAS-CMS/cf-python/issues/961)
* Dataset reads are now entirely managed by `xnetcdf` (via

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is umfive optional for cf-python as it now is for cfdm? In which case umfive is also relevant when it comes to reading. So it's more a team effort between xnetcdf and umfive, really.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point: d99f91e

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this has made me realise, since I was thinking more of how umfive is an optional dependency to cfdm now: umfive is set as an optional dependency to cfdm but having checked is also a hard dependency to xnetcdf, therefore cfdm already has it as a indirect hard dependency, so there is redundancy (and its also misleading) in having umfive stated as optional for cfdm. I'll raise an Issue over on the cfdm repo.

@sadielbartholomew sadielbartholomew Aug 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But thanks, the update commit in question is a nice one - makes it clearer that xnetcdf handles the backends. So marking as resolved for this PR.

Comment thread docs/source/check_docs_api_coverage.py
Comment thread cf/mixin/utils/grid_mapping.py
Comment thread cf/mixin/utils/grid_mapping.py
Comment thread cf/mixin/utils/grid_mapping.py Outdated
Comment thread cf/mixin/utils/grid_mapping.py
Comment thread cf/mixin/utils/grid_mapping.py Outdated
davidhassell and others added 14 commits August 28, 2026 10:55
Co-authored-by: Sadie L. Bartholomew <sadie.bartholomew@ncas.ac.uk>
Co-authored-by: Sadie L. Bartholomew <sadie.bartholomew@ncas.ac.uk>
Co-authored-by: Sadie L. Bartholomew <sadie.bartholomew@ncas.ac.uk>
Co-authored-by: Sadie L. Bartholomew <sadie.bartholomew@ncas.ac.uk>
@davidhassell

Copy link
Copy Markdown
Collaborator Author

Though the umread C library files & headers have been deleted, the umread_lib directory still remains on this branch and it would be good to tidy by removing it.

Hopefully a13e32c will do that

pyproj should be listed in the package files and docs as an optional dependency, at least - but (thinking about the imminent release) can you confirm it isn't intended as a compulsory one (it isn't fully clear without some investigation how widespread the grid mapping logic is to the codebase)?

It is optional: 6aca0fd

There is a pyproj warning emerging from the test_Field_2d_create_latlon_coordinates_albers_equal_area which could be important

I see that this is wholly benign - we're only calling proj.to_dict() (from test funtion field_paris) to get the projection name. 77997f7 refactors the line so the warning disappears :)

@davidhassell

davidhassell commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Hi Sadie.

Fantastic. All comments resolved, or responded to. I'm just re-running the whole test suite ... and all tests pass :)

@sadielbartholomew

Copy link
Copy Markdown
Member

Hopefully a13e32c will do that

Nice. We might need a final rm umread_lib or similar but that's certainly good enough (can clean it up that way later at release time if you prefer).

It is optional: 6aca0fd

Aha sorry I should have been more specific and said it is missing in an 'extras_require' category in the setup.py where we can track the purpose:

cf-python/setup.py

Lines 240 to 263 in f7e655d

extras_require = {
"required C libraries": ["udunits2==2.2.25"],
"regridding": ["esmpy", "ESMF>=8.0"],
"convolution filters, derivatives, relative vorticity": ["scipy>=1.1.0"],
"subspacing with multi-dimensional construct cells": ["matplotlib>=3.0.0"],
"documentation": [
"sphinx>=7.0.0",
"sphinx-copybutton",
"sphinx-toggleprompt",
"sphinxcontrib-spelling",
],
"pre-commit hooks": [
"pre-commit",
"black",
"docformatter",
"flake8",
],
"zarr": [
"zarr>=3.1.3",
],
"xarray": [
"xarray>=2026.7.0",
],
}

Ideally you can put it there in an appropriate place, else I can do it for the release.

I see that this is wholly benign - we're only calling proj.to_dict() (from test funtion field_paris) to get the projection name. 77997f7 refactors the line so the warning disappears :)

Ah good to know, thanks for investigating and preventing it. All good there then.

Doing a final sanity check then you can merge!

@sadielbartholomew sadielbartholomew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All feedback addressed, thanks, though feel free to tackle the minor tweaks mentioned in my latest here: #963 (comment), if not I can do those alongside the release checklist, before merging. Good to merge either way.

@davidhassell

Copy link
Copy Markdown
Collaborator Author

setup.py modified: bf4c4d7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dataset read Relating to reading datasets enhancement New feature or request regridding Relating to regridding operations um/pp Relating to UM or PP format files xarray Related to xarray datasets zarr Related to Zarr datasets

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consolidate cf.read backends with xnetcdf

2 participants