Skip to content

docs: install the published wheel in notebooks 02 and 03 - #197

Merged
shivamlalakiya merged 1 commit into
mainfrom
docs/notebook-install-cells
Sep 8, 2026
Merged

docs: install the published wheel in notebooks 02 and 03#197
shivamlalakiya merged 1 commit into
mainfrom
docs/notebook-install-cells

Conversation

@shivamlalakiya

Copy link
Copy Markdown
Contributor

Do not merge until 0.7.1 is on PyPI. The draft release for v0.7.1 is created and waiting on the maintainer's Publish release click. Until that click lands, pip install "philanthropy[viz]>=0.7.1" has nothing to resolve to on a fresh machine, and merging this would break the Colab badge in the window between merge and publish. Right now the git+main fallback on main still works in fresh Colab, so there is no urgency to land this early.

The bug

Notebooks 02 and 03 import philanthropy.datasets.make_donor_panel, which the published 0.7.0 wheel does not contain. Verified in a clean venv from a neutral cwd, because the repository root shadows site-packages and an earlier check run from the root reported the opposite:

$ ./v070/bin/python -c "import philanthropy.datasets as d; print(sorted(d.__all__))"
['fetch_kdd98_donors', 'generate_synthetic_donor_data', 'load_ciob_fundraising']

So both notebooks fell into this:

try:
    from philanthropy.datasets import make_donor_panel
except ImportError:
    !pip install -q "philanthropy @ git+https://github.com/PhilanthroPy-Project/PhilanthroPy@main"
    from philanthropy.datasets import make_donor_panel

That cannot work in-process. The failed from philanthropy.datasets import make_donor_panel leaves the already-imported philanthropy.datasets module object cached in sys.modules, so the second from ... import after a successful install re-reads the same stale module and raises the identical ImportError. The fallback only works where philanthropy is absent from the environment entirely, which is fresh Colab and nowhere else.

Two consequences:

  1. A reader who followed the README's pip install philanthropy and then opened a notebook locally got a hard failure, on the two notebooks that demonstrate the package's central argument.
  2. The "zero install, try it now" Colab badge ran an unreleased main snapshot rather than the archived release paper.md points at. That is the one a JOSS reviewer sees.

The fix

# Colab and other fresh environments only; an existing install satisfies this
# and pip exits without touching the network.
!pip install -q "philanthropy[viz]>=0.7.1"

from philanthropy.datasets import make_donor_panel

import philanthropy
print("philanthropy", philanthropy.__version__)

Unconditional rather than guarded, because the guard was the bug. In fresh Colab this installs 0.7.1 from PyPI before anything imports philanthropy, so sys.modules is clean. In a local checkout or in CI the requirement is already satisfied by the editable install, so pip short-circuits against installed metadata and never queries the index.

Notebook 01 is deliberately untouched. Its guard is try: import philanthropy, which succeeds against any published release, so it never misfires and the in-process staleness problem does not arise there. Changing it would be churn.

What is verified, and what is not

$ python -m pytest --nbmake examples/notebooks -q --no-cov
3 passed

$ python -m flake8 philanthropy tests examples
(clean)

Being explicit about the gap rather than letting a later reader assume it away, because plan.md D.1 says never to anticipate what a notebook does against the published wheel: that nbmake run does not exercise the PyPI path. The editable 0.7.1 install already satisfies >=0.7.1, so pip short-circuits and the wheel is never fetched. CI has the same blind spot, because CI installs the working tree. The only real check is pytest --nbmake in a clean venv from a neutral cwd after the wheel exists on PyPI, and that is a post-publish step, not something this PR can carry.

Sequence, so nothing is verified out of order:

  1. Maintainer clicks Publish release on the v0.7.1 draft. publish.yml uploads to PyPI.
  2. Confirm pip install philanthropy==0.7.1 in a clean venv from a neutral cwd, and that datasets.__all__ now contains make_donor_panel.
  3. Run pytest --nbmake examples/notebooks against that venv.
  4. Then merge this.

Also worth knowing: pypi-smoke.yml runs Mondays at 12:00 UTC against the published wheel on Linux, macOS and Windows, so it becomes an independent check on step 2 without anyone having to remember it.

Both notebooks imported datasets.make_donor_panel, absent from the 0.7.0
wheel, and fell back to a try/except ImportError that pip-installed from
git+main. That fallback cannot work in-process: the failed import leaves
the stale philanthropy.datasets module cached in sys.modules, so the
re-import after a successful install raises the same ImportError. It only
worked where philanthropy was absent entirely, which is fresh Colab, so a
reader who followed the README's pip install and then opened a notebook
locally got a hard failure, and the Colab badge ran an unreleased snapshot
rather than the release paper.md points at.

Replaced with a plain pip install -q "philanthropy[viz]>=0.7.1". 0.7.1
satisfies that from PyPI, and an existing install satisfies it without a
network call, so a local checkout and CI are unaffected.

Notebook 01 keeps its try/except deliberately: a bare import philanthropy
succeeds against any published release, so its guard never misfires and
the in-process problem does not arise there.

pytest --nbmake examples/notebooks: 3 passed. That run does NOT verify the
PyPI path, because the editable 0.7.1 install already satisfies the
requirement and pip short-circuits. The wheel path can only be verified
after 0.7.1 is published, from a neutral cwd in a clean venv, which is why
this stays unmerged until then.
@shivamlalakiya

Copy link
Copy Markdown
Contributor Author

0.7.1 is on PyPI, so the gate on this PR is satisfied and the verification the body said could not be done before publish has now been done properly: clean venv, published wheel, neutral cwd.

$ python -c "import urllib.request, json; print(json.load(urllib.request.urlopen('https://pypi.org/pypi/philanthropy/json'))['info']['version'])"
0.7.1

$ python -m venv v071 && ./v071/bin/pip install "philanthropy[viz]==0.7.1"
$ ./v071/bin/python -c "..."
file /private/tmp/.../v071/lib/python3.13/site-packages/philanthropy/__init__.py
version 0.7.1
make_donor_panel True
GiftIntervalCalibrator True
interval_score True
panel keys ['donors', 'gifts']

philanthropy.__file__ resolves inside site-packages, which is what makes this a test of the wheel rather than a second test of the working tree.

Then the notebooks themselves, run from a scratch directory outside the repository so the checkout cannot shadow the install:

$ cd /private/tmp/.../nb && /private/tmp/.../v071/bin/python -m pytest --nbmake . -q
3 passed in 19.26s

That is all three notebooks, including 01, executing end to end against the published 0.7.1 wheel with the new unconditional pip install -q "philanthropy[viz]>=0.7.1" cell. The install resolves, make_donor_panel imports, and neither 02 nor 03 touches git+main any more.

CI on this PR is green across all fifteen checks on 1584ab0, including the nbmake leg of the lint job.

Merging.

@shivamlalakiya
shivamlalakiya merged commit fd873d5 into main Sep 8, 2026
15 checks passed
@shivamlalakiya
shivamlalakiya deleted the docs/notebook-install-cells branch September 8, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant