[PATCH] hgdemandimport: avoid Python 3.15 errors for heapq and copy

[Manuel Jacob]

On 28/03/2026 17.13, Mads Kiilerich wrote:
> # HG changeset patch
> # User Mads Kiilerich <mads at kiilerich.com>
> # Date 1774706053 -3600
> #      Sat Mar 28 14:54:13 2026 +0100
> # Branch stable
> # Node ID 4ef356e1dd1f573af87bc4ab5aa3d1a0be925346
> # Parent  1afb8f260d18a38a94e81398a778f9430f103ba8
> hgdemandimport: avoid Python 3.15 errors for heapq and copy
> 
> Got:
> 
>      ImportError: cannot import name 'nlargest' from 'heapq'
> 
> The new PEP 810 lazy imports might enable new ways of doing things. In the
> future.
> 
> diff --git a/hgdemandimport/__init__.py b/hgdemandimport/__init__.py
> --- a/hgdemandimport/__init__.py
> +++ b/hgdemandimport/__init__.py
> @@ -65,6 +65,8 @@ IGNORES = {
>       'warnings',
>       'threading',
>       'collections.abc',
> +    'heapq',
> +    'copy',

When looking at the full traceback, it looked suspicious. Why is an 
attribute missing from a module when there is no import cycle? At the 
point where `difflib` can’t import `nlargest` from `heapq`, `heapq` has 
all the other attributes (including `nsmallest`, which is defined right 
before `nlargest`). `heapq.__dict__['nsmallest']` is an ordinary 
function, while `heapq.__dict__['nlargest']` is an object of type 
`lazy_import`, which is a new type introduced to Python 3.15 by the 
implementation of PEP 810 lazy imports. The attribute was set at [1] (I 
found this place with rr by setting a watchpoint on the dict entry for 
`nlargest` in `heapq.__dict__` and going backwards). `attrs_updated` 
contains only `nlargest`. The difference between `nlargest` and the 
other attributes of `heapq` is that the former is PEP 810-lazily 
imported by standard library modules (e.g. `collections`).

It turns out that our way of doing lazy imports (using LazyLoader) and 
the PEP 810 way of doing lazy imports don’t work well together. Here’s a 
minimized reproducer for the problem:

```
import importlib.util
import sys

class LazyFinder:
     def find_spec(self, fullname, path, target=None):
         if fullname in ["heapq"]:
             for finder in sys.meta_path:
                 if finder is not self:
                     spec = finder.find_spec(fullname, path, target)
                     if spec is not None:
                         spec.loader = 
importlib.util.LazyLoader(spec.loader)
                         return spec

sys.meta_path.insert(0, LazyFinder())

import collections
import heapq

heapq.nsmallest
heapq.nlargest
```

It is possible to turn lazy imports off or on completely (the default is 
that only imports explicitly prefixed with `lazy` are lazy) [2]. There 
is ongoing discussion on whether the standard library should rely on 
lazy imports or not (e.g. in case of circular imports) [3].

I suggest the following:
- In the stable branch, we turn PEP 810 lazy imports off, at least when 
hgdemandimport is enabled. I see no reason to disable PEP 810 lazy 
imports if hgdemandimport is disabled.
- In the default branch, we set the global lazy imports mode to "all" if 
available and disable hgdemandimport in that case. In a quick test, most 
things seemed to work and there was a small startup speedup over 
hgdemandimport (around 5%).

[1] 
https://github.com/python/cpython/blob/4d0e8ee649ceff96b130e1676a73c20c469624a9/Lib/importlib/util.py#L218
[2] https://docs.python.org/3.15/library/sys.html#sys.set_lazy_imports
[3] https://lwn.net/Articles/1061112/