[Bug 6932] New: recursion depth exceeded with lxml importing extension

[mercurial-bugs at mercurial-scm.org]

https://bz.mercurial-scm.org/show_bug.cgi?id=6932

            Bug ID: 6932
           Summary: recursion depth exceeded with lxml importing extension
           Product: Mercurial
           Version: 6.8
          Hardware: PC
                OS: Linux
            Status: UNCONFIRMED
          Severity: bug
          Priority: wish
         Component: Mercurial
          Assignee: bugzilla at mercurial-scm.org
          Reporter: jakub.kulik at oracle.com
                CC: mercurial-devel at mercurial-scm.org
    Python Version: ---

Created attachment 2174
  --> https://bz.mercurial-scm.org/attachment.cgi?id=2174&action=edit
potential fix

Hi. When running mercurial 6.8 with Python 3.11 and the following extension:

> cat /tmp/xml.py
import lxml.etree as etree
parser = etree.XMLParser()

with `hg --config extensions.hgext.xml=/tmp/xml.py status --traceback`, the
command fails with the following:

*** failed to import extension "hgext.xml" from /tmp/xml.py: maximum recursion
depth exceeded while calling a Python object
Traceback (most recent call last):                                             
  File "/usr/lib/python3.11/vendor-packages/mercurial/extensions.py", line 305,
in loadall
    load(ui, name, path, loadingtime)
  File "/usr/lib/python3.11/vendor-packages/mercurial/extensions.py", line 212,
in load
    mod = _importext(name, path, bind(_reportimporterror, ui))                  
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/vendor-packages/mercurial/extensions.py", line 119,
in _importext
    mod = loadpath(path, 'hgext.%s' % name)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/vendor-packages/mercurial/extensions.py", line 100,
in loadpath
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/xml.py", line 2, in <module>
    parser = etree.XMLParser()
             ^^^^^^^^^^^^^^^
  File "<frozen importlib.util>", line 261, in __getattribute__
  File "<frozen importlib._bootstrap_external>", line 1241, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "src/lxml/etree.pyx", line 117, in init lxml.etree
  File "<frozen importlib.util>", line 282, in __delattr__
  File "<frozen importlib.util>", line 282, in __delattr__
  File "<frozen importlib.util>", line 282, in __delattr__
  [Previous line repeated 967 more times]
  File "<frozen importlib.util>", line 281, in __delattr__
  File "<frozen importlib.util>", line 228, in __getattribute__
RecursionError: maximum recursion depth exceeded while calling a Python object


The issue seems to be the lazy loading; when it's disabled with
`HGDEMANDIMPORT=disable`, it works as expected. I found out that
`_LazyModule.__getattribute__()` in importlib.util (which mercurial overloads)
is called repeatedly with `__getattribute__` and `_ImmutableMapping` as
arguments until it reaches the recursion limit (`_ImmutableMapping` is from
pylxml). And when I added 'lxml.etree' into the `IGNORES` list in
`hgdemandimport.__init__.py` it started working again.

I tested it with pylxml 4.9.3 and 5.3.0, and there is no difference in what is
happening.

I am attaching a simple patch that, though I am not sure whether this is the
best/proper fix or just a workaround.


We originally encountered this issue on Solaris (mercurial 6.8 and Python
3.11), but I can also reproduce the same issue on Fedora Linux (6.7.4 and
Python 3.12). It does not happen with Python 3.9.

-- 
You are receiving this mail because:
You are on the CC list for the bug.