Skip to content

Decompose asyncio.IocpProactor._poll() into parts to improve integration with other event loops #154971

Description

@Oliver-Leigh

Feature or enhancement

Proposal:

This is a niche use case in the spirit of #110771 that has arisen in recent development of Toga.

GUI toolkits all have event loops, and it is highly desirable to be able to use Python asyncio calls in a GUI app. To this end, it necessary to make Python's asyncio event loop co-exist with the GUI's event loop and any platform specific issues.

On the Windows platform, a desirable property is to be able to listen for and process IOCP events independently. This can allow GUI-asyncio event loops to be reactive and avoid the need to polling.

The proposed change in asyncio.IocpProactor is to decompose asyncio.IocpProactor._poll() as follows:

def _process_completion_status(self, status):
    err, transferred, key, address = status
    try:
        f, ov, obj, callback = self._cache.pop(address)
    except KeyError:
        if self._loop.get_debug():
            self._loop.call_exception_handler({
                'message': ('GetQueuedCompletionStatus() returned an '
                            'unexpected event'),
                'status': ('err=%s transferred=%s key=%#x address=%#x'
                            % (err, transferred, key, address)),
            })

        # key is either zero, or it is used to return a pipe
        # handle which should be closed to avoid a leak.
        if key not in (0, _overlapped.INVALID_HANDLE_VALUE):
            _winapi.CloseHandle(key)
        return

    if obj in self._stopped_serving:
        f.cancel()
    # Don't call the callback if _register() already read the result or
    # if the overlapped has been cancelled
    elif not f.done():
        try:
            value = callback(transferred, key, ov)
        except OSError as e:
            f.set_exception(e)
            self._results.append(f)
        else:
            f.set_result(value)
            self._results.append(f)
        finally:
            f = None

def _poll(self, timeout=None):
    if timeout is None:
        ms = INFINITE
    elif timeout < 0:
        raise ValueError("negative timeout")
    else:
        # GetQueuedCompletionStatus() has a resolution of 1 millisecond,
        # round away from zero to wait *at least* timeout seconds.
        ms = math.ceil(timeout * 1e3)
        if ms >= INFINITE:
            raise ValueError("timeout too big")

    while True:
        status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)
        if status is None:
            break
        ms = 0

        self._process_completion_status(status)

    # Remove unregistered futures
    for ov in self._unregistered:
        self._cache.pop(ov.address, None)
    self._unregistered.clear()

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-asynciotype-refactorCode refactoring (with no changes in behavior)

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions