Talks: Overcoming GIL with subinterpreters and immutability

Saturday - May 18th, 2024 11:15 a.m.-11:45 a.m. in Ballroom BC

Presented by:

Description

The talk will focus on a novel idea of efficient structured data sharing between Python subinterpreters. I'll be presenting a new Python framework "memhive", which implements a worker pool of subinterpreters, efficient RPC mechanism between them, async/await-ready API, as well as fundamental data structures ( tuples, mappings and scalar types).

I'll demonstrate that it's possible to unlock true parallelism with subinterpreters without paying the overhead of data serialization. I'll explain how this is possible and what algorithms under the hood drive this.