Understanding the Python GIL on HackerTube: recommended by 4 Hacker News commenters across 4 separate months, 2013–2023.
talk · intermediate · dated
David Beazley's PyCon 2010 talk explains how the Python Global Interpreter Lock works and shows experiments where multithreaded code runs slower, even on more cores, due to lock contention and context-switching overhead.
The canonical recommendation for understanding the GIL, praised both for its clarity and for how well it explains a deep technical topic.
“My favorite example is Beazley's GIL talk”
8 Hacker News comments link this video. Each one links back to its thread.
"The point of such a presentation is to take a piece of code that the presenter has understood deeply and for them to help the audience understand the core ideas..." I do get lots of value out of that. My favorite example is Beazley's GIL talk: ▶ this video
in “Code is not Literature” · View on HN
I am not expert, and this might be dated information, but historically there is a "thrashing" introduced between the GIL and OS scheduled threads which has surprising performance implications. David Beazley does a much better illustrating the nuances than I ever could: ▶ this video Including a wild phenomenon where a threaded Python program's performance can degrade by increasing number of CPU cores available to a system. Coroutines avoid this by not requiring individual thread scheduling by the OS. So they effectively release the GIL like threads on IO but avoid OS level thread switching.
in “How to choose the right Python concurrency API” · View on HN
David probably isnt the best guy for that. But here's one of the famous ones on GIL. ▶ this video Beware though, his talks are known to be devastating to the programmer, resulting in serious head injuries, trauma, headaches and other fun stuff.
in “Code with Mu: a simple Python editor for beginner programmers” · View on HN
"I never actually got around to learning -- or even thinking about!-- what exactly the GIL is" David Beazley does a good job on explaining the GIL. There are multiple videos of his GIL talks on YouTube. This one, for instance: ▶ this video He has a page about this at [link]
in “Python's Hardest Problem (2012)” · View on HN
There's a GIL presentation [1] from 2010 which shows Python 3.2 on a 1 CPU machine was faster than on a dual core. In all, there might be reason to expect GILless python to be faster single core in certain scenario's. [1] ▶ this video
in “Progress on No-GIL CPython” · View on HN
> > Python's multiprocessing library is needed to overcome the GIL > No it's not, just use threads. I just wanted to expand on this a little to describe some of the downsides to threads in Python. Multi-threaded logic can be (and often is) slower than single-threaded logic because threading introduces overhead of lock contention and context switching. David Beazley did a talk illustrating this in 2010: ▶ this video He also did a great talk about coroutines in 2015 where he explores threading and coroutines a bit more: [link] In workloads that are often "blocked" like network calls our I/O bound work loads, threads can provide similar benefits to coroutines but with overhead. Coroutines seek to provide the same benefit without as much overhead (no lock contention, fewer context switches by the kernel). It's probably not the right guidelines for everyone but I generally use these when thinking about concurrency (and pseudo-concurrency) in Python: - Coroutines where I can. - Multi-processing where I need real concurrency. - Never threads.
in “Async Python is not faster” · View on HN
I would recommend you use coroutines rather than multiprocessing.dummy for balancing blocking operations (like IO). I believe multiprocessing.dummy is just a wrapper for threading which has context switching overhead [1]. Python3 has asyncio as part of the stdlib but for Python2 there are a variety of coroutine libraries out there (I think the most widely recognized being gevent). [1] ▶ this video
in “The multiprocessing module in Python” · View on HN
> If you want speed you go to any other scripting language (other than Ruby). Ruby has historically had the same issues. Most Pythonistas I know aren't so evangelical. It's mostly a question of how to go about integrating C/C++ code. Many people complaining about the GIL (and the like) have some naive microbenchmark, don't understand the trade-offs/limitations of their runtime etc. That doesn't mean critique isn't important and required, but it's going to be better when it's properly researched and improves on the body of work out there ( ▶ this video ). > Transpiling it to another language like Google did, shows that the underlying technology is not worth much. How is this any general indicator of the worth of the language? It shows for some cases, that Google thought this was a worthwhile investment. Google has experimented for a long time with ways to improve how Python code can be run. They ran the Unladen Swallow project, but spent more time on LLVM issues at the time making it infeasible to continue the project. They'll discontinue one path and try another. None of this is really a commentary from Google on CPython, the community, or the value that it has for most people. The people working on this stuff interact in a pretty friendly basis.
in “Problems I Have with Python” · View on HN