User-level threads....... with threads. - Paul Turner - Google

The term to search for prior art is user-mode scheduler / scheduling. Basically you add additional kernel features that allow making some scheduling decisions in the application, it's not something you'd just build on a vanilla pthreads implementation. Examples: Windows 7 UMS: [link] google3 fibers / switchto: ▶ this video

in “Async hazard: MMAP is blocking IO” · View on HN

I think it's also worth mentioning that for certain specific use cases coroutines are much more efficient than a full goroutine, because switching to/from a coroutine doesn't require context switching or rescheduling anything. If you have two cooperating tasks that are logically synchronous anyway (e.g. an iterator) it's much more efficient to just run everything on the same CPU because the kernel doesn't have to reschedule anything or power down/wake up CPU cores, and the data is in the right CPU caches, so you'll get better cache latency and hit rates. With goroutines this may happen anyway, but it's not guaranteed and at the minimum you have the overhead of going through the Go runtime goroutine scheduler which is fast, but not as fast as just executing a different code context in the same goroutine. Coroutines offer more predictable scheduling behavior because you know that task A is switching directly to task B, whereas otherwise the goroutine scheduler could switch to another available task that wants to run. The last section of the blog post goes into this, where Russ shows that an optimized runtime implementation of coroutines is 10x faster than emulating them with goroutines. Google has internal kernel patches that implement cooperating multithreading this way (internally they're called fibers), and the patches exist for precisely this reason: better latency and more predictable scheduling behavior. Paul Turner gave a presentation about this at LPC ~10 years ago that

in “Coroutines for Go” · View on HN

Here: ▶ this video Jump to the 15-minute mark for the three new system calls, switchto_wait, switchto_resume, switchto_switch.

in “Moving Google toward the mainline” · View on HN

Thanks for the link, it is explained in the linked video: ▶ this video They explain it around 15:01 - google added it's own syscall switchto_thread - that puts the current thread to sleep and is switching to the argument thread id. (and some other calls too). That one helps with cutting down latency in inter thread calls for m:n threading. The real effort is to make latency for individual application requests predictable, while keeping it low.

in “Moving Google toward the mainline” · View on HN

Oh, yes. I've been looking forward to this since I saw ▶ this video (User-level threads....... with threads. - Paul Turner - Google) in 2013.

in “Fibers, Oh My” · View on HN

I don't think "fiber" has a universally agreed definition, which makes these things super confusing to read without clarification. It's hard to come up with a good definition and stick to it because people keep coming up with new variations. An interesting case is a different Google fiber library, not open-sourced but described in a public talk. [1] Real kernel threads but (mostly) scheduled in userspace via new syscalls [2] that suspend the current thread and unsuspend a userspace-chosen other thread. The idea is that most of the overhead of kernel threads is the scheduling and userspace has the ability to make a quick good choice. [3] Thus you can get most of the CPU performance of an async implementation but the relatively easy-to-read callback-free code of a threaded implementation. Anyway, people can disagree on what to call such a combination. [1] ▶ this video [2] the rseq stuff described in that talk has been upstreamed [4] but the switchto stuff still hasn't. [3] I'm not sure if the performance argument is as strong since the Meltdown/Spectre mitigations added to the performance cost of any syscall. [4] [link]

in “A hybrid thread / fiber task scheduler written in C++ 11” · View on HN

Could you describe what makes the Google Fibers so nice? I'm also really curious why they require modifications to the Linux kernel. My first guess would be stronger integration with the IO model at the syscall boundary (similar to io_uring). Edit: is this the talk your referring to? ▶ this video

in “A hybrid thread / fiber task scheduler written in C++ 11” · View on HN

Although you are technically correct in that the cost of system calls on most systems is high, your post also implies to a casual reader that the cost is fundamental. This is not true. As demonstrated in the benchmark, the hardware cost of systems calls/trapping is quite low. Even accounting for hardware workarounds (which were not even known about when the common consensus decided system calls were slow), excepting the case of hardware virtualization which has much more expensive mitigations, the fundamental cost is a fraction of the actual cost. The majority of the cost (~90+%) is actually due to software. In the system call case this is the implementation of the system call. In the context switch case this is usually the cost of the scheduling decision. These operations take material amounts of time and access memory which evicts user caches. It is in speeding up the software component that real gains can be made. This is not to say that system calls could ever be made "free" since function calls/loads/stores will still be ~100x faster, but the cost would probably be ~10% of the current cost if they software component could magically be made instant. As an example of what I am talking about, there is this talk by Google from 2013 [1] which describes a "fast context switch" system call (benchmark at 18:40). This "context switch" knows what it is switching to which lets it bypass the scheduling decision in normal context switches. As a result it takes ~170 ns (which is prett

in “Ways of Implementing a System Call” · View on HN

> Can we identify why exactly 1:1 threading is uncompetitive with async/await schemes, and fix that? I think we can identify it, but fixing it is not easy, at least at the kernel level. The easy part is the cost of scheduling. The kernel must schedule threads with very different behaviors -- say, encoding a video or serving requests over a socket -- so it uses a scheduling algorithm that's a compromise for all uses. But we can let the language's runtime take the scheduling of kernel threads with something like this: ▶ this video The harder part is managing the memory required for the stack. Even on 64-bit systems, the OS can't shrink and grow stacks finely enough. For example, I don't think there's a hard requirement that, say, memory below the sp is never accessed, so the OS can't even be sure about how much of the stack is used and can't uncommit pages. But even if there were such a requirement, or, that the language could tell the OS it follows such a requirement, still the OS can only manage memory at a page granularity, which is too much for lightweight concurrency. Any finer than that requires knowing about all pointers into the stack. We can do it in languages that track the location of all pointers in all frames, though, which is what we're attempting to do in Java, and this allows us to move stacks around, grow them and shrink them as necessary, even at a word granularity. > All this, by the way, is not to say that Rust made the wrong decision in focusing on async/await.

in “Why async fn in traits are hard” · View on HN

I am fully aware of the approach, especially in languages/systems like Erlang, and the freedom that very cheap threads give you. My first point was that you actually have much more of this freedom than most people are aware of, even with (comparatively) heavy kernel threads. For example, see the "Replace user threads with ... threads" by Paul Turned (Plumber's Conference): ▶ this video More on that point, I see a lot of user-threading/async craziness on clients such as iOS that would have been easily been handled by less than a dozen kernel threads, most of which would be sleeping most of the time anyhow. That's a number of kernel threads that is easily manageable and not particularly resource intensive. My second point is that there is one thread that this blocking-happy approach mostly doesn't apply to, and that is the UI thread. You really don't want that UI waiting for (network) I/O and therefore must employ some sort of asynchronous mechanism for data flow and/or notifications.

in “An introduction to reactive programming” · View on HN

Relevant HN thread: [link] BTW, I wonder if Paul Turner's work on fibers will change this equation if/when it makes it into the production kernel. ▶ this video Google has had very promising initial results with it.

in “An Opinionated Guide to Modern Java, Part 3: Web Development” · View on HN

They're actually talking about pursuing a "N:1" concurrency model (i.e. userland-managed coroutines in the kernel). See: ▶ this video

in “This Week in Rust – Static linking, Win64” · View on HN

It's hinted at in the video linked in the email[1]: > Over the past year we've been looking at how a uniform, performant, synchronous programming model can be implemented across all of Google's core languages (C++, Java, Go, Python), while maintaining compatibility with existing code. ▶ this video

in “The future of M:N threading” · View on HN

There is no User-Mode scheduling in the Linux Kernel to my knowledge, but the video[1] mentioned the post describes Linux patches that add support for it. [1] ▶ this video

in “The future of M:N threading” · View on HN