RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy on HackerTube: recommended by 2 Hacker News commenters across 2 separate months, 2017–2025.
talk · intermediate · evergreen
Joe Duffy's RustConf 2017 closing keynote on his experience at Microsoft building Midori, an operating system written in a Rust-like safe systems language. He covers the technical and cultural challenges of applying those lessons to Windows.
HN repeatedly cites it for Midori lessons: compatibility matters for adoption, and the Windows team stayed sceptical even with Midori running real workloads.
“I summarized this excellent talk here [1], but one of the main points is that compatibility with existing systems is important for adoption.”
14 Hacker News comments link this video. Each one links back to its thread.
See also "Safe Systems Software and the Future of Computing" by Joe Duffy, the closing keynote at RustConf 2017: ▶ this video Midori and Rust have several striking similarities, and Microsoft's recent uptick in interest in Rust (and Rust-like languages) bodes well for improved software quality.
in “Blogging about Midori (2015)” · View on HN
Related: "Safe Systems Software and the Future of Computing by Joe Duffy" at RustConf 2017. ▶ this video I summarized this excellent talk here [1], but one of the main points is that compatibility with existing systems is important for adoption. (They learned that the hard way -- by having their entire project cancelled and almost everything thrown out.) He advocates unit-by-unit rewrites rather than big-bang rewrites, just like Kell does in this conference article. And compatibility with C in Windows should be easier than it is in the Unix world, because the whole OS is architected around a binary protocol AFAIK -- COM. My sense is that Rust may not have thought enough about compatibility early in its life. Only later when they ran into adoption problems did they start talking more about compatibility. Also, it seems Rust competes more with C++ than C, and there seems to be very little attempt to be compatible with C++ (although perhaps that problem is intractable.) Personally I don't think Rust will be a successful C replacement. It will have some adoption, but the Linux kernel will still be running on bajillions of devices 10 years from now, written in C. And in 20 years, something else will come along to replace either C or Linux, but that thing won't involve Rust. [1] [link]
in “Some Were Meant for C: The Endurance of an Unmanageable Language [pdf]” · View on HN
Mostly because of internal politics, it was used in production for Bing. > While never reaching commercial release, at one time Midori powered all of Microsoft’s natural language search service for the West Coast and Asia. From [link] Enjoy this recording of an internal presentation, while it is still available on YouTube, [link] Joe Duffy also did a few presentations, on one of them (too lazy to search for the exact moment), he mentions that even with Midori running in front of them, the Windows team was very sceptical of it, "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" ▶ this video "Safe Systems Programming in C# and .NET" [link]
in “New OS aims to provide (some) compatibility with macOS” · View on HN
I agree with your point of view, this was the second coming of Longhorn, WinRT was after all pushed by the same folks that killed the Longhorn effort, those on Windows team that sided with Steven Sinofsky, using COM in Vista as future foundation for .NET ideas in Longhorn. The Hilo tutorial for Windows 7 remarks regarding C++ are quite telling of their point of view, on which programming language one should be using, [link] "The rich user experience of Windows 7 is best accessed through a powerful, flexible language, and that means C++: by using C++ you can access the raw power of the APIs for Windows 7." From [link] Joe Duffy has some remarks on how even with Midori running Asia Bing infrastructure, and proving workloads in front of the Windows team, the reception to it was rather cold. ▶ this video Eventually he left MSR and created Pulumin on top of Go. Meanwhile Apple/NeXT and Google, decided to push languages like Objective-C, Swift, Java, Kotlin, over classical C and C++, proving the point that change is possible if management is willing to support the team, even if that is a very long run. See Apple's Metal Swift bindings, versus Microsoft's Agility SDK for DirectX.
in “.NET 10 Preview 6 brings JIT improvements, one-shot tool execution” · View on HN
Project Oberon, [link] The evolution of Oberon based OSes, Ethos, [link] Active Oberon which is the Oberon variant I prefer, [link] [link] [link] Some screenshots at my article, take it while the site still exists, [link] SPIN, done in Modula-3 [link] Singularity, [link] [link] Midori, although no source code, the blog posts, existing talks and internal session at Microsof do provide some nice overview, [link] "The Midori Operating System Overview" [link] "Safe Systems Programming in C# and .NET" [link] "Safe Systems Software and the Future of Computing" ▶ this video Xerox PARC Mesa, used on the Xerox Star OS [link] [link] [link] Xerox PARC Cedar, used on Dorado platforms [link] [link] [link] [link] [link] Xerox PARC Smalltalk, original documentation can be taken from [link] - Smalltalk-80, Bits of History, Words of Advice - Smalltalk-80: The Language and its Implementation - Smalltalk-80, The Interactive Programming Environment Squeak and Pharos linage, [link] [link] Xerox PARC Interlisp-D, with Medley [link] [link] House, written in Haskell [link] MirageOS, written in OCaml, partially used by Docker (TCP/IP stack), and Xen Hypervisor [link] [link] [link] AmigaOS, [link] [link] [link] [link] Solo in Concurrent Pascal, The solo operating system: A concurrent pascal program [link] Lillith in Modula-2 [link] [link] Now go out and explore, UNIX is not the be all, end all of how an OS is supposed to be, neither is C the ultimate systems programming language.
in “Recommendation for non-DOS/Unix open source OS outside x86/X64” · View on HN
See also from him, Safe Systems Programming in C# and .NET [link] RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing ▶ this video The latter has some remarks on how the WinDev disregarded Midori efforts even with the system proving its value in workloads like Bing[0]. [0] - [link]
in “Safe Native Code: The Midori Compiler” · View on HN
Regarding Midori, besides the blog posts, Joe Duffy did two talks about the subject, "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" ▶ this video "Safe Systems Programming in C# and .NET" [link] In one of them, he mentions that even with Midori proving its value to the Windows team, they were quite dimissive of it. It appears to have also been yet another victim of the usual DevDiv vs WinDev politics.
in “The wild world of non-C operating systems” · View on HN
The usual politics, from .NET vs C++ at Microsoft. Here Joe Duffy mentions towards the end that even with Midori running in front of them, the Windows team was sceptical of it. ▶ this video Since .NET's introduction, Microsoft seems to lack the same kind of culture that Apple and Google have towards into steering their platforms into safer languages (e.g. how constrained NDK happens to be, or first class bindings to all OS APIs in Swift). It appears that every attempt to do so ends up being sabotaged in some way to assure C++'s reign at Microsoft and Windows subsystems. Note that Windows is the only desktop/mobile OS where the GUI stack is still fully C++ aware, and they even make a point out of it. [link] > WinUI is powered by a highly optimized C++ core that delivers blistering performance, long battery life, and responsive interactivity that professional developers demand. Its lower system utilization allows it to run on a wider range of hardware, ensuring your sophisticated workloads run with ease.
in “Singularity – Microsoft’s Experimental OS” · View on HN
To fully get where I am coming from, you have to go back to when .NET was released. .NET was supposed to be the great reunification of VB, C++ and COM runtimes, then also got a Java touch into the mix and .NET happenend (initially was known as Ext-VOS). [link] Hence why CLR is just like WASM + GC if you prefer a modern comparisasion. If you go back into web archives, when Visual Studio.NET was released, it was going to be .NET everywhere, across the whole stack. However a big management mistake happened, .NET was part of DevTools business unit, while C++ was kept under WinDev, up until Satya started to change the culture, it has been pretty much WinDev vs DevTools. So Managed DirectX comes, eventually gets killed, XNA and Silverlight take over Windows Phone 7, get killed by WinRT and DirectXTK and so on. Going back to the originally statement, if you Google for why Longhorn did not work out, you will find many .NET blaming. [link] Yet Android, ChromeOS, Midori are examples of what happens when actually everyone works into the same direction bringing an OS into production. Joe Duffy does some remarks on his two talks, where he hints at why it was a failure to fight Windows culture "Systems Programming in C# " - [link] "Safe Systems Software and the Future of Computing" - ▶ this video Note that for some time the Asian Bing nodes were actually running on top of Midori as production test. A big decision of Vista, was to replicate the .NET design using COM instead (hello WinDev), hence
in “A Look into CBL-Mariner, Microsoft’s Internal Linux Distribution” · View on HN
At RustConf 2017 keynote presented by Joe Duffy regarding Midori at a certain moment, he mentions that even with Midori running in front of them, WinDev guys were still not open to the idea of such kind of system being possible. "Safe Systems Software and the Future of Computing" ▶ this video Sorry a bit lazy to track down the exact moment.
in “A History of .NET Runtimes” · View on HN
First of all, there is no such language as C/C++. C++ although tainted by copy-paste compatibility with C89, does offer the language features for security conscious developers to make use of, and C++17 is quite a pleasure to use. C on the other hand could be nuked for what I care, it has been clear since 1979 that security and C would never go together. Now regarding your assertions. z/OS was initially written in a mix of Assembly and PL/I. C and C++ came later into the picture as the system got a POSIX personality. The way to write libraries exposed to all languages available on z/OS is via the z/OS Language Environment. Chapter 9.12 of z/OS Basics, [link] .NET has hardly any pure C code, rather C++. Even so, the C# and VB.NET compilers have been bootstrapped thanks to Rosylin and now C++ is gone from the compiler side. F# was bootstrapped from the early days. Since Roslyn came into production, the .NET team started planning moving parts of the runtime from C++ to C#. "So, in my view, the primary trend is moving our existing C++ codebase to C#. It makes us so much more efficient and enables a broader set of .NET developers to reason about the base platform more easily and also contribute." [link] Also the first version of .NET's GC was actually prototyped in Common Lisp, [link] Unity introduced HPC# at GDC 2018 and they are now in the process of migrating C++ engine code into HPC#. Both the requirements of HPC# as C# subset for performance critical code, as the experience wi
in “The Evolution of C Programming Practices: A Study of Unix (2016) [pdf]” · View on HN
It has been proven multiple times, at Xerox PARC, ETHZ, DEC Olivetti, Microsoft that garbage collection systems programming languages are doable. However technology adoption always needs to fight against non-believers. Joe Duffy has stated on his Rustconf 2017 presentation [0] that even Midori running on front for their eyes, with some use cases where System C# was even shoulder-with-shoulder with C++ [1], that Windows devteam was not getting it. Nowadays he is an happy Go coder instead of trying to convince them otherwise. D has a big problem with their GC implementation, which provides good arguments for the anti-GC crowd in systems programming. However they have been improving it quite a bit during the latest releases, as well as, making the runtime library more @nogc friendly. As for reaching mainstream, I share your thoughts. It is not only Rust and Go. There is also Swift, C++17 and the ongoing improvements to make Java and C# more machine friendly (memory management and AOT compilation). Still, even if just a small group gets to use it, it is already a victory. Most languages never achieve it. [0] - ▶ this video [1] - [link]
in “D on embedded Linux (ARM)” · View on HN
I don't know why but I'm actually less excited about Fuchsia than the old Singularity/Midori. As Joe Duffy said [0] when they worked on Midori they also wrote UI, editors, browsers and many more applications in months so it's not surprising Google engineers also managed that. [0]: ▶ this video For me it looks like the classic Not Invented Here syndrome - we already have microkernels, even formally verified ones (seL4) but AFAIK Google didn't want that to "be more flexible". Using C for kernel also looks like a poor choice nowadays, unless Google has some magic static analysis (for example Singularity/Midori had custom languages with ownership tracking akin to Rust). Maybe Fuchsia is for bringing microkernel architecture for masses using "boring technology"?
in “Google’s Fuchsia OS on the Pixelbook: It Works It Actually Works” · View on HN
It doesn't matter what Rob Pike says regarding Go and systems programming, because Google, the company that employs him, has decided Go makes sense to write system components of Fucshia in Go. That is a fact, easily validated in Fuchsia's repository. According to you, Rob Pike should call management and let them know it is not a good idea to use Go for writing Fuchsia's TCP/IP stack. You only read Joe's blog years ago, yet you missed the talks he gave. Two years ago is not that long time ago, [link] "My biggest regret is that we didn’t OSS it from the start, where the meritocracy of the Internet could judge its pieces appropriately. As with all big corporations, decisions around the destiny of Midori’s core technology weren’t entirely technology-driven, and sadly, not even entirely business-driven. But therein lies some important lessons too." [link] "Over the course of 8 years, we were able to significantly narrow the gap between our version of C# and classical C/C++ systems, to the point where basic code quality, in both size of speed dimensions, was seldom the deciding factor when comparing Midori’s performance to existing workloads. In fact, something counter-intuitive happened. The ability to co-design the language, runtime, frameworks, operating system, and the compiler – making tradeoffs in one area to gain advantages in other areas – gave the compiler far more symbolic information than it ever had before about the program’s semantics and, so, I dare say, was able to e
in “Re: Moving from PHP to Go and Back Again” · View on HN