Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic on HackerTube: recommended by 5 Hacker News commenters across 3 separate months, 2017–2021.
lecture · advanced · dated
John Gustafson presents posits, a proposed drop-in replacement for IEEE 754 floats that is simpler in hardware and more flexible in precision. The seminar includes a live demo and machine learning results.
Cited in floating-point threads as the main introduction to posits and unums as an alternative to IEEE floats.
“Here's a nice introductory presentation”
13 Hacker News comments link this video. Each one links back to its thread.
For anyone interested in this sort of thing you'd probably also be interested in John Gustafson's work on unums/posits. Here's a nice introductory presentation ▶ this video
in “Floating-Point Determinism (2013)” · View on HN
You may like: ▶ this video It turns out the math is slightly harder but it's faster than IEEE FP in hardware, probably because of less conditionals in the spec. There is better performance in terms of numerical accuracy at the cost of substantially more difficult error analysis. Disclaimer: I'm in the video.
in “Floating Point Visually Explained” · View on HN
Have you seen "Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic"? ▶ this video
in “Why does e to pi i equal -1? (2015) [video]” · View on HN
Well unums in particular (especially the new type 3 "posits" ones) are very interesting and practical for hardware implementation. John Gustafson, the creator of unums, gave the first talk on type 3 unums at Stanford earlier this month, and there are already multiple hardware implementation efforts (one of which I am indirectly overseeing). While many people have recently hyped up low precision floating point for things like machine learning, Posits provide greater dynamic range AND greater accuracy for very few bits (as low as four, but with good ML starting 8 to 10 bits). On top of all of that, it costs less area and power in silicon than IEEE hardware. Check out the talk: ▶ this video
in “The “high-level CPU” challenge (2008)” · View on HN
I'm very surprised nobody has brought up John Gustafson's Unum work. His latest iteration is the posit, which he presented at Stanford earlier this month[0][1] EDIT: apologies to CalChris[2]. What I should have said is discussed Gustafson's Unum work. [0] ▶ this video (relevant question at 1h22m37s) [1] [link] [2] [link]
in “GCC optimized code gives strange floating point results” · View on HN
2003, was working in a science lab, lab tech came in and said he (or maybe it was his brother) had just bought a house, really cheap, in hawaii. He said it was in lanikai estates. Family is from hawaii and I'm a bit of a geography geek, so I hopped onto mapquest to see where it was... Noticed it was due east of pu'u o'o, and I know the volcano is moving east, so I told him, look you have 10-15 years to sell that house before it winds up under a volcano. Fast forward 15 years and volcano! The volcano just snuck in under my time window, haha. To be fair, I do not know for sure if HIS house went under; there are houses in the neighborhood that were spared. I didn't bother to ask because when I stalked him it said he was an employee of Theranos, and I decided it was probably not a good idea to poke a hornet's nest. Speaking of Theranos, back when I was in my 2nd postdoc (2013) we were talking about how theranos was a fraud and it was going to come crumbling down, Some other crazy short term things I predicted: Sarah Palin VP candidate, Feb 2008, Simone Giertz becoming famous (ca winter 2015). I also predicted the failure of the first starliner launch (documented here in HN). I also have managed to pick some ascendant "obscure" PLs just before hockey sticking. Used Julia professionally in 2015 (proof of use in 2017: ▶ this video ), used Elixir in 2017, been using Zig since 2019. And I haven't really used any other non-mainstream PLs, so I have a pretty good hit record I think. And a co
in “Boeing to ground Starliner indefinitely until valve issue solved” · View on HN
There was no paper, sorry, we talk about some of the research here ▶ this video big julia demo 40 minutes in. But the video doesn't feature the results I talk about (that happened a few months after the video) the specific results are somewhere in a DARPA presentation (but I don't have those slides, and the data are like 5 laptops back for me). The parallelism was, well, Gustafson parallelization, I pushed a randomly-generated 10k x 10k matrix solve to each of the nodes with a custom soft floating point to measure the relative numerical accuracy. Each node did all of one run per core, it was done very fast , but it would have taken months on a pc. As it was a gustafson scaling at its finest, there was zero cross-node communication during the computation. The containerization I used was singularity. I don't really love them much anymore, for various non-technical reasons, but it got the job done.
in “JuliaCon2020: Julia Is Production Ready” · View on HN
I should probably post evidence; this was a live demo I did of the floating point stuff at Stanford (demo begins ~53 minutes in) ▶ this video
in “Matlab–Python–Julia Cheatsheet” · View on HN
Most likely related: John L. Gustafson presenting "Beyond Floating Point: Next Generation Computer Arithmetic" ▶ this video With papers here: - UNUMS: [link] - Posits: [link]
in “High Expectations for Low Precision at Cern” · View on HN
Unums are unlikely to gain much usage. Posits[1][2], also by Gustafson, are a more reasonable alternative to IEEE-754 floating point (but will still have a difficult time displacing IEEE-754, if they can at all). [1] [link] [2] ▶ this video
in “16-bit math look-up tables – the unexpected power of scaled-integer math” · View on HN
so I'm the author of one of the first usable unum implementations. We're actually moving to something called sigmoid numbers which are even better. The lecture doesn't cover "valid mode" but that's more like the "unum". At the end of the video, there's a demonstration where I show some very interesting results concerning machine learning. ▶ this video You can try out sigmoid numbers in julia here: [link] Also, I am working on some verilog implementations, so we may have this in hardware rather soon. A few things about sigmoid numbers: 1) they have fixed sizes, so you don't have to worry about keeping track of things (which was a pain). 2) they have a property I call "isomorphic structure" - which means that zero-padding does not affect the value, and truncation gives you the nearest value at a lower bit resolution. 3) they really do perform better than floating points in several applications. There's a higher bit-per-bit entropy for 'most things you want to do', so the information content is richer. The distribution of the numbers is "more human" - that is to say at a certain point you stop caring about the details around the number, and more care that it's 'really big' or 'really small'; and the distribution reflects that.
in “How many floating-point numbers are in the interval [0,1]?” · View on HN
Are these the Neuromorphic chips [1] Jeff Hawkins of Numenta [2] has been talking about? [1] Neuromorphic Chips [link] [2] Numenta papers/videos [link] It would also be good to see a major chip manufacturer or cloud provider that makes its own chips (Google/IBM) get serious about graph processing chips [3,4] and moving beyond floating point [5]. [3] Novel Graph Processor Architecture [link] [4] Novel Graph Processor Architecture, Prototype System, and Results [link] [5] Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic ▶ this video
in “Intel Gets Serious About Neuromorphic, Cognitive Computing Future” · View on HN
> Points near the origin have dark colors, with the color assigned to a complex number z approaching black as z approaches 0. Points far from the origin are light, with the color of z approaching white as |z| approaches infinity. Every complex number has a different color in this picture, so a complex number can be uniquely specified by giving its color. So I was wondering if this approach could also be useful in the context of floats and integers. The implementation becomes simpler in many ways (we only have a finite number of possible values to consider). For example, start with a unique mapping of 8-bit integers to a continuous colour ramp. I suppose the easiest option would be to start in the top left, then go from left to right, top to bottom, resulting in a 16x16 image. If we then take a formula that also returns an 8-bit value, map every possible input to an output, and show the results as an image, it should make it easy to see where we have an integer overflow, for example (if we see sudden jumps from very bright to dark or the other way around). Then I realised: I have seen this in the wild! It's used all over the place in the demo-scene. For example: [link] [link] Aside, this also reminds me of Gustafson's closure plots for comparing his number encoding to standard floating points, although it's not really the same thing: ▶ this video
in “The Fundamental Theorem of Algebra: A Visual Approach (2015) [pdf]” · View on HN