Exploring. ~ Past: rendering lead
@figma
, creator of PARTICULATE for PC VR, founder of
@workflowy
, search engine backend @ Google, graphics research @ Stanford
Sparse allocation/rendering of SDF bricks is now working!
The pipeline is fully GPU-driven - a series of compute shaders that manages brick allocation/baking in a 3D texture atlas, followed by draw-indirect for rendering.
Still a lot more work to do, including automatic LOD.
Finally finished this book! (after starting it a few years ago).
It covers a lot of topics related to building a Google-Earth-style engine, including terrain LOD approaches (geometry clipmaps, chunked LOD), floating point precision stuff (to fix vertex jittering), etc.
[cont]
I wrote an article about the fast compute shader particle rendering technique I developed for my VR app's custom engine:
More posts coming! Appreciate any RTs as I don't have a large following currently.
I just finished an article on the process of learning graphics! Focus is on general principles/strategies.
Contained within:
- Managing info overload
- Turning unknown unknowns into known unknowns
- Building a tolerance to feeling dumb :D
I just finished an article about hierarchical depth buffers (Hi-Z).
Contained within:
- How to generate them for non-power-of-2 dimensions (easy to mess this up)
- A fast compute-shader-based alternative technique
Appreciate any RTs for visibility!
I really like this AMD presentation from last year that describes their technique for converting scene geometry into a sparse SDF "brick" representation that can be ray-traced.
It's similar to Dreams in some ways, ...
Reading this great paper on ray-tracing "baked" SDF grids (where the surface in a voxel is defined via trilinear interpolation of 8 SDF samples).
The fidelity comparison vs. marching cubes is fascinating.
"Path Tracing of Signed Distance Function Grids"
The GigaVoxels paper mentioned in Alex Evans's Dreams talk is great.
It goes into a lot of detail about the implementation of a sparse octree of voxel data that's managed as an LRU cache (so only the LODs needed for current view are loaded).
1/
I liked this paper (thesis) from 2015 that describes a system for rendering sparse grids of voxel bricks. It's presented as an alternative to more-complex sparse voxel octree (SVO) based approaches.
1/
I've found Im3d is a nice complement to Dear ImGui, which is excellent for 2D UI/vis, but doesn't do 3D at all.
Im3d fills the hole, providing gizmos for 3D object manipulation and a bunch of immediate-mode 3D debug drawing features.
[cont.]
It's cool to see another GigaVoxels paper come out! (after what, more than a decade?)
This one is focused on making everything fully-GPU driven. I am intrigued by the use of clipmaps for LOD, and am likely to try out this approach in my own project.
Just re-read this
@iquilezles
article, which discusses how texture interpolation on GPUs is (generally? always?) 8-bit fixed point, which is usually fine for colors, but often bad for textures used to represent geometry (like baked SDFs in my case).
1/n
"In graphics APIs, shaders and cpu code are like water and oil, and this fundamental design decision has not been challenged in twenty years of programmable shading. It’s part of the untold, un-challenged 'culture' of graphics programming."
Enjoyed reading this paper, which is one of the few I've found that covers the topic of dynamic updates to sparse voxel octrees.
(In this case, specifically sparse voxel DAGs, which are essentially deduped octrees.)
1/
Great new post on the Box2D blog that describes implementing SIMD-accelerated contact solving via graph coloring (which breaks the contacts into sets that can be solved in parallel).
This lesser-known Youtube channel is worth checking out!
In the most recent video, Douglas describes building a voxel physics system inspired by the implementation in Teardown.
I've been putting together an engine to experiment with some ideas related rendering Dreams-style SDFs via baked distance field textures.
To start, I built the standard "ray-march analytic SDFs" prototype. Each frame the AABB containing all SDF shapes is rasterized.
[cont'd]
Benefits of hanging around Twitter for years:
I vaguely remembered a thread involving Dennis Gustafsson (Teardown) about ray-marching volume textures and the early-z test.
This is now relevant for my SDF renderer. I found the thread! (from 3.5 yrs ago!)
The problem with drawing backfaces is that you can't use conservative depth, since you might be Z-writing a value closer to the camera. Thus hardware pre-Z culling is off. I measured up to 6x performance difference with/without conservative depth in some Claybook raster protos...
It's funny to read old graphics papers written prior to compute shaders that try to do compute-y things by jumping through all sorts of hoops.
For example:
(from "Octree Textures on the GPU")
I promise I'm not making an SDF modeling program, but I couldn't resist making Mr. Potato Head in my fledgling engine (because... SDFs).
This is still brute-forcing the rendering, but with more features - baking to volume texture bricks to come!
Interesting SDF ray-marching edge case (literally) when # iterations is capped - you have to consider reaching max-iterations a hit to avoid spurious gaps in concave SDFs (I think), but the distance when the max is reached can be higher than the minimum distance.
Wanted to shout out
@SebAaltonen
's recent SIGGRAPH presentation on the HypeHype engine, which is absolutely jammed with insights on rendering beautiful (user-generated) scenes on low-end mobile devices*
* which are still supercomputers by any standard :D
I added an addendum to my article from last week on rendering particles with compute shaders.
The NV_shader_atomic_int64 extension (if available, it's supported on AMD GPUs as well) enables a further optimization - now 32‑400% faster than rasterization!
Got clipmaps working for baked SDF LOD in my engine (sorta, kinda!)
The basic idea is to bake the SDF at progressively coarser res in progressively-larger cubic regions centered on the camera.
In the vid, clipmap levels and SDF bricks are colored.
1/
After more than 3 years, today was my last day at Figma. I left to take more time to explore some paths related to graphics and gamedev - very excited! I learned so much about 2D rendering at Figma, and I'm excited to see what the rendering team there cooks up next.
Just encountered the bite-sized book series, "Game Physics in One Weekend" (obviously inspired by the Ray-Tracing equivalent). Looks promising, going to take a closer look!
All this reading about volume rendering is reminding me of implementing the Kajiya & Kay "Rendering Fur With 3D Textures" paper in Matt Pharr's "image synthesis" class over 20 years ago (!).
Somehow our writeup is still online!
The same SDF baked at different voxel resolutions (and rendered with ray-marching).
Going to experiment with dilating the SDF as voxel size increases due to the shrinkage.
This is true for WebGPU, but if you have a native codebase you can link to Dawn/wgpu and use native graphics debuggers like RenderDoc/PIX.
There's also a great guide on attaching PIX to Chrome itself for debugging/profiling WebGPU:
Also I want to say that GPU debugging and profiling for web is still horrible. Lags behind all desktop/mobile platforms by a decade (two decades vs consoles). I don't see WebGPU solving this. This makes developer experience much worse for graphics programmers in web vs native.
Just got a nice email from someone saying this article (about learning graphics) that I wrote 3 years ago was helpful. Nice to see at the end of a bit of a rough week!
As someone who's worked a lot on improving interaction/rendering latency...
Watching the screen update in response to the guy drawing lines w/ his finger in this video is paaaiiiinful😬
3 caution points for Democrats...
1. Trump wins if the difference between the polls at this point & the result matches what we saw in 2016 or 2020.
2. Trump's more popular now than at this pt in 2016 or 2020.
3. Dems don't say they're more likely to vote now than in May
My web build (using WebGPU) now resizes to fit the entire browser window - does a lot to make you forget you're still in a browser.
(Apologies for the horror I created.)
I somewhat randomly encountered this 3-part series on antialiased text rendering in
@SuperluminalSft
's profiler, and it is an impressively-detailed explanation of a sophisticated technique - recommend!
"16xAA font rendering using coverage masks"
@TheAnimator
Don't expect to understand new concepts fully the first time you encounter them (or the 2nd, etc.). I've found understanding graphics to be like peeling off the layers of an onion - at this point I just try to understand each thing a bit more than the last time I encountered it.
This is what happens when you render SDF bricks and naively calculate the normals using finite differences (there is a discontinuity at the edge of the bricks due to not normalizing properly, but also due to the brick texture discontinuity).
Currently investigating solutions!
There is part of me that wants to spend the next 10 years building an engine from scratch that procedurally generates infinitely detailed and beautiful explorable worlds. I feel like this is a common graphics programmer god fantasy :D
I have to say that the way buffer uploads/updates were designed in WebGPU is refreshingly simple compared to the nightmare of options in legacy OpenGL (and their associated perf characteristics on different hardware)
The book is getting a bit old at this point (~2011), but I love that it is a full treatise on building a specific type of renderer/engine.
There aren't many resources like that. (Perhaps because the market for them isn't that large, sadly.)
In 2024 there is really *still* no built-in C++ language support for converting an enum value to a string?!
Currently looking at the magic_enum library...
Really enjoyed
@_adamwiggins_
's retrospective/postmortem on
@MuseAppHQ
. A lot of the article reminded me of challenges we faced while working on WorkFlowy, another "thinking tool" that I cofounded (that launched in 2010, and is still going!).
(cont'd)
@ID_AA_Carmack
Would personally hate this for creative apps (including my own), where trigger sensitivity gives a lot of nuance to brush stroke heaviness and things like that.
3 weeks after leaving my job, I am reppin' Mist Technologies at GDC, inspired by MST Enterprises (my initials) that I used as an 11 year old for my Basic programs 😄
Yesterday was the 10 year anniversary of the launch of
@workflowy
, the startup I cofounded with
@jessep
(commemorated by this TechCrunch article that misspells my name). I left 3 years ago, but the company is still going strong after a full decade!
My experience building any early-stage software project is that the to-do list grows about 10x as fast as I tick things off it.
Every task forks in 5 different future directions, causes small bugs, introduces minor polish issues, etc., etc., etc.
It's a little discouraging that I wrote my engine using OpenGL (Vulkan was not an option for VR when I started) and now that I want to write some articles about it, OpenGL has become largely irrelevant. Oh well, there are things I can write about that are API-independent.
For the last couple months, our team has been working hard on a few paradigm-changing features for HTML Canvas.
One of them is the ability to have HTML elements in a canvas.
Here's a ThreeJS demo with a live HTML texture.
Excited to be starting a new job at
@figmadesign
next week! Will be working on graphics and other stuff. Figma makes a design and prototyping tool built on C++ compiled to WebAssembly with its own WebGL renderer (and is a pioneer of this type of app!).
Tip: to avoid wasting time on long Youtube videos, you can click "Open transcript" to scroll through the auto-generated transcript (and even text-search it).
Great for deciding whether a video is worth watching or skipping around.
One of the hardest things about working by yourself on a large project is the massive context switches: going from low-level engine optimization to making art assets in Photoshop to filling out tax forms to doing UI design. A bit exhausting :D
Inigo links a useful Shadertoy in the article that demonstrates the difference between using hardware interpolation vs. interpolating "manually" in the shader - you can see discontinuous derivatives in the former case.
3/end
Got async render/compute pipeline creation set up in my WebGPU app.
Nice to see how straightforward this was compared to all the past WebGL nonsense associated with this.
I'm not sure how common this is, but I don't particularly like learning new programming languages. Have used only C++, Python, and JS in the past decade. Keeping an eye on Rust, might look into it a bit more in 5 years if it's still alive :D
but also very different in that they're concerned with SDF'izing triangle meshes. (E.g., to support ray-tracing scene geometry on older GPUs.)
It's really detailed and clearly explained. There is also a follow-up presentation from this year on how they use this for GI.
It occurred to me that my render of the furry dog in the PBRT book is now older (17 yrs) than the Kajiya-Kay fur rendering paper it used was at the time (14 yrs). The paper was from 1989 and seemed like ancient history at the time. Hmm :)
@m_schuetz
Yeah, this has crossed my mind as well for some situations, but I guess that in-order process is too baked into GPU hardware to switch off for perf benefit?
When I quit my startup job in 2014 and started explored my personal interests in 2015 & 2016 it lead to so much personal growth that I can't imagine anything else other than grad school that lead to that kind of progress. Key take away: Invest in yourself if you have the means
I wonder if the trend toward massive monolithic game engines that eat the world (the two U's) will continue indefinitely. Would personally prefer a set of mix-and-match building blocks from independent orgs. Reminds me of the "libraries vs. frameworks" distinction.
Got around to reading this paper that implements the HW rasterization pipeline in software (on the GPU) - with the goal of seeing what additional hardware support would benefit software gfx the most (by allowing added flexibility without large perf cost).
I had a really exhausting October doing job interviews - did 4 "onsites" and a total of about 30 interviews in the span of a few weeks. Took me a few weeks to recover. It was the first time I interviewed for a job since 2005! Happy with how things turned out.
Maybe it's just my personal bubble, but it seems like Rust has been picking up a lot of steam lately. (I am a language late adopter so continuing to wait and see.)
Guy Godin says Virtual Dekstop cleared $3 million in gross revenue (that's before Facebook's cut of sales) on just the Oculus Quest standalone headset.