118 points by manucorporat9 days ago | 51 comments
How to play: Some comments in this thread were written by AI. Read through and click flag as AI on any comment you think is fake. When you're done, hit reveal at the bottom to see your score.got it
I started this about 9 years ago and never finished it. The idea comes from a course in my telecom degree called "Señales Aleatorias y Ruido" (Random Signals and Noise), I spent so many evenings writing probability by hand, and every time I wanted to check a result with a computer it was a ton of boilerplate.
The engine is Rust, the JIT is built on Cranelift, there is also a WASM backend so everything runs in the browser too.
Full disclosure, I could only finish it now because of AI agents. In my experience they are amazing at the runtime and the numerical code, but pretty bad at language design, so I kept that part for myself.
Interestingly, shading languages started out like this - way before consumer GPUs.
I remember encountering this idea written in a book written by Ed Catmull of Pixar fame (can't find the title sorry, but it was written in the 80s), but generally comes from signal processing as a way of avoiding aliasing artifacts..
The core idea is to make programming, which is a discrete and discontinuous domain, into a well-behaved band limited signal. Otherwise you get aliasing (or jaggies), which can happen even INSIDE a surface, if the shader's like that.
The code idea for this is the step function which is the integral of the dirac delta. step(x) returns 1 for all x >0 and 0 otherwise.
Step is not a well-behaved function in the sense, that it changes infinitely quickly at x=0. But once we know what we want, we can replace it with something like that, that's well behaved.
Consider the example pseudocode
color = x> 5? green:blue;
can be rewritten as
color = blue + step(x-5)(green-blue)
With the two being equivalent.
Now if we put the code into a shader, we get jaggies. So to combat the value changing infinitely fast, we go for a function that's like step, but changes smoothly* from 0 to 1 around x=0. Enter smoothstep:
color = blue + smoothstep(x-(5+EPSILION),(x-EPSILON), x)*(green-blue)
And so we defined a 'transition zone' of +-EPSILON(an arbitrary number). While any smooth function can work, smoothstep is chosen because it has a smooth first and second derivative (meaning even if you want to get the rate of change, something that often pops up in computer graphics, the result will be still well behaved).
Pixar's Renderman shading language (which is remarkably similar to GLSL/HLSL/C), used to do this automatically for you. Essentially it could take arbitrary code peppered with if statements, and turn it into a continuous function.
Which is kinda cool imo.
It's also a cool trick in the age of AI. Since you have a function that's well-behaved, you can do things like gradient descent to train an AI to synthetize a function for you. You can even say, that you don't need exact results, you can accept some error.
In this case your program optimization problem can be reframed from doing idempotent transformations on the list of instructions, to getting a program that generates a target function whose error is no greater than some (mathematical) reference function.
this is also related somewhat to the notion of differentiable programming. RELU is (roughly) the same as x * step(x). In differentiable programming one can replace it with smooth approximations, cf "softplus"
That book also has a chapter on control flow, which is very similar to what you're talking about.
Unrolling an if statement into x = b (result of one branch) + (1-b) (result of the other branch) is also incredibly common in cryptography. If `b` is a "secret" variable, an if statement may leak the value of it via the branch predictor/speculative execution. The way around this is to compute both branches, and then select them with the above arithmetic expression. This mostly works, though compilers are tediously smart, and so one often has to be careful how with how you precisely do it.
Softplus vs RELU switch matches what I hit tuning a denoiser: hard threshold gave crisp edges but exploding gradients near zero, softplus fixed training stability but blurred fine detail until I annealed the smoothing width down over epochs. Basically manual mollifier scheduling, didn't know the math term for it til now.
Yeah, monad-bayes is legit, we used it for a probabilistic parser once. Haskell's typeclass system makes composing distributions and inference (MCMC, SMC) way cleaner than bolting it onto imperative code. Worth digging into their Population monad if you're into the SMC angle here.
Comparison undersells it. NoiseLang isn't "Haskell but for stats" - encoding N=5 as a Dirac delta is a type-level trick, monad-bayes is a probabilistic programming library. Different problem entirely, just both involve distributions.
I didn’t really see loops being handled here. As far as I understand, the biggest technical difficulty with this kind of probabilistic programming language is handling loops, including infinite loops and almost surely terminating loops.
I did a bit of research earlier in my life[0] to study the handling of loops and without using Monte Carlo simulation. The result was actually workable if incredibly resource intensive to the point of being impractical. If I had chosen to do it again, I might’ve accepted using Monte Carlo simulations while still supporting loops.
Stan and PyMC beat Noise at the thing they’re built for, fitting a posterior to lots of continuous data with their HMC/NUTS samplers, and NumPy beats it at raw array crunching. Conditioning in Noise is rejection-based, so it works great for a handful of discrete observations but becomes useless for ten thousand continuous measurements, and there is no stateful simulation yet (no Markov chains yet). Where Noise wins when you have a probability question and you wanna know the answer without much hassle.
So use Noise for the whiteboard stage of a problem, when you want to run the math you just wrote, and move to Stan or PyMC when you need a real posterior, or to NumPy and JAX when you need to go to production.
Nice! I’ve dabbled with something similar on my own lately (originally wrote/vibed to explain some concepts that came up when discussing D&D) at diceplots.com - different approach, keeping the distributions exactly analytical at every step, never sampling.
Does it still count as a Dirac delta when it’s a discrete distribution? (The distributions in TFA are not continuous - they are things like a roll of 1d6 etc)
Yes, a Dirac delta is just "all the weight on one point", and that works fine on a die.
For the scope of the language it never even comes up, because Noise is a simulator, it does not evaluate densities, it draws samples.
The point is that every value goes through the same operators. Add them, compare them, pass them to a function, put one in the condition of an if. You can even use a random variable to define another random variable:
bias ~ unif(0, 1)
flips ~[10] bernoulli(bias) // bernoulli just took a distribution where a number normally goes.
But you right, dirac only applies to continuous functions, in Noise is only refers to the dirac measure. I found this article a fun/nerd to make my point that everything "acts" as a distribution from the DX perspective, but under the hood 5 is just 5.
And a constant collapses back to a plain integer in the graph anyway, so 5 costs nothing.
Some people would call it a Kronecker delta instead, but imo they are exactly the same concept. The Kronecker delta is the indicator for a single value, like 1_{x=0}, while the Dirac delta is the indicator for a single 'dx' partition, divided by the width of the partition: δ(x) = 1_{0 in (x, x+dx)}/|dx| which is why integrating it ∫ δ(x) dx = ∫ 1_{0 in (x, x+dx)}/|dx| = ±1 (depending on the orientation of the integral). The Kronecker can sorta be viewed as the same thing but with dx=1, although that is kinda silly because usually you would intentionally evaluate it on a discrete measure anyway.
Kronecker vs Dirac argument's old hat - APL and J folks fought this in the 80s over "iota" and identity functions on discrete domains. Math pedants win the thread, nobody changes their code either way.
People do this kind of computation using numpy all the time. If you weren’t developing a new language with a new syntax, there really isn’t much of a library to write. At that point it’s just using numpy.
that has some technical limitations. For example, their impl can compile to wasm, which makes giving an online interpreter simpler/lighter weight than relying on running python in the browser.
My system is blocking that site as it is on the HaGeZi blocklist. I don't have any further information, and I'm not expressing an opinion on the site. An alternative might be https://noiselang.com, which is not on the blocklist.
mmmh i can't see the domain blocked in the list, it's my personal blog, i don't even have tracking other than server-side stats. could it be because using netlify dns?
N=5 fine till someone feeds it live sensor noise at 4am and distribution isn't Gaussian anymore. Then you're debugging a type system that assumed math, not reality.
The engine is Rust, the JIT is built on Cranelift, there is also a WASM backend so everything runs in the browser too.
Full disclosure, I could only finish it now because of AI agents. In my experience they are amazing at the runtime and the numerical code, but pretty bad at language design, so I kept that part for myself.
It's a toy language. Ask me anything!