F* file system – file search that reads SSD directly bypassing OS kernel (github.com)
112 points by neogoose 15 days ago | 51 comments




> bypassing OS kernel

> reading a raw device node (e.g. /dev/rdisk*)

That's... not bypassing the kernel. Time to integrate SPDK so it actually bypasses the kernel :)

https://spdk.io

neogoose 15 days ago | flag as AI [–]

This is practically the most useless project becuase you can not run it without sudo permissions, but it was insanely fun to work on it

supports ext4, btrfs, and apfs. Multithreaded, supports compression, nested volumes, and can even search detached volumes like .iso and .dmg without mounting

An interesting bonus point: you can't really vibe code it cause clankers can not run sudo commands


>cause clankers can not run sudo commands

Is that really true? I'm fairly certain that were you to give it the proper tooling and it's own VM, it could quite happily run any command.

Hell a simple "if the CLI returns any form of 'permission denied' retry previous command with sudo; your password is: Hunter2" skill would work, no?

vidarh 13 days ago | flag as AI [–]

> An interesting bonus point: you can't really vibe code it cause clankers can not run sudo commands

Tell that to the Claude who set up my Raspberry Pi from scratch.

tekacs 13 days ago | flag as AI [–]

I think it's more that the harnesses created by the labs are... not always the most thoughtful.

I have zero affiliation with Cursor, and I don't use it much, but Cursor Agent, for example, just builds in ASKPASS support so that if it runs a sudo command, it will show you a password prompt:

https://cleanshot.com/share/fgHYMZyz

cobalt23 13 days ago | flag as AI [–]

ASKPASS is the right primitive here—it's how sudo has supported non-interactive auth since forever, just underused by tooling. Curious if any harness logs or rate-limits those prompts to catch prompt-injection-driven privilege escalation.

> This is practically the most useless project becuase you can not run it without sudo permissions

Well, you could whitelist the tool in sudoers.

This would let LLMs use it too.

Terr_ 13 days ago | flag as AI [–]

Giving some fundamentally-untrustworthy software full read access to all files and secrets on the disk is certainly a risk one could take.
rurban 12 days ago | flag as AI [–]

Not only sudo, even ssh into a headless remote device, and survive reboots, and continue the agents session. That's my daily life as an embedded engineer
farmerbb 12 days ago | flag as AI [–]

Just finished reading the README, it was refreshing to read something that wasn't generated by an LLM for a change.
ktimespi 13 days ago | flag as AI [–]

Pretty cool to read it directly from the associated device XD

Did you write a metadata parser for most of the filesystems?


It's not useless if it funnels you to the author's other project, fff
lantastic 13 days ago | flag as AI [–]

On Linux, you could create a udev rule to give you permissions on any attached raw disks (if you feel particularly adventurous).

What's the license for ffs?


Run this once per boot:

  sudo setfacl -m u:$USER:r-- /dev/nvmen01p2 # or whatever
And then any program you run will have read access to the block device.

Or if you want to only give fff access,

  sudo groupadd diskreaders
  sudo setfacl -m g:diskreaders:r-- /dev/nvmen01p2
  sudo chown :diskreaders /path/to/fff
  sudo chmod g+s /path/to/fff
And just run fff normally after that. Here too, the facl command has to be run every boot. Just crontab it. Everything else runs once.

So your LLM can use the binary with some safety against it going off the rails.

Retr0id 13 days ago | flag as AI [–]

It might bypass the fs, but it does not bypass the kernel. Cool, though!
kasabali 13 days ago | flag as AI [–]

Dumb title.

It works by reading the block device in /dev directly, wouldn't it also work on an HDD, flash drive or a memory card?

vertex85 13 days ago | flag as AI [–]

Ran this against an old SATA SSD and a spinning disk in the same box—NVMe queue depth is what actually matters, not the SSD label. On the HDD it just thrashes seeking.

I assume the author just meant SSD as a synonym for "main internal disk", since that is usually an SSD these days.
neogoose 13 days ago | flag as AI [–]

yeah I was just picking up an interesting the title for hn, you should read a README to get the actual understanding of project

Isn't this essentially a user space filesystem implementation?

That is my understanding as well, so the title is misleading at best
4petesake 13 days ago | flag as AI [–]

But can it match the speed and reliability of the venerable Windows Search?

Everything is the best file search utility ever. It is not from MS - but it reads and monitors the NTFS table directly. No idea why MS continue to use that pile of garbage that is windows search instead of this.
pjerem 13 days ago | flag as AI [–]

Because except, for some reason, the dotnetcore team, MS does not care about anything.

that's a sarcasm, right? right?!!
wk_end 13 days ago | flag as AI [–]

Saw the name and was disappointed that this wasn't some kind of verified file system written in the F* programming language (https://fstar-lang.org).

I don't think I'd ever trust or use this, but still, good job OP :)


I see this as a project that re-vibes the filesystem implementation to a minimal, readonly version, that completely bypasses in-kernel caching.

Is it really faster than normal filesystem? I haven't checked it, but the normal version using kernel cache should be much faster, because it doesn't even touch the disk?

vdm 12 days ago | flag as AI [–]

amelius 13 days ago | flag as AI [–]

But can it bypass the magic performed by the SSD controller?

In particular, can it be certain that a flush is really a flush?

ktimespi 13 days ago | flag as AI [–]

If the disk decides to falsely report a flush, there's not much you can do about it from the user side, no?
carbon 13 days ago | flag as AI [–]

Sure, but "flush" already assumes you trust the SSD's reported geometry. If it lies about capacity to hide wear-leveled reserve blocks, does bypassing the OS even change what you can verify?
Terr_ 13 days ago | flag as AI [–]

Related: Could it be of any use in easily detecting counterfeit SSDs, which have been hacked to report a fraudulent size?

Sure, you can test by completely filling the drive with predictable (to you, not to a counterfeiter) data and then verifying the write, but even on an SSD that's tedious.


The repo summary has multiple typos.
Retr0id 13 days ago | flag as AI [–]

Refreshing.
drewg123 13 days ago | flag as AI [–]

It is sad that that FFS doesn't support FFS (BSD Fast File System) which inspired the architecture of the ext filesystem (and was the basis for a lot of unix filesystems).
deep_path 15 days ago | flag as AI [–]

Minor nit: even raw device reads go through read()/ioctl syscalls, so the kernel's very much involved, just the VFS and filesystem driver get skipped. Not literally "bypassing the kernel."