I had an issue. A documents folder with over 12k objects in it. A hodgepodge of folders and sub-folders. That over time had created a mess that no amount of file movement was ever going to make it usable. I wanted: 1) To keep my data local 2) be able to filter out PII and other data 3) Be able to find and delete duplicates 4) Get short synopsis of what a document is 5) Semantic and keyword search 6) All of this kept local to me requiring no internet access and no tokens spent to train someone elses AI.

The result I call DocuBrowser and in it's current form is FOSS (GPL-3) licensed for your personal use. The UI is in your browser. The AI models used are held local and are tiny, Available for Linux(RPM,Deb, and tgz) Windows and Mac. Let me know what you think and thanks for taking the time to try it out.

Personal use? I need this at work, dragging useful info from tarpits like Teams and GitLab.

Also need to search git repos including all branches and history (TIL/xkcd#153'd GitLab's web search can basically only do one branch at a time).

I learned a solution is to turn the documents into vectors in say PostgreSQL (with pgvector) and do a cosine similarity search with a search vector. Doing a search for embed models on HuggingFace shows nomic-ai/nomic-embed-text-v1.5 and Qwen/Qwen3-Embedding-0.6B. I might have used a larger one like Qwen/Qwen3-Embedding-4B.

There's some info for AnythingLLM[0] which supports RAG. AnythingLLM has LanceDB out of the box but also supports others including pgvector.

[0] https://docs.anythingllm.com/features/embedding-models

We need projects like this. Automatically classifying the files is smart.

I'm working on a similar application called Hister (https://github.com/asciimoo/hister). I should borrow some of your ideas. =]

I just had a wild thought. Combine Hister with my RepoSearch app. Point it at a companies Internal github/gitlab and have a searchable knowledge base of your git repos.

Tried this internally with grep+ripgrep across old gitlab repos, ended up more useful than our actual wiki. Real risk: perms. One leaked API key in a dead repo, now it's searchable by everyone.

I have not set up Hister yet but it's on my list to try out. How would I do something like host it on my Unraid box but have it index/persist my local MacBook browsing history?

Wanted to share Antfly which I think serves a similar niche:

https://antfly.io/

https://github.com/antflydb/antfly

They’ve put a lot of effort into optimizing the local llm pipelines and I have a lot of faith in the devs working on it.

Not a fan of pushing every personal document to someone else's cloud. Nice to see a tool that keeps everything on disk instead.

Anyone getting a bunch of permission errors when running (e.g. Traceback (most recent call last): File "/Users/tron/Applications/DocuBrowse/venv/lib/python3.9/site-packages/psutil/_psosx.py", line 347, in wrapper return fun(self, args, *kwargs) File "/Users/tron/Applications/DocuBrowse/venv/lib/python3.9/site-packages/psutil/_psosx.py", line 508, in net_connections rawlist = cext.proc_net_connections(self.pid, families, types) PermissionError: [Errno 1] Operation not permitted (originated from proc_pidinfo(PROC_PIDLISTFDS) 1/2)

Living in bizarro world of AI. Install open source project, fails, feed into OpenCode w/DeepSeekFlash 4 -> feed error into it get fixed.

The kill_port function only catches ImportError from the psutil block, so when psutil is installed but raises AccessDenied (common on macOS), it crashes instead of falling back to lsof.

In platform_paths.py - add two lines after line 250:

except psutil.Error: pass

Fixed. Now when psutil raises AccessDenied (as happens on macOS without elevated privileges), it falls through to the lsof/fuser fallback instead of crashing. Try docubrowser start again.

I was building for mac on a VM install of OSX. Will take this as a bug, Thanks.

Honestly. This with Paperless-NGX might be game changing if both pointed to the same folder.

How do you feel about supporting an S3 compatible target as a feature request?

I'm actually thinking of this for a commercial product feature. However, if you use a tool like Rclone on Windows, Linux or Mac. Mount the s3 bucket and you can then run DocuBrowse as if the s3 bucket were local.

I love your project on many fronts. One, you're using Claude. Two, you used Python - but most importantly, you personally care about it.

I will be using this, and I will be making contributions to it as well.

> I'm actually thinking of this for a commercial product feature

Would you consider writing down which features you would like to make commercial product features and how you would like to price them?

Many such open source projects already (which is fantastic). I lose track of them.

Today it happened I needed a simple way to embed & query 1TB+ of documents, and I was looking at open source options. Can anyone tell me what their go-to solution is now? Could this be the one?

And what are the key differences vs. other open source RAG tools like kotaemon?

Looks good, definitely going to try it. Extra thanks for creating something fully local, we need more projects like this one!

thankyou

Local's great till disk fills up or model file corrupts and nobody remembers where the backups are. Fine for a hobby box, good luck running it unattended in prod.

This is really cool. Can it play nice with gdrive or Dropbox? For better or worse, that's just where my data lives now but I'd love this layer.

I use rclone to "mount" them locally. Then it becomes searchable.

Rclone's fine till you hit Google's API rate limits at scale, learned that the hard way circa 2018. WebDAV or native SDK integration holds up better long term than shelling out to a mount.

I'm a huge fan of recall, going to test this out. This looks very interesting.

Did you mean Recoll (https://www.recoll.org/)?

Indeed

Recoll: proof grep needs marketing.

The hardest part of these projects is usually not making documents searchable

I just installed this and, after a few hiccups, got it up and running on my Ubuntu system. Works great, looks great. Thank you for this. Half of my documents are OpenDocument format. Is there any chance you'll be supporting ODF in the future?

Yes, not supporting it is an oversight I will correct.

Will have version 0.9.1 out later today to support ODF formats.

Can you make a dockerfile and docker compose file?

You can.

Ok, I made them, they work well.

Nice, what are you hoping to accomplish with this project?

- Filling a need I personally have. - Learning how to leverage AI for real world use not just to fill up a data center. - Personal knowledge -developing skills

Pretty much in that order

Care to elaborate?

A resume

Auto-classification into folders sounds great until it's wrong. Has anyone measured false-classify rate at 12k+ docs? A misfiled tax document you never look for again is worse than a messy pile you can at least grep.