Sounds cool, I want to try this kind of things out, but do you have or planned to have a sandboxing environment, where the agent can try running the query in let say duckdb first to confirm its validity/result before sending it over to bq? Or use something like tablesample when developing the query to reduce cost? One more thing, how do you compare with nao ( https://github.com/getnao/nao ), it's something I've followed for a while and seem to answer similar issue as what ktx build

So, and I may be oversimplifying, you are creating awesome documents and references that I would have loved to have for my different jobs 5-10 years ago (or more).

It’s just that making such docs had next to no ROI 10 years ago. But today they are the difference between success and failure.

It’s fascinating - thank you

(and who writes the wiki / business rules ? Can they be reverse engineered from existing query stack? )

Sounds great - all the best

Edit: don’t take the above as criticism - just trying to fit new ideas into an old dog.

How does this compare with Wren 2.0, OpenVikings etc

Hey, ktx maintainer here! Great question, we actually have quite a bit in common with Wren, they're in the same category. The main difference is that with Wren you author the semantic model (write MDL) and it executes SQL through its own Rust engine (similar to Cube as well). ktx instead builds the context for you - ingesting definitions already in dbt, Looker, Metabase, and Notion, auto-detecting joins and fan/chasm traps, flagging contradictions. It also builds both : the SL and wiki that accumulates and maintains free-form tribal knowledge (that’s also interlinked with ktx SL)

The ingestion angle is what gets me — we wasted months hand-authoring a semantic layer nobody kept up to date. If ktx can pull from dbt and stay in sync automatically, that's the actual hard part. Does it handle when your dbt docs are incomplete or contradictory? That's usually our situation.

This sounds like it might be exactly what I need!

Does `ktx setup` need Claude specifically?

> LLM - picks a Claude backend. The default uses your local Claude Code session, so no API key is required. You can also use an Anthropic API key or Vertex AI.

I'm currently on Copilot at work

Next questions...

I'm skim reading the docs but I just want to know a bit more about the architecture

Does it produce artefacts that I can commit to version control and share with my team? Is it a tool that everyone runs locally in the project? Or there's a component (the wiki?) that I should deploy as an internal service?

We currently support anthropic models for the setup (whether through claude pro/max plan) or through API. Adding support for openai API / codex should be pretty straightforward - would love to get you in the community slack to get more details on your copilot setup

Curious what "pretty straightforward" actually means here. If the setup logic is tuned around Claude's specific tool-calling or reasoning behavior, porting to GPT-4o isn't just an API swap — the agent might produce subtly different schemas or miss edge cases. Has anyone tested cross-model consistency on non-trivial contexts?

just tried from claude code.

some observations: setup (without openai embeddings) was pretty easy. Nice! Not sure why it took ~19 minutes for a postgres database with 20sh tables. Also the data from pg_desc did not get into the wiki

I am going to try to embed it in our agentic flow and see how it fits. Very cool project though.

Hi, thanks for trying it! 19m for ~20 tables is slower that I'd expect, I assume you activated historic queries, right? As Luca said we're focusing on performance improvements right now. 99% of the latency is LLM calls, so we're optimizing them.

As for the descriptions: `pg_description` should be used already, we store raw pg descriptions in the semantic-layer/<connection>/_schema/*.yaml files under descriptions.db. there's also an AI generated description next to it. AI is the combination of the raw DB description + bunch of metadata we manage to collect about a table/column.

could you run `ktx sl` and see if the descriptions appear in the CLI ?

also feel free to join our community Slack - ktx.sh/slack We'll be happy to help with other questions

Thanks for the feedback - the length is typically because we’re also processing query history to infer acceptable joins, etc. But we have a lot of optimizations to implement, so we’re positive this will run faster in the coming weeks. Looking into the pg_desc issue now, thanks for reporting!

We ran into the pg_desc gap too. Ended up manually dumping `obj_description` queries into a markdown file and feeding that as a static doc. Not elegant but it worked until the native support landed. Worth checking if your tables actually have comments set — a lot of schemas don't bother and you think they do.

How are you measuring the accuracy? Are you running this against any benchmarks?

I see this covers a file based approach, was there ever a consideration for a graph based approach?

For business context, how do you handle context that evolves over time?

Great questions! we're currently working on Spider 2 submission, hope to have first results soon. It's true that we took file-first approach and not a graph DB. Main reason is that the ktx wiki and semantic layer entities while being written in plain text files (md or yaml) still contain links to each other. This allows an agent to find the right entry point (with the help of lexical and semantic searches merged with RRF) and then traverse these links to collect enough context.

As for the business context evolution - that's exactly the reason we have ingestion reconciliation and git versioning. The idea is to give ingestion agent a way to deduplicate/consolidate knowledge during the ingestion and leave complex conflicts to humans to resolve

And development was done around link detection and text to sql benchmarks to measure/compare different approaches

cool idea. but for ex over time if users update db schemas, agents run old queries and got errors, is there a mechanism in your lib for agents to examine and self-correct? also is there an auto memory compression the moment the context becomes too big?

good point! yes, there's a feedback mcp tool that agents are instructed to use when they want to save/update ktx "memories". This is a way to keep the wiki+semantic layer up to date beyond the initial ingestion. Plus every time there's a new ingestion the reconciliation mechanism will gather potentially drifting context in one place.

We don't have a manual compression yet, it's part of our other product that ktx used to be a part of before it became open-source. But we decided to move that feature to OSS too, so expect it to come out in the next releases !

cool

What happens when the context layer goes stale at 2am and your agent starts making decisions on bad data? That's the part I'd want answered before touching this in prod.

Thanks!

Interesting approach. Context management for agents is an underexplored area. I've been looking at similar problems - the key challenge is keeping token usage low while giving the agent enough context to be useful. Tiered retrieval (facts first, full text only whenneeded) seems to work well in practice.

Agreed, in addition to this I'd say it's better to spend tokens once while ingesting the sources and storing the canonical reusable definition rather than having agents do the full exploration over and over again

So the context layer needs context to understand the context layer.