Last week I tried the new Unity Catalog wikis (Beta). I wrote a business rule into a table's wiki ("revenue = total_price of orders with status F only") and asked a Genie agent for revenue. It summed every order. When I asked who owns the table, it suggested checking a "wiki page outside this environment".
I asked about it on the Databricks Community, and someone from Databricks replied: Genie agents don't use wikis as context today (no timeline yet). They read table and column comments. For a rule like this, use a metric view or a SQL expression, and keep text instructions for clarifications.
So I tested that. Setup, in my own Azure workspace with the TPC-H sample data:
- `samples.tpch.orders` copied into a table, all years
- I asked about 1995 because it has F, O and P orders. The right answer (F only) is 32.69B. All orders is 171.63B
- 5 Genie agents, each with ONE kind of context
- 3 benchmark questions per agent ("What was our revenue in 1995?", "How much revenue did we make in 1995?", "What was our revenue in Q2 1995?"), run once in Agent mode and once in Chat mode
Results:
| Context | Agent mode | Chat mode |
|---|
| Nothing | 0/3 | 0/3 |
| Table + column comments | 3/3 | 3/3 |
| One text instruction | 3/3 | 3/3 |
| SQL expression (measure) + one example query | 3/3 | 3/3 |
| Metric view (filter inside the measure) | 3/3 | 3/3 |
Things I noticed:
- With no context, Agent mode wrote "This includes orders of every status." It states the assumption, but it doesn't ask.
- The comments show up as the table and column descriptions under Configure > Sources. Two ALTER statements and it was fixed.
- The measure editor wants table-qualified columns. `SUM(CASE WHEN status = 'F' THEN total_price END)` fails with "Table name or alias is required for column 'status'". `orders.status` works.
- In Chat mode, Genie reused my example query word for word, `:year` parameter included. In Agent mode it didn't use the measure and wrote `WHERE status = 'F'` itself (still right).
- The metric view agent used `MEASURE(revenue)` every time. The measure:
```yaml
- name: revenue
expr: SUM(total_price) FILTER (WHERE status = 'F')
```
Bonus: I also published a Unity Catalog Page called "Revenue". Genie One found it through the ontology ("Read page: Revenue"), answered 32.69B, said the definition "isn't certified yet" and listed the orders it left out. Best answer of the lot.
Caveats: one run per mode, a simple rule, one table. Genie One also looked at an older table and agent in my workspace, but its reasoning says the Page decided the answer. And for a rule this simple even plain text worked, so this doesn't settle "structured context beats text" either way.
Questions for you:
- Has anyone seen text instructions break where a metric view or SQL expression held up? What kind of rule was it?
- Where do you keep business rules for Genie today?
Links:
- The Databricks answer to my question: https://community.databricks.com/t5/data-governance/do-genie-agents-use-unity-catalog-wikis-as-context/td-p/175921
- Genie benchmarks: https://docs.databricks.com/aws/en/genie/benchmarks
- Unity Catalog Pages: https://docs.databricks.com/aws/en/uc-semantics/pages