Ingest multiple data-sources and join them to analyze simulated bank fraud transactions and build a machine learning model to predict fraud.

The dataset is based on the PaySim, dataset (GitHub repo and LICENSE for more details). The Databricks demo version augmented this dataset to introduce simulated geo-data as well. The Databricks version of this data has a LICENCE for use.

Files are staged in Unity Catalog volumes and then ingested into Delta Tables.

Most steps described can be executed from a local IDE running with a local clone of this Git repo. Use the Databricks SDK and CLI to authenticate to a remote workspace and it's services.

The Databricks CLI profiles are configured in ~/.databrickscfg.

• Check the available CLI profiles to connect to: databricks auth profiles
• Authenticate to a configured profile using U2M OAuth: databricks auth login -p my_profile_name.

Either

• set the DEFAULT profile to work in with the Databricks CLI by editing ~/.databrickscfg and setting the profile name in the [DEFAULT] section
  Or
• Add the -p <my_profile_name> to the end of all Databricks CLI commands.
  in order to make sure they get executed against the correct workspace.

• Identify / Create the Unity Catalog (UC) Catalog
• Identify / Create the UC Schema
• Create the Volume bank-fraud in the UC catalog.schema

Use the create.sh and destroy.sh scripts to deploy or tear down the demo from the repo root. They support both non-interactive (one-shot, suitable for CI or AI agents) and interactive (step-by-step prompts) modes.

Non-interactive (all parameters on the command line):

./create.sh --non-interactive --profile myprofile --catalog main --schema default --volume bank-fraud
# ... later, to tear down:
./destroy.sh --non-interactive --profile myprofile --catalog main --schema default --volume bank-fraud

Interactive (prompted for profile, catalog, schema, volume):

./create.sh
# or with partial args:
./create.sh --catalog main --schema default

Optional: sync notebooks to the workspace and create the UC volume if it does not exist:

./create.sh -y -p myprofile -c main -s default -v bank-fraud --workspace-path /Users/me@example.com/dbx-bank-fraud --create-volume

For full options run:

• ./create.sh --help
• ./destroy.sh --help

After the base data and tables are in place (e.g. via create.sh), use deploy.sh to deploy and build the ML artefacts and the Retail Bank Fraud dashboard:

1. Notebooks — Imports fraud_model_training.py, fraud_model_deploy.py, and fraud_model_run.py from ./notebooks into your workspace.
2. ML build — Runs the training notebook then the deploy notebook (so the model is registered in Unity Catalog). By default uses serverless compute (no cluster ID needed). Optionally pass --cluster-id to use an existing cluster. If a notebook run fails, the script prints the run ID and fetches the run output; you can also run databricks -p <profile> jobs get-run-output <run_id> to view error details.
3. Dashboard — Creates and publishes the Lakeview dashboard from dashboards/Retail_Bank_Fraud_Dashboard.lvdash.json, with your catalog and schema applied to the datasets.

NOTE: make sure a folder structure is in place in the workspace for the notebooks to be copied to. This needs to include a pre-created notebooks folder, i.e. pre-create /Users/me@example.com/dbx-bank-fraud/notebooks

Example (serverless; no cluster):

./deploy.sh -p myprofile -c my_catalog -s my_schema --workspace-path /Users/me@example.com/dbx-bank-fraud --warehouse-id <sql-warehouse-id>  --serve-model

Example (with existing cluster):

./deploy.sh -p myprofile -c my_catalog -s my_schema --workspace-path /Users/me@example.com/dbx-bank-fraud --warehouse-id <id> --cluster-id <cluster-id>

Example (non-interactive, serverless):

./deploy.sh -y -p myprofile -c my_catalog -s my_schema --workspace-path /Users/me@example.com/dbx-bank-fraud --warehouse-id <id>

Example (with model serving endpoint):

./deploy.sh -y -p myprofile -c my_catalog -s my_schema --workspace-path /Users/me@example.com/dbx-bank-fraud --warehouse-id <id> --serve-model

Example (serve-model only; no workspace-path or warehouse-id):

./deploy.sh -y -p myprofile -c my_catalog -s my_schema --skip-notebooks --skip-ml --serve-model

Example - Use the Databricks CLI to flag a model version production then serve that version

# Set the production alias on a model version
databricks -p myprofile registered-models set-alias <catalog>.<schema>.bank_fraud_predict production <version_num>
# Deploy "bank-fraud-predict" serving endpoint on bank_fraud_predict@production
./deploy.sh -y -p myprofile -c my_catalog -s my_schema --skip-notebooks --skip-ml --serve-model

Options:

• --skip-ml — do not run training or deploy notebooks.
• --skip-dashboard — do not create the dashboard.
• --skip-notebooks — do not import notebooks (e.g. dashboard-only or serve-model-only).
• --genie — create a Genie space for gold_transactions (warehouse ID required when using --genie).
• --cluster-id — optional; omit for serverless.
• For deploy types that only create the dashboard or --serve-model fraud endpoint, use --skip-notebooks --skip-ml; then --workspace-path and (for serve-model-only) --warehouse-id are not required. --deploy-agent always needs --workspace-path.

Model serving (when using --serve-model):

• Endpoint name: --endpoint-name (default bank-fraud-predict) is the name of the serving endpoint in the workspace.
• Model name: --model-name (default bank_fraud_predict) is the Unity Catalog model name; full name is catalog.schema.model_name.
• Version: Omit --model-version to use the version with alias production, or if none, the latest version.

Run ./deploy.sh --help for all options.

Undeploy: Run undeploy.sh to trash the dashboard, any Genie space, and the model serving endpoint created by deploy (and optionally remove the workspace path with --remove-workspace). It does not unregister ML models or delete experiments.

./undeploy.sh -p myprofile
./undeploy.sh -p myprofile --remove-workspace

The dashboard template lives in dashboards/Retail_Bank_Fraud_Dashboard.lvdash.json; a copy remains in sql/ for reference.

Rest API JSON Payload - predicts 1 (Fraud)

{
  "dataframe_records": [
    {
      "countryOrig_name": "Turkey",
      "diffDest": 400000,
      "diffOrig": -400000,
      "oldBalanceDest": 0,
      "amount": 400000,
      "type": "TRANSFER",
      "newBalanceDest": 400000,
      "countryDest_name": "Canada"
    }
  ]
}

Rest API JSON Payload - predicts 0 ( Not Fraud)

{
  "dataframe_records": [
    {
      "countryOrig_name": "Russian Federation",
      "diffDest": 0,
      "diffOrig": 358291.03,
      "oldBalanceDest": 1278730.92,
      "amount": 358291.03,
      "type": "CASH_IN",
      "newBalanceDest": 1278730.92,
      "countryDest_name": "Russian Federation"
    }
  ]
}

• Create two UC functions: explain_transaction_risk() (historical, reads gold_transactions) and explain_live_transaction_risk() (live, reads live_transactions with CDC dedup; baselines from gold_transactions).
• Register the UC model fraud_model_explain (LangChain + ChatDatabricks + foundation LLM) — one agent, both functions registered as tools.
• Serve it as a model serving endpoint (default name via deploy: bank-fraud-explain — hyphens; UC model uses underscores)

The agent picks which tool to call from the conversation:

• Explicit tag: "Why does live txn 10003599 look risky?" or "... historical txn 620650 ...".
• Session context: open the chat with "We are working off live transactions for this session." then ask bare ids.
• If unspecified, the agent defaults to live and tells you which dataset it queried.

See Live agent and dual-tool routing below for the schema migration and prompt-pattern details.

Automated run (assumes agent notebooks are already in the workspace, or omit --skip-notebooks to import everything first):

./deploy.sh -y -p myprofile -c my_catalog -s my_schema \
  --workspace-path /Users/you@example.com/dbx-bank-fraud \
  --skip-notebooks --skip-ml --skip-dashboard \
  --deploy-agent

** NOTE ** allow up to 20 minutes after the register and deploy agent has completed. It takes a while for the serving endpoint to come on-line.

Optional: --llm-endpoint <name> (default databricks-gpt-5-2), --agent-serving-endpoint-name <name> (default bank-fraud-explain), --agent-register-timeout <seconds> (default 5400 for the register/deploy notebook). Uses serverless compute unless you pass --cluster-id.

Test the agent to explain Fraud using the AI Playground. Try transaction ID 3402687 as an example.

Pre-reqs:

1. Lakebase Database: create a lakebase database in the workspace.

• Create the database in a project called retail-fraud

• The App connects to the database with this base URL: projects/retail-fraud/branches/production/endpoints/primary

• Note the Lakebase hostname (use the "Connect" button to find this)

• Note the endpoint details

• Get the lakebase project ID: databricks postgres list-projects -p <my-dbx-cli-profile>

• Edit the local app/.env and supply the correct hostname (used for local workstation connectivity only)

The app.yaml file has the following default configuration

env:
  - name: MODEL_SERVING_ENDPOINT
    value: "bank-fraud-predict"
  - name: LAKEBASE_ENDPOINT
    value: "projects/retail-fraud/branches/production/endpoints/primary"

For simplicity, make sure the Lakebase deployment in the following steps map to this Lakebase endpoint, or update and deploy a new app.yaml

Check the Lakebase services are there and connectivity is set up correctly in /app/.env

• python scripts/lakebase_smoke_test.py

Tables are created on App Startup if they don't already exist. Deploy the app, get it connected to the database with the correct privs, then the table should appear.

• Multiple manual steps are required to achieve this.
• These could be wrapped in another deploy.sh style script, but they have been left as manual steps to make the underlying CLI process transparent
• See Deployment Steps below and also Post-deploy Lakebase bootstrapping:

1. Ensure frontend is built:

cd <repo-dir>/app/frontend && npm run build 

2. Create the app (first time only):

databricks apps create fraud-analytics --profile <my-profile>

3. Sync app files to workspace:

databricks sync <repo-dir>/app /Workspace/Users/<user.name>@databricks.com/fraud-analytics --profile <my-profile>

4. Upload the built frontend separately (since .gitignore excludes dist/):

databricks workspace import-dir <repo-dir>/app/frontend/dist /Workspace/Users/<user.name>@databricks.com/fraud-analytics/frontend/dist --profile <my-profile> --overwrite

5. Deploy

Note that the app/app.yaml configuration binds the app to the Lakebase DB. Check that this links up with the target lakebase DB.

databricks apps deploy fraud-analytics --source-code-path /Workspace/Users/<user.name>@databricks.com/fraud-analytics --profile <my-profile>

6. Check status and get the app URL:

databricks apps get fraud-analytics --profile <my-profile>

7. Get the Service Principal Name for the app:

databricks apps get fraud-analytics --profile <my-profile> | grep service_principal

(Alternatively Compute → Apps → → identity in the Workspace UI)

8. Grant permissions for the App Service Principal to Query the bank-fraud-predict endpoint

Databricks Workspace UI -> Serving -> Permissions -> Add <service_principal_client_id> with "Can Query" privs

9. Restart the app (Re-Deploy)

10. Subsequent changes to code: Updates after code change - after changing backend/frontend, repeat build → sync → import-dir dist → deploy (or at least sync + dist when only UI changes).

Declaring resources: in app/app.yaml is not enough on its own to make the deployed app talk to Lakebase. The Databricks Apps platform only injects PGHOST / PGPORT / PGDATABASE / PGUSER (and sometimes PGPASSWORD) once the resource is bound to the app in the UI, and the app's service principal (SP) still has to be granted Postgres-level privileges separately from any Databricks-level ACLs.

Work through this checklist the first time you deploy to a new workspace. The same steps apply after restoring or recreating the Lakebase instance.

1. Bind the Lakebase + Serving resources in the App UI.

   Compute → Apps → `fraud-analytics` → Settings → Resources → Add resource:

• Database (Lakebase)
  • Resource key: lakebase-db ← must match the name: in app.yaml
  • Database instance: your Lakebase instance
  • Database name: databricks_postgres
  • Permission: Read and write

After saving, the app restarts and the Environment tab should show PGHOST, PGPORT, PGDATABASE, PGUSER. If these never appear, the binding did not take — fix that before anything else.

2. Grant Postgres privileges to the app SP. "Can manage" on the Lakebase instance is a Databricks-level ACL — Postgres inside the database needs its own grants. Connect via the Lakebase web-UI and run:

 -- Replace with the SP UUID from PGUSER (also = DATABRICKS_CLIENT_ID in the
 -- app Environment tab). Keep the double quotes.
 GRANT USAGE, CREATE ON SCHEMA public TO "<APP_SP_UUID>";
 GRANT SELECT, INSERT, UPDATE, DELETE ON scored_transactions TO "<APP_SP_UUID>";
 ALTER DEFAULT PRIVILEGES IN SCHEMA public
   GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO "<APP_SP_UUID>";

Link to Lakebase Debug Queries

3. Set up Lakebase to Lakehouse Delta replication

• Use the notebook ./notebooks/setup_lakebase_to_live_txns.py to guide checking the setup of syncing transaction from Lakebase back to a Delta table in Unity Catalog.
• The Lakebase live_transactions table should already have been set with full replica status enabled. This is done automatically by the App. o it needs to be done this way as the App is the table owner (it created the Lakebase table).
• Raw replica data gets written back to Delta table lb_scored_transactions_history
• The notebook is used to to create the live_transactions view over the raw transactions history. This is used by the AI Agent for explaining transactions

The fraud-explanation agent serves both historical and live transactions from a single endpoint (bank-fraud-explain). The LLM picks which UC function to call from the conversation context. There is also a one-time schema migration to bring the live data up to the column shape the risk function needs.

Schema migration — adding the four balance columns to scored_transactions.

The risk-scoring function uses oldBalanceOrig / newBalanceOrig / oldBalanceDest / newBalanceDest to compute the "drains origin" and "empty destination" signals (50 of the 100 risk-score points). The Lakebase table did not originally persist them, so the live Delta replica did not have them either. The DDL in app/backend/db.py now includes these columns (old_balance_orig, new_balance_orig, old_balance_dest, new_balance_dest); the generator already produces the values, so no other app-side code change is needed.

Migration steps (one-time, in order):

1. Stop the fraud-analytics app (or click Reset Data in its UI to truncate first).

2. In Lakebase psql, drop the table so the app SP recreates it as the owner on next start (the existing table is owned by your user, so ALTER TABLE ADD COLUMN would also work but the rest of the demo's drop/recreate muscle memory applies):

   DROP TABLE IF EXISTS scored_transactions CASCADE;

3. Restart the app. ensure_tables() recreates the table with the four new columns, the indexes, and REPLICA IDENTITY FULL.

4. In the Lakebase Sync UI, Disable the existing sync to live_transactions, drop the Delta target table, then Enable sync again so it picks up the new schema.

After a few transactions stream through, confirm with:

SELECT id, type, amount,
       old_balance_orig, new_balance_orig,
       old_balance_dest, new_balance_dest,
       fraud_prediction
FROM live_transactions
ORDER BY _pg_lsn DESC
LIMIT 5;

The two UC functions. Both are created by notebooks/agent/1. Create Function.py:

The live function wraps its tx lookup with a small CDC dedup so it scores the current state of an id rather than every event:

SELECT *
FROM (
  SELECT *, ROW_NUMBER() OVER (PARTITION BY id ORDER BY _pg_lsn DESC) AS _rn
  FROM live_transactions
  WHERE id = CAST(tx_id AS BIGINT)
)
WHERE _rn = 1 AND _pg_change_type <> 'delete'

_pg_lsn is monotonically increasing, so MAX(_pg_lsn) per id is the most recent event. We exclude delete events so a deleted id reports as not-found rather than as a stale row. Type-percentile baselines come from gold_transactions because the live table is too small/cold early in a demo to give meaningful p95/p99 amounts.

The dual-tool agent. notebooks/agent/2. Register Agent.py registers a single UC model fraud_model_explain whose toolkit includes both functions. Re-running the notebook end-to-end logs a new MLflow version and redeploys the same bank-fraud-explain serving endpoint — the URL is preserved, so notebooks/agent/3. Query Endpoint.py and any other downstream consumer keeps working without changes.

Prompt patterns for picking the right tool.

• Explicit tag (most reliable): "Why does live txn 10003599 look risky?" / "Why does historical txn 620650 look risky?". The keyword forces the right tool regardless of session context.
• Session context: open with "We are working off live transactions for this session." then ask bare ids. The LLM applies that context to subsequent turns until you say otherwise.
• Unspecified: defaults to live and the agent tells you which dataset it queried.

The web UI has a second tab, Operations, which simulates a simple business operations screen on top of the agent. It reuses the live SSE stream (no extra API polling), so the same data powers both tabs.

• Left pane — Fraud Inbox. Rolling list of the most recent 10 fraud-flagged transactions. Each row shows time, id, customer, type, amount, country pair, and a probability bar. A "Verified" checkbox is included as a cosmetic stub to illustrate a future verification workflow — it stores state in local React only and does nothing on the backend.
• Right pane — Fraud Explanation Agent. Chat dialog that talks to the bank-fraud-explain serving endpoint. Selecting a transaction pre-fills "Why does live txn {id} look risky?" (explicit tag, so the dual-tool agent reliably picks explain_live_transaction_risk). Pressing Send posts the chat history to the new backend proxy /api/explain-fraud; the proxy calls the agent via the Databricks SDK, keeping the workspace bearer token server-side.
• Backend proxy. app/backend/main.py exposes POST /api/explain-fraud, which accepts {messages: [{role, content}]}, calls WorkspaceClient().serving_endpoints.query(...) on FRAUD_EXPLAIN_ENDPOINT (defaults to bank-fraud-explain, overridable via env var), and returns {role: "assistant", content: "..."}. The blocking SDK call is offloaded with asyncio.to_thread, and failures surface as HTTP 502 so the UI can render a graceful fallback.
• Resource binding. app/app.yaml now declares a fraud-explain-agent resource for the bank-fraud-explain endpoint. After redeploy you must bind it in App → Settings → Resources (same drill as the original fraud-model binding) so the app SP gets CAN_QUERY on the agent endpoint.

Operational checklist after pulling these changes.

Follow the same four-command deploy flow as Deployment Steps - Databricks CLI. Skipping the import-dir step is the most common failure mode: the built frontend bundle lives in app/frontend/dist/, which is listed in .gitignore, so databricks sync will not upload it on its own. Without the import step the app keeps serving the previously-deployed bundle and any new UI (for example the Operations tab) is invisible.

# 1. Rebuild the React bundle (required after any frontend source change)
cd app/frontend && npm run build && cd ../..

# 2. Sync Python sources + app.yaml to the workspace
databricks sync app \
  /Workspace/Users/<user.name>@databricks.com/fraud-analytics \
  --profile <my-profile>

# 3. REQUIRED: push the built bundle separately (dist/ is gitignored)
databricks workspace import-dir app/frontend/dist \
  /Workspace/Users/<user.name>@databricks.com/fraud-analytics/frontend/dist \
  --profile <my-profile> --overwrite

# 4. Redeploy
databricks apps deploy fraud-analytics \
  --source-code-path /Workspace/Users/<user.name>@databricks.com/fraud-analytics \
  --profile <my-profile>

Then:

5. Bind the agent resource (one-time, required). In the app UI: Compute → Apps → fraud-analytics → Edit → Resources → Add resource. Type Serving endpoint, resource key fraud-explain-agent (must match app.yaml), endpoint bank-fraud-explain, permission Can Query. Save and let it redeploy. Without this the chat returns 502 PermissionDenied: You do not have permission to query the endpoint.
6. Restart the app and confirm /logz is clean.
7. Hard-reload the browser (Cmd+Shift+R on macOS). The index.html references a hash-named JS bundle, so a cached index.html can still point at the previous hash even after a redeploy.
8. Open the Operations tab, start streaming on the Dashboard tab, wait for a fraud to appear, select it, then press Send on the pre-filled prompt. You should see an explanation within a few seconds.

Sanity check if the UI looks stale. Open browser devtools → Network and inspect the request for /assets/index-*.js. Compare the hash against the file you just built locally (ls app/frontend/dist/assets/). A mismatch means the import-dir step either did not run or imported into the wrong workspace path.

GET /api/db-status Confirms lakebase vs mock, connectivity, host, database.

GET /api/metrics Top-line counters (total, fraud count/rate, total amount).

GET /api/transactions?limit=50 Latest scored transactions (limit up to 200).

GET /api/country-flows Aggregated source/target country flow stats.

GET /api/time-series Time-bucketed totals/fraud/legit for charting.

GET /api/events SSE live stream (what drives the “Live” badge/data updates).

GET /api/generator/status Current generator state/speed.

POST /api/generator/start Start generating transactions.

POST /api/generator/stop Stop generating transactions.

POST /api/generator/speed with body {"tps": 5} Set transactions-per-second.

POST /api/reset Stops generator, clears data, resets counters.

POST /api/explain-fraud Chat proxy to the fraud explanation agent (bank-fraud-explain).