অলীকবচন — Bengali LLM Hallucination Detection

Inference pipeline for the IUT Datathon Bengali hallucination-detection challenge. Given a Bengali (context?, prompt_bn, response_bn) triple, predict 1 = faithful or 0 = hallucinated (macro-F1).

1. Open kaggle_inference.ipynb on Kaggle.
2. Accelerator: GPU T4 × 2, Internet: ON.
3. Attach the two public datasets (see Datasets below).
4. Run All → produces /kaggle/working/submission.csv.

The competition test path is hard-coded exactly as required and must not be changed:

/kaggle/input/competitions/bengali-hallucination/test set.csv

Everything you might need to change (dataset paths, model IDs) is in the single CONFIG block at the top of the notebook. Nothing else needs editing.

Runtime: ~3–4 h on the 2,516-row Phase-1 set; ~6–7 h on ~5,000 rows — within the 9 h limit.

Each item is routed (CONTEXT / NULL_MATH / NULL_FACT / NULL_PROVERB). Evidence is retrieved from a prebuilt BGE-M3 dense index over our Bengali knowledge base (QA banks + grammar/proverb corpora), reranked by bge-reranker-v2-m3. Qwen2.5-7B then answers the question — from evidence (RAG mode) or step-by-step with a \boxed{} final value (MATH mode). The generated answer is compared to response_bn by a dual comparator: a VALUE check (numbers, years, and Bengali dates must agree — e.g. "২৫ মার্চ ১৯৭১" ≠ "১৯৭২ সালের...") plus a SEMANTIC check (near-equality → containment → embedding cosine → NLI). A verified-answer overlay (*.csv keys and final_qlabel.csv) matched by normalized question text takes top priority. Rows where the system abstains fall back to the majority-class prior.

To point at different datasets, edit KB_DS / KEYS_DS in the CONFIG block.

No fine-tuned checkpoint is used; all weights are downloaded from public repos.

The attached KB_DS was built locally with build_index.py from our knowledge sources (qa_banks/, corpus/). Evaluators do not need to rebuild it — it is provided prebuilt so the notebook stays within the time limit. To rebuild:

python build_index.py --kb-dirs qa_banks corpus --out rag_index

kaggle_inference.ipynb   the submission notebook (documented; Run All)
kaggle_inference.py      same code as a plain script (for diffing/review)
build_index.py           builds the BGE-M3 index (produces KB_DS contents)
README.md                this file