Ujwal Dinesha

TL;DR. ARC-AGI-3 drops an agent into interactive grid games with hidden mechanics, sparse rewards, and a tight interaction budget. The open secret of the benchmark is that the harness — the scaffold of tools, memory, and control logic around the LLM — matters as much as the model. ARC-SCOPE is a testbed that finally lets you compare harnesses fairly: same environments, same representations, same token / wall-clock budgets.

Within it we compared reactive analyzers, multi-agent orchestrators, and executable-world-model agents, and built DISH — a deterministic information-state harness that maintains a programmatic world model, verifies it against observed transitions, and plans through it with a thin event-driven controller, designed for sub-30B models.

The result is a capability–efficiency split: the strongest frontier-style baseline tops out at 36/37 levels, while DISH clears 28–31 levels at roughly half the tokens per level (≈20M vs. ≈43M). Everything runs on vLLM with FP8, from a single-GPU Qwen-27B up to GLM on an 8×H200 node.

TL;DR. Most alignment pipelines stack two fragile pieces — fit a reward model on noisy preference data, then fine-tune the LLM against it. PITA throws both out. We learn a small guidance policy that nudges the base model’s next-token distribution at inference time, directly from preferences, with zero LLM fine-tuning.

The trick is to frame alignment as identifying a latent preference distribution and solve it with stochastic search. The guidance model produces exponentially-weighted Q-values that re-shape the LLM’s logits on the fly. The result: a much cheaper alignment recipe that side-steps reward-model instability.

We test PITA on math reasoning, TL;DR summarization, and sentiment control. The base LLM stays frozen the whole time — you can swap it out, swap the guidance policy in, and keep going.

TL;DR. Restless multi-armed bandits (RMABs) usually assume the decision-maker observes a clean scalar reward when they pull an arm. In the real world — healthcare, online ads, app engagement — you almost never get that. You get which option did the user prefer. DOPL is the first online algorithm for RMABs in this setting with provable regret.

We introduce Pref-RMAB, where the planner sees only pairwise preference feedback between activated arms. DOPL interleaves exploration of the unknown transition kernels with preference-data collection, then plans directly against preferences instead of trying to reconstruct rewards. The analysis lands at Õ(√(T · ln T)) regret.

Practically, we show it works on three benchmark domains — a CPAP adherence model, the Armman maternal-health setup, and an app-marketing engagement environment. Codebase is built around a config-driven runner so it’s easy to plug in your own RMAB environment.

TL;DR. Standard RLHF maximizes expected reward, so it’s perfectly happy with a model that’s great on average but occasionally spits out something toxic. RA-RLHF swaps the objective for Conditional Value at Risk — the average reward of the worst-α tail — which directly punishes those rare bad generations.

The fix is surprisingly clean: at each PPO step, we sort the batch by reward and weight the tail more aggressively. No new architecture, no new reward model, drop-in over TRL.

On IMDB sentiment-control and Jigsaw toxicity-mitigation benchmarks, RA-RLHF cuts the rate of toxic / off-target outputs without giving up on the generation quality you’d get from vanilla RLHF.

TL;DR. A base station is streaming video to N devices over a shared wireless link with a tight energy budget. Each device’s buffer / channel state arrives back at the BS with a deterministic delay. That delay is the whole problem — the controller is acting on stale information.

We cast this as a cooperative multi-agent constrained POMDP and use strong duality to split it into N independent transmitter-receiver pairs that all share a single Lagrange multiplier (the energy price). Each pair is then solved using the common-information approach with approximate information states.

The neural architecture that drops out of this analysis is delay-aware by construction and beats transmitter-only baselines in simulation. It’s a nice example of how a careful POMDP decomposition can give you tractable learning where the naive joint problem would be hopeless.

TL;DR. Open RAN’s near-real-time RIC sits in the cloud and takes > 15 ms to decide anything. That’s far too slow for per-TTI scheduling in a real cell. EdgeRIC pushes the RIC down to the edge, co-located with the DU, and gets the decision loop under 1 ms.

The system exposes a real-time E2 interface (we call it RT-E2) built on ZMQ + protobuf, with TTI-level synchronization and per-UE KPI reports. Above that we run μApps — small policies, including a PPO-trained MAC scheduler — that can react fast enough to actually matter.

End-to-end, the AI-driven scheduler beats classical heuristics (max-CQI, proportional fairness, round-robin) by 5–25% on throughput and downstream application metrics, all on real srsRAN hardware.

TL;DR. Two hands-on threads outside the paper mill. First, navigation on the comma body robot: pure imitation learning with a VQ-GAN visual tokenizer feeding a GPT-2-style transformer that doubles as policy and world model. Trained on teleoperation laps I collected around the lab, the robot autonomously repeats those laps end-to-end.

Second, manipulation with LeRobot and SO-101 arms, where I mentored an undergraduate team training ACT (from the ALOHA paper), Diffusion Policy, and π₀-style policy architectures on real hardware.