Puts the converged pick-and-place policy into an indoor environment by compositing
the live drone+cube over a Gaussian-splat reconstruction of a room (the GS pipeline
we used before, applied as a rollout backdrop; physics stays on the flat floor).

- pick_place_env: flag-guarded indoor_room (visual-only floor/walls/ceiling/furniture)
  + semantic_tags on drone/cube + depth & semantic_segmentation on the render camera.
  All behind cfg flags so training is unaffected.
- render_rollout_gs.py: orbits the camera to fuse ONLY room pixels (drone/cube masked
  via semantic seg) into a GaussianMap, then each rollout frame renders the GS room at
  the follow-cam pose and composites the segmentation-masked drone+cube on top.
  Requires PYTHONUTF8=1 (gsplat JIT-compiles its CUDA kernels; torch JIT needs UTF-8).

Output: videos/isaac_pickplace_gs.mp4 -- drone flies in, grasps, and carries the cube
inside the splat room.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add cfg knobs so the cube/goal can be placed further and rollouts rendered higher-res:
- obj_spawn_diam (cube uniform +/- diam/2 around origin; was hardcoded 0.8)
- goal_offset_diam (goal uniform around cube; was hardcoded 0.5)
- cam_w / cam_h (render-camera resolution; was hardcoded 720x540)
Defaults preserve prior behavior. Used by the object-further training panel + HQ render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ild)

Sim-trained Neural Aerial Transport and Capture, per snatch/DESIGN.md. Built by a
4-agent swarm + orchestrator integration; single-DOF caging gripper, direct velocity
control, dual-camera visuomotor policy, sim-to-real GAP ANALYSIS (no real flights).

Modules (each unit-tested):
- assets/drone_snatch.urdf (+USD): single-DOF 4-jaw cage gripper (no lower DOF; drone
  descends bodily), top/bottom camera mounts. Scripted grasp gate PASS.
- snatch/perception.py: dual TiledCamera depth sensors (top fwd 87deg, bottom down 120deg)
  + ResNet-18 depth encoders -> 1024-d latents. TiledCamera (not Camera) so visuomotor RL
  scales to many envs without exhausting the RTX descriptor pool.
- snatch/randomization.py: full DR table + apply_vio_drift (headline localization-gap knob)
  + apply_detection_noise + depth noise/latency/ground-effect. 7 tests pass.
- snatch/rewards.py: 4-component reward (nav/align/alt/grasp/transport/place) + heuristic
  grasp trigger. 13 tests pass.
- snatch/pick_place_env.py: DirectRLEnv wiring all four; 5-action [vx,vy,vz,yaw,gripper];
  obs = 1024 latents + 11 state (VIO-drifted pose, detection-noised block).
- scripts/{convert_snatch,snatch_grasp_test,train_snatch,eval_snatch}.py; eval has the
  VIO-noise success sweep (the core sim2real gap result).

Verified: state smoke PASS, visuomotor smoke PASS, rsl_rl train pipeline PASS (TiledCamera,
128 envs, iterations stepping with finite reward + metrics).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…no_cams state variant

The bare SNATCH spec reward is too sparse to bootstrap grasp from altitude. Add (additively,
on top of A4's spec reward) the proven straddle-start curriculum (anneal 0.85->0.15) + a
strong-but-low-plateau lift gate + held carry-up/carry-progress shaping. train_snatch gains
--no_cams for the tractable state-based variant (the VIO-drift gap study target).
Verified learning: grasp_rate climbing from iter ~20 at 2048 envs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…-gap sweep)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

First state-based run reached grasp 0.67 but place~0 and obj_to_goal blew up to 6.5m: the
lift/grasp shaping dominated A4's weak placement so the policy grabbed and flew off. Replace
with the proven balance (60*cube_h gate + 40*held*place + 25*carry_prog + 80*success), A4's
spec reward kept as 0.1x aux. Verified: obj_to_goal now stable ~0.64 and place_success rising.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…weeps the gap

Training under vio_drift_scale=1.0 corrupted the drone's own pose estimate so it could not
descend onto the cube -> grasp_rate DECAYED (0.20->0.12) and place stalled. Correct design:
train clean (vio_drift_scale=0, detection_noise_scale=0), and eval_snatch sweeps vio drift to
measure the sim2real gap. Also restored speed=1.5 (proven) for precise bodily descent.
Verified: grasp_rate 0.43 by iter 103 (was 0.18), place_success rising, obj_to_goal stable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… up later

At diam 1.2/1.0 grasp oscillated ~0.4-0.58 and place stalled ~0.04. Drop to the proven
0.8/0.5 (drove the base task to 99.8%) to converge the single-DOF variant; distance is
parameterized so it can be raised after. Localization stays clean for training.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

place_success plateaued ~0.15 because the goal waypoint (randomized per episode) was NOT in
the obs -- the policy could not know where to deliver. Add goal_rel = target-drone to the
state (11->14). Key blocker for directed placement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Expose _grasped/_placed/_grasp_pos_err aliases for eval_snatch's metric contract (were
_held/_success/_d_reach -> eval read 0%). Add eval --cur_p to pin the start distribution for
an honest from-altitude eval.

RESULT (state-based, model_650, cur_p=0.3):
  clean localization (drift 0.0): pick 96.3% / place 94.7% / e2e 94.7% / grasp_err 1.6cm
  full VIO drift   (drift 1.0):  pick 56.3% / place 51.0% / e2e 51.0% / grasp_err 2.4cm
The headline sim2real gap: directed aerial pick-and-place degrades 94.7% -> 51.0% as
localization goes from perfect to realistic VIO drift.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…nore .claude

- drone_snatch.usd references configuration/drone_snatch_{base,physics,sensor}.usd which were
  untracked -> the committed asset would not load on a fresh checkout. Commit them.
- Relocate A2's snatch_cam_smoke.py into skyvla_isaac/scripts/ (was at a stray top-level scripts/).
- gitignore .claude/ (agent worktrees / local state).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Adds cfg.render_camera (RGB follow cam) to DroneSnatchEnv and scripts/render_snatch.py to
record a state-based pick-and-place rollout to mp4. Verified the policy lifts+carries
(cube 0.025->0.37m, delivered within 3cm of goal, 100% on straddle starts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

User: gripper must be DIRECTLY UNDER the drone, not dangled on a standoff. Moved the 4-jaw
cage from -0.33 to -0.07 (flush beneath the body). A flush cage + a FLOOR cube forces the
body down ~9cm so the base scrapes the ground (physics ~8x slower, grasp stalls). Fix: pick
off a raised table (kinematic platform, top 0.30m) -- the cage grasps flush under the body
while the body stays well clear of the floor. Heights made relative to cfg.surface_z.
Verified: physics speed restored (45k steps/s), grasp+place climbing together (no fly-away).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…etting

The 0.85->0.15 anneal destabilized PPO on the harder flush-gripper/table task: success
peaked ~0.59 then DECLINED to ~0.27 as the distribution shifted faster than the policy
tracked. A fixed 0.6 straddle/fly-in mix is a stationary target -> stable (no decline);
plateaus ~0.49 overall (straddle ~0.75; from-altitude-onto-table remains the hard wall).
Flush-under-body gripper picks+carries cleanly at straddle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…overing)

Per user: the raised surface must read as a real table on the floor, not floating. Narrow the
kinematic platform to a 1.0x1.0m solid pedestal spanning z=[0,0.30] (bottom flush on the ground).
Physics unchanged: top surface still at 0.30 so the trained policy works as-is; static collision
like the floor (no hardcoded grasp anywhere -- flight=forces, grasp=jaw contact).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…>lift (was stuck hovering)

Picking failure root cause: the policy parked in a HOVER local optimum (hover_ema 0.97,
grasp_ema 0.0007 flat for 900 iters) -- it could score by hovering near the cube, and
descending risked the table-crash, so it never went down to pick.

Fix (user's idea): the approach/hover reward now ANNEALS down (approach_w 1.0->0.2 over
~100k steps) and is sharpened to 3D distance (/0.3) so it pulls the gripper DOWN onto the
cube, not just laterally over it. Added a held-grasp milestone bonus (+5). The constant,
large lift(60*cube_h)+carry+place rewards then dominate -> a strict ladder
descend->grab->lift->carry->place with no flat hover plateau. Use the non-staged path
(drop --staged_curriculum, which perfected hover first and caused the trap) with the
straddle curriculum (--cur_p 0.6) that demonstrably taught grasp (v6 -> 0.84).

Verified: grasp climbs 0.02->0.13 and rising (vs 0.0007 stuck), place tracking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- train_snatch: reverse/staged/adaptive curriculum flags, --rc_distance_mode
  (spawn-distance curriculum to --rc_dist_max), --resume, --reset_std,
  --entropy_coef, cube mass/size and speed overrides
- pick_place_env: distance-mode reverse curriculum support
- gs: cacheable GS backdrop rendering (gs/cache.py, CACHING.md,
  render_gs_cache.py) so splat rooms re-render Isaac-free
- render_snatch/render_rollout_gs: quality presets, cache integration
- diagnostics: _eval_rate.py, _traj_diag.py
- gitignore GS caches (*.npz, gs/cache/)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>