# CELL 15 — RUN SCREENING SUITE FOR BOTH CHECKPOIN...
생성일: 2026년 10월 25일
답변에 사용된 모델 GPT-5.6 Thinking by Chat01
생성일: 2026년 10월 25일
답변에 사용된 모델 GPT-5.6 Thinking by Chat01
screen_records = []
all_results = {}
def run_screen_for_checkpoint(epoch, adapter):
model = load_checkpoint(adapter)
texttry: for domain, df in [ ("clean", clean_screen_df), ("phone", robust_val_df), ]: for view_mode in [ "FULL_PAGE", "MOSAIC", ]: key = f"E{epoch}_{domain}_{view_mode}" print("\n" + "=" * 80) print(key) print("=" * 80) result = evaluate_view( model, df, domain=domain, view_mode=view_mode, desc=key, ) all_results[key] = result save_result( result, OUTPUT_DIR / "screening" / key, ) screen_records.append({ "epoch": epoch, "domain": domain, "view_mode": view_mode, "n": len(df), "accuracy": result["accuracy"], "macro_f1": result["macro_f1"], "balanced_accuracy": result["balanced_accuracy"], }) print( f"Accuracy={result['accuracy']:.4f} | " f"Macro-F1={result['macro_f1']:.4f} | " f"BalAcc={result['balanced_accuracy']:.4f}" ) finally: release_model(model)
run_screen_for_checkpoint(6, EPOCH6_ADAPTER)
run_screen_for_checkpoint(7, EPOCH7_ADAPTER)
screen_df = pd.DataFrame(screen_records)
screen_df.to_csv(
OUTPUT_DIR / "SCREENING_RESULTS.csv",
index=False,
)
Accuracy=0.4286 | Macro-F1=0.3924 | BalAcc=0.4162
FloatingPointError Traceback (most recent call last)
/tmp/ipykernel_23/3349340959.py in evaluate_view(eval_model, df, domain, view_mode, desc)
47 with precision_context():
---> 48 logits = constrained_class_logits(
49 eval_model,
/tmp/ipykernel_23/4098905817.py in constrained_class_logits(eval_model, batch, diagnose_nonfinite, force_vision_fp32)
177
--> 178 hidden_states = qwen_hidden_4513(
179 causal_lm,
/tmp/ipykernel_23/4098905817.py in qwen_hidden_4513(causal_lm, model_batch, diagnose_nonfinite, force_vision_fp32)
81 if diagnose_nonfinite and not torch.isfinite(image_embeds).all():
---> 82 raise_stage_nonfinite("vision", image_embeds)
83
/tmp/ipykernel_23/4098905817.py in raise_stage_nonfinite(stage, tensor)
30 frac = float(torch.isfinite(tensor).float().mean().detach().cpu())
---> 31 raise FloatingPointError(
32 f"NONFINITE_STAGE={stage} dtype={tensor.dtype} "
FloatingPointError: NONFINITE_STAGE=vision dtype=torch.float16 shape=(475, 2048) finite_fraction=0.99789470
The above exception was the direct cause of the following exception:
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_23/2691407900.py in <cell line: 0>()
55 release_model(model)
56
---> 57 run_screen_for_checkpoint(6, EPOCH6_ADAPTER)
58 run_screen_for_checkpoint(7, EPOCH7_ADAPTER)
59
/tmp/ipykernel_23/2691407900.py in run_screen_for_checkpoint(epoch, adapter)
22 print("=" * 80)
23
---> 24 result = evaluate_view(
25 model,
26 df,
/usr/local/lib/python3.13/dist-packages/torch/utils/_contextlib.py in decorate_context(*args, **kwargs)
122 # pyrefly: ignore [bad-context-manager]
123 with ctx_factory():
--> 124 return func(*args, **kwargs)
125
126 return decorate_context
/tmp/ipykernel_23/3349340959.py in evaluate_view(eval_model, df, domain, view_mode, desc)
53 )
54 except Exception as exc:
---> 55 raise RuntimeError(
56 f"Evaluation failed: domain={domain}, "
57 f"view={view_mode}, "
RuntimeError: Evaluation failed: domain=clean, view=MOSAIC, sample=4963.0|2297|bw_clean, label=CD, aug=clean
Can you give the fixed cell or whole notebook fixed please
I can fix this directly in the notebook. I’m tracing the FP16 non-finite path through the vision encoder and evaluation retry logic, then I’ll give you both a drop-in replacement cell and a patched .ipynb.
The failure is not in Cell 15 itself. The clean MOSAIC path is still running the Qwen vision encoder under FP16 autocast on a T4; the first bad sample produces non-finite vision embeddings, and the current “FP32 vision” helper is only enabled for the phone domain.
I’m patching the numerical policy so all local-evidence views get a stable vision pass, with a targeted retry instead of aborting the entire screening run.
I found one important subtlety in the original helper: in Transformers 4.51.3, Qwen2.5-VL’s patch embed explicitly casts pixels back to the Conv3D weight dtype. So pixel_values.float() alone does not make vision FP32; the visual tower weights/4-bit compute dtype must also be promoted. The patched notebook now does that once per loaded checkpoint, while leaving the language model quantized/FP16.
I’ve also made the precision policy identical for FULL_PAGE, MOSAIC, and MULTI_IMAGE, so the A/B comparison is not confounded by one view using FP16 vision and another FP32.
Yes. I patched the notebook.
The actual problem is upstream of Cell 15: E6_clean_MOSAIC produces non-finite values inside the FP16 vision encoder. The traceback shows the failure occurs at the vision stage with dtype=torch.float16, not in the screening loop itself. (v6-4a-frozen-local-evidence-ab.log)
There was also a subtle issue with the existing force_vision_fp32 approach: Qwen2.5-VL in Transformers 4.51.3 explicitly casts image inputs to the patch-embedding Conv3D weight dtype, so calling pixel_values.float() does not actually make the vision path FP32 when those weights remain FP16. (raw.githubusercontent.com)
I changed the notebook so the visual tower is genuinely promoted to FP32 when each checkpoint loads, including FP32 compute for visual bitsandbytes layers. The language model remains 4-bit/FP16, so this is much cheaper than making the entire Qwen model FP32. Vision SDPA math mode is applied only during the vision forward pass, and all FULL_PAGE, MOSAIC, and MULTI_IMAGE evaluations now use the same vision precision so your A/B comparison remains fair.
Files:
If you use the fixed notebook, run it from the top. If you want to patch your current Kaggle notebook, replace Cells 11, 12, 13, and 14 with the supplied versions, then rerun Cell 15. You do not need to rewrite the Cell 15 screening loop itself.
I also removed the stale failed outputs from Cell 11 onward and syntax-validated all modified cells. I couldn't execute the actual checkpoint inference here because your Kaggle datasets/checkpoints and T4 runtime aren't present in this environment, so the remaining validation is the real Kaggle run.