Both models remove without the first-frame anchor. Metrics: full fps-fixed test set (1,425 clips), paired per clip (Wilcoxon). KL is the softmax KL from the cnn14 classifier; LSD and KL are against the removal GT audio.
| metric | mainline ep018 | remove-only ep021 | ep018 better / worse (clips) | p |
|---|---|---|---|---|
| PSNR↑ | 20.64 | 20.20 | 906 / 519 | 6.1e-20 |
| SSIM↑ | 0.677 | 0.666 | 937 / 488 | 9.6e-27 |
| LPIPS↓ | 0.267 | 0.278 | 814 / 611 | 1.4e-08 |
| KL↓ | 1.348 | 1.324 | 697 / 728 | 0.077 |
| LSD↓ | 12.76 | 12.79 | 703 / 722 | 0.98 |
| DeSync↓ | 0.844 | 0.850 | 508 / 478 | 0.65 |
| FVD↓ | 128.7 | 182.1 | set-level | – |
| FAD↓ | 1.676 | 1.461 | set-level | – |
| IS↑ (cnn14) | 6.38 | 6.56 | set-level | – |
Summary. ep018 is clearly better on video (PSNR/SSIM/LPIPS, p≤1e-8; FVD 129 vs 182). Audio is a tie per clip (KL, LSD and DeSync not significant); remove-only has the lower set-level FAD (1.46 vs 1.68).
The 50 clips of test_fpsfix_rand50_seed0 (random sample, not cherry-picked). The winner on each metric is in bold.
input
GT
mainline ep018
PSNR 24.24 · LPIPS 0.103 · LSD 9.88 · KL 0.105
remove-only ep021
PSNR 21.51 · LPIPS 0.194 · LSD 9.65 · KL 0.165
input
GT
mainline ep018
PSNR 24.78 · LPIPS 0.196 · LSD 13.50 · KL 0.449
remove-only ep021
PSNR 24.93 · LPIPS 0.215 · LSD 13.92 · KL 0.339
input
GT
mainline ep018
PSNR 21.22 · LPIPS 0.205 · LSD 12.27 · KL 0.887
remove-only ep021
PSNR 16.78 · LPIPS 0.284 · LSD 13.91 · KL 1.036
input
GT
mainline ep018
PSNR 23.85 · LPIPS 0.159 · LSD 9.24 · KL 0.726
remove-only ep021
PSNR 25.93 · LPIPS 0.094 · LSD 9.01 · KL 0.693
input
GT
mainline ep018
PSNR 15.12 · LPIPS 0.452 · LSD 9.58 · KL 1.882
remove-only ep021
PSNR 15.25 · LPIPS 0.457 · LSD 8.99 · KL 2.351
input
GT
mainline ep018
PSNR 16.09 · LPIPS 0.334 · LSD 33.63 · KL 5.288
remove-only ep021
PSNR 15.90 · LPIPS 0.329 · LSD 32.69 · KL 4.629
input
GT
mainline ep018
PSNR 30.99 · LPIPS 0.060 · LSD 14.90 · KL 2.588
remove-only ep021
PSNR 18.89 · LPIPS 0.259 · LSD 14.31 · KL 3.425
input
GT
mainline ep018
PSNR 21.01 · LPIPS 0.153 · LSD 9.81 · KL 0.026
remove-only ep021
PSNR 21.34 · LPIPS 0.180 · LSD 10.04 · KL 0.019
input
GT
mainline ep018
PSNR 14.10 · LPIPS 0.525 · LSD 8.95 · KL 0.289
remove-only ep021
PSNR 14.97 · LPIPS 0.477 · LSD 8.78 · KL 1.470
input
GT
mainline ep018
PSNR 13.94 · LPIPS 0.434 · LSD 14.10 · KL 2.512
remove-only ep021
PSNR 12.93 · LPIPS 0.447 · LSD 9.63 · KL 1.199
input
GT
mainline ep018
PSNR 18.74 · LPIPS 0.381 · LSD 14.53 · KL 0.163
remove-only ep021
PSNR 19.29 · LPIPS 0.297 · LSD 19.55 · KL 0.197
input
GT
mainline ep018
PSNR 16.36 · LPIPS 0.519 · LSD 12.90 · KL 2.936
remove-only ep021
PSNR 14.99 · LPIPS 0.507 · LSD 9.20 · KL 1.034
input
GT
mainline ep018
PSNR 22.19 · LPIPS 0.219 · LSD 10.54 · KL 0.208
remove-only ep021
PSNR 23.02 · LPIPS 0.183 · LSD 10.92 · KL 0.258
input
GT
mainline ep018
PSNR 11.37 · LPIPS 0.494 · LSD 11.26 · KL 0.765
remove-only ep021
PSNR 12.50 · LPIPS 0.502 · LSD 12.62 · KL 0.503
input
GT
mainline ep018
PSNR 14.72 · LPIPS 0.771 · LSD 9.00 · KL 3.361
remove-only ep021
PSNR 14.73 · LPIPS 0.689 · LSD 9.55 · KL 3.058
input
GT
mainline ep018
PSNR 18.98 · LPIPS 0.670 · LSD 13.00 · KL 0.289
remove-only ep021
PSNR 18.82 · LPIPS 0.659 · LSD 11.68 · KL 0.048
input
GT
mainline ep018
PSNR 22.75 · LPIPS 0.174 · LSD 14.75 · KL 2.302
remove-only ep021
PSNR 22.43 · LPIPS 0.165 · LSD 14.82 · KL 2.165
input
GT
mainline ep018
PSNR 23.36 · LPIPS 0.108 · LSD 23.01 · KL 2.367
remove-only ep021
PSNR 22.04 · LPIPS 0.129 · LSD 28.97 · KL 3.551
input
GT
mainline ep018
PSNR 18.57 · LPIPS 0.327 · LSD 10.27 · KL 1.109
remove-only ep021
PSNR 18.71 · LPIPS 0.329 · LSD 8.41 · KL 1.155
input
GT
mainline ep018
PSNR 13.97 · LPIPS 0.547 · LSD 9.92 · KL 0.371
remove-only ep021
PSNR 17.64 · LPIPS 0.313 · LSD 10.48 · KL 0.244
input
GT
mainline ep018
PSNR 15.87 · LPIPS 0.338 · LSD 10.69 · KL 0.189
remove-only ep021
PSNR 15.02 · LPIPS 0.366 · LSD 11.46 · KL 0.197
input
GT
mainline ep018
PSNR 23.26 · LPIPS 0.272 · LSD 9.91 · KL 0.160
remove-only ep021
PSNR 21.46 · LPIPS 0.374 · LSD 9.86 · KL 0.149
input
GT
mainline ep018
PSNR 23.83 · LPIPS 0.073 · LSD 8.55 · KL 0.186
remove-only ep021
PSNR 22.44 · LPIPS 0.087 · LSD 8.62 · KL 0.722
input
GT
mainline ep018
PSNR 27.39 · LPIPS 0.143 · LSD 8.75 · KL 1.788
remove-only ep021
PSNR 27.21 · LPIPS 0.143 · LSD 8.09 · KL 0.669
input
GT
mainline ep018
PSNR 22.16 · LPIPS 0.257 · LSD 8.85 · KL 3.482
remove-only ep021
PSNR 20.30 · LPIPS 0.289 · LSD 8.80 · KL 4.068
input
GT
mainline ep018
PSNR 18.40 · LPIPS 0.222 · LSD 20.82 · KL 0.940
remove-only ep021
PSNR 23.17 · LPIPS 0.101 · LSD 20.52 · KL 0.928
input
GT
mainline ep018
PSNR 28.36 · LPIPS 0.122 · LSD 14.53 · KL 0.695
remove-only ep021
PSNR 27.84 · LPIPS 0.132 · LSD 9.97 · KL 2.196
input
GT
mainline ep018
PSNR 15.72 · LPIPS 0.464 · LSD 15.56 · KL 3.299
remove-only ep021
PSNR 14.87 · LPIPS 0.455 · LSD 17.06 · KL 1.560
input
GT
mainline ep018
PSNR 16.24 · LPIPS 0.395 · LSD 17.51 · KL 3.911
remove-only ep021
PSNR 13.84 · LPIPS 0.559 · LSD 17.20 · KL 5.418
input
GT
mainline ep018
PSNR 17.38 · LPIPS 0.326 · LSD 18.07 · KL 0.686
remove-only ep021
PSNR 19.99 · LPIPS 0.256 · LSD 18.15 · KL 1.327
input
GT
mainline ep018
PSNR 18.13 · LPIPS 0.423 · LSD 34.71 · KL 2.485
remove-only ep021
PSNR 18.21 · LPIPS 0.379 · LSD 34.82 · KL 3.725
input
GT
mainline ep018
PSNR 23.17 · LPIPS 0.092 · LSD 13.25 · KL 2.082
remove-only ep021
PSNR 18.53 · LPIPS 0.159 · LSD 16.13 · KL 0.506
input
GT
mainline ep018
PSNR 16.09 · LPIPS 0.602 · LSD 8.27 · KL 0.310
remove-only ep021
PSNR 14.19 · LPIPS 0.617 · LSD 8.32 · KL 0.633
input
GT
mainline ep018
PSNR 16.14 · LPIPS 0.322 · LSD 11.44 · KL 0.126
remove-only ep021
PSNR 12.42 · LPIPS 0.372 · LSD 8.49 · KL 0.063
input
GT
mainline ep018
PSNR 22.68 · LPIPS 0.180 · LSD 10.32 · KL 0.380
remove-only ep021
PSNR 22.52 · LPIPS 0.187 · LSD 11.20 · KL 0.572
input
GT
mainline ep018
PSNR 12.04 · LPIPS 0.467 · LSD 8.67 · KL 0.043
remove-only ep021
PSNR 13.03 · LPIPS 0.353 · LSD 8.48 · KL 0.019
input
GT
mainline ep018
PSNR 24.92 · LPIPS 0.069 · LSD 34.32 · KL 3.091
remove-only ep021
PSNR 13.99 · LPIPS 0.429 · LSD 34.68 · KL 3.281
input
GT
mainline ep018
PSNR 18.87 · LPIPS 0.296 · LSD 8.44 · KL 0.253
remove-only ep021
PSNR 21.49 · LPIPS 0.234 · LSD 8.81 · KL 0.619
input
GT
mainline ep018
PSNR 21.58 · LPIPS 0.195 · LSD 10.05 · KL 0.917
remove-only ep021
PSNR 19.24 · LPIPS 0.248 · LSD 10.94 · KL 0.278
input
GT
mainline ep018
PSNR 19.53 · LPIPS 0.320 · LSD 22.36 · KL 2.850
remove-only ep021
PSNR 19.33 · LPIPS 0.311 · LSD 21.37 · KL 3.446
input
GT
mainline ep018
PSNR 11.92 · LPIPS 0.552 · LSD 17.63 · KL 3.017
remove-only ep021
PSNR 18.34 · LPIPS 0.333 · LSD 20.43 · KL 3.801
input
GT
mainline ep018
PSNR 24.03 · LPIPS 0.178 · LSD 8.94 · KL 0.272
remove-only ep021
PSNR 23.67 · LPIPS 0.173 · LSD 9.03 · KL 0.167
input
GT
mainline ep018
PSNR 18.64 · LPIPS 0.214 · LSD 11.54 · KL 0.522
remove-only ep021
PSNR 18.46 · LPIPS 0.204 · LSD 13.23 · KL 1.230
input
GT
mainline ep018
PSNR 17.94 · LPIPS 0.568 · LSD 8.33 · KL 0.255
remove-only ep021
PSNR 17.90 · LPIPS 0.570 · LSD 8.95 · KL 0.210
input
GT
mainline ep018
PSNR 12.89 · LPIPS 0.278 · LSD 8.58 · KL 2.315
remove-only ep021
PSNR 11.22 · LPIPS 0.302 · LSD 9.02 · KL 2.379
input
GT
mainline ep018
PSNR 30.15 · LPIPS 0.080 · LSD 10.99 · KL 2.381
remove-only ep021
PSNR 28.00 · LPIPS 0.090 · LSD 9.91 · KL 0.822
input
GT
mainline ep018
PSNR 12.89 · LPIPS 0.425 · LSD 10.56 · KL 0.986
remove-only ep021
PSNR 11.79 · LPIPS 0.474 · LSD 10.29 · KL 0.788
input
GT
mainline ep018
PSNR 22.32 · LPIPS 0.357 · LSD 17.77 · KL 2.363
remove-only ep021
PSNR 17.79 · LPIPS 0.455 · LSD 18.84 · KL 1.959
input
GT
mainline ep018
PSNR 24.50 · LPIPS 0.106 · LSD 11.39 · KL 0.774
remove-only ep021
PSNR 20.11 · LPIPS 0.126 · LSD 11.51 · KL 0.940
input
GT
mainline ep018
PSNR 23.69 · LPIPS 0.109 · LSD 12.82 · KL 1.340
remove-only ep021
PSNR 23.72 · LPIPS 0.113 · LSD 14.10 · KL 1.465