Skip to main content

CaloDiT on Dataset-3

Details about training dynamics, nevertheless the model can scale to a higher granularity.
Last updated: February 19, 2025


Training stability:

Training loss usually corresponds to how good the generated images are. Change in training loss implies a change in the quality of showers. But if the loss hovers around the same delta, that doesn't mean that all models perform similarly.

Training seems a bit unstable for wider networks. 450 wider < 900 wider < 450 even wider, this is the unstability. This is vaguely proportional to the ratio of embedding size to patch size. Hence, their plots are good at certain epochs. Looks like the gradients are unstable. Try clipping them.

Edit: Gradients were already being clipped to 1.0. Making LR smaller helped.

6 selected runs