We've upgraded! Visit our new site at www.undermind.ai. The new deep search tool covers all of science, not just ArXiv, and is 10x faster. Your previous data is still available at app.undermind.ai.

Search Topic:

Strategies for weighting different noise levels when training diffusion models. In particular, any work proposing strategies for determining either (1) how often to sample each noise level during training, or (2) how heavily to weight the loss associated with each noise level when computing the overall loss.

Additional Context Provided:

To train a diffusion-based generative image model, it is necessary to train on images with noise added across a range of different noise magnitudes, often known as “noise levels.” Although we must sample each noise level with some nonzero frequency during training, when designing the training process we get to choose the <em>proportion</em> of training examples that use one noise level versus another, and we also get to choose how heavily the model’s prediction error at each noise level is <em>weighted</em> in the model’s overall loss function. These decisions about how to weight different noise levels can affect the compute-efficiency of training, as well as the quality of the samples generated by the final trained model. I am interested in this topic because I would like to train a new diffusion model as efficiently as possible.