Thank you so much for sharing the source code!
While running the code, I noticed a possible issue in the implementation of the Glow model. In model_tools.py, within the InvBlock class, it seems that the Jacobian calculation might be incorrect. Specifically, in the forward and reverse functions of class InvBlock, where the multi-scale architecture is applied, it appears the Jacobian term was unintentionally omitted after splitting the input and feeding it into the prior network. To address this, after line 987 and line 1027, consider updating the Jacobian calculation as follows: logdet = logdet + log_sd.view(b_size, -1).sum(1).
Additionally, I found that modifying the sigmoid function to tanh in the AffineCoupling class can improve the model's performance. Adding gradient clipping helps stabilize training. I've shared my own implementation of the Glow model here for reference: https://github.com/a-little-hoof/Glow.
I hope these suggestions are helpful for your research. Thanks again, and have a great day!
Thank you so much for sharing the source code!
While running the code, I noticed a possible issue in the implementation of the Glow model. In model_tools.py, within the InvBlock class, it seems that the Jacobian calculation might be incorrect. Specifically, in the forward and reverse functions of class InvBlock, where the multi-scale architecture is applied, it appears the Jacobian term was unintentionally omitted after splitting the input and feeding it into the prior network. To address this, after line 987 and line 1027, consider updating the Jacobian calculation as follows: logdet = logdet + log_sd.view(b_size, -1).sum(1).
Additionally, I found that modifying the sigmoid function to tanh in the AffineCoupling class can improve the model's performance. Adding gradient clipping helps stabilize training. I've shared my own implementation of the Glow model here for reference: https://github.com/a-little-hoof/Glow.
I hope these suggestions are helpful for your research. Thanks again, and have a great day!