Stable Affine Alignment

It is a well known but unpublished fact of image registration that if you want to compute the wibbly wobbly vector field that align two images, it's as easy as training a U-Net to output that field directly. However, if you want to predict a rigid or affine transform between two images, the equivalant method "Just predict the matrix" is horribly unstable.