Our paper "Rich Information is Affordable: A Systematic Performance Analysis of Second-order Optimization Using K-FAC" was accepted at
#KDD2020
. We conducted a step-by-step performance analysis of distributed K-FAC and show the overhead can be reduced to the same amount as SGD.