resnet50
- Europe > Switzerland > Zürich > Zürich (0.14)
- North America > United States > Pennsylvania > Philadelphia County > Philadelphia (0.04)
- Law (0.46)
- Health & Medicine > Diagnostic Medicine (0.46)
- Europe > Spain > Basque Country > Biscay Province > Bilbao (0.04)
- Asia > China (0.04)
- Europe > Netherlands > North Holland > Amsterdam (0.04)
- Europe > Finland > Uusimaa > Helsinki (0.04)
- Information Technology > Artificial Intelligence > Machine Learning > Unsupervised or Indirectly Supervised Learning (0.56)
- Information Technology > Artificial Intelligence > Machine Learning > Inductive Learning (0.53)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.30)
Supplemental Material for AC-GC: Lossy Activation Compression with Guaranteed Convergence
The appendices of this supplemental material are focused on providing detailed proofs (Appendix A), per-layer derivations for activation errors (Appendix B), algorithm and implementationdetails(AppendixC),datasetsandhyperparameters(AppendixD),extended experimental data (Appendix E) and additional experiments (Appendix F) to accompany the main paper. A code example and trained models are available for CIFAR10/ResNet50 by accessing https://github.com/rdevans0/acgc. L and η depend on the model being trained and dataset, and are thus problem-dependent constants. Preliminary on Separation of Norms Given two, independent random vectorsA= (an) RN and B =(bn) RN, whereE[bn]=0 n. Given f which obeys (4), and a convex functionD( X) which bounds the gradient error from above for all X, θ, and X; provided that D( X) e2V2 the variance of the compressed gradients satisfies E[kˆ θf(θ,Xnt)k2] (1+e2)V2 (16) Proof.
- North America > Canada > Ontario > Toronto (0.14)
- North America > United States (0.05)
- North America > Canada > British Columbia (0.04)
- Research Report > Experimental Study (0.47)
- Research Report > New Finding (0.46)
- North America > United States > California > Santa Barbara County > Santa Barbara (0.14)
- North America > United States > California > San Francisco County > San Francisco (0.14)
- South America > Chile > Santiago Metropolitan Region > Santiago Province > Santiago (0.04)
- (2 more...)