Supermassive black holes (SMBHs) are commonly found at the centers of most massive galaxies. Measuring SMBH mass is crucial for understanding the origin and evolution of SMBHs. Traditional approaches, on the other hand, necessitate the collection of spectroscopic data, which is costly. We present an algorithm that weighs SMBHs using quasar light time series information, including colors, multiband magnitudes, and the variability of the light curves, circumventing the need for expensive spectra. We train, validate, and test neural networks that directly learn from the Sloan Digital Sky Survey (SDSS) Stripe 82 light curves for a sample of 38, 939 spectroscopically confirmed quasars to map out the nonlinear encoding between SMBH mass and multi-band optical light curves. We find a 1 scatter of 0.37 dex between the predicted SMBH mass and the fiducial virial mass estimate based on SDSS singleepoch spectra, which is comparable to the systematic uncertainty in the virial mass estimate. Our results have direct implications for more efficient applications with future observations from the Vera C. Rubin Observatory. Our code, AGNet, is publicly available at https: //github.com/snehjp2/AGNet.