Background: Governments have implemented population-wide physical distancing measures to control COVID-19, but metrics evaluating their effectiveness are not readily available. Methods: We used a publicly available mobility index from a popular transit application to evaluate the effect of physical distancing on infection growth rates and reproductive numbers in 40 jurisdictions between March 23 and April 12, 2020. Findings: A 10% decrease in mobility was associated with a 14.6% decrease (exp(β) = 0.854; 95% credible interval: 0.835, 0.873) in the average daily growth rate and a -0.061 (95% CI: -0.071, -0.052) change in the instantaneous reproductive number two weeks later. Interpretation: Our analysis demonstrates that decreases in urban mobility were predictive of declines in epidemic growth. Mobility metrics offer an appealing method to calibrate population-level physical distancing policy and implementation, especially as jurisdictions relax restrictions and consider alternative physical distancing strategies. Funding: No external funding was received for this study.