The COVID-19 pandemic is causing mass disruption to our daily lives. We integrate mobility data from mobile devices and area-level data to study the walking patterns of 1.62 million anonymous users in 10 metropolitan areas in the United States. The data covers the period from mid-February 2020 (pre-lockdown) to late June 2020 (easing of lockdown restrictions). We detect when users were walking, distance walked and time of the walk, and classify each walk as recreational or utilitarian. Our results reveal dramatic declines in walking, particularly utilitarian walking, while recreational walking has recovered and even surpassed pre-pandemic levels. Our findings also demonstrate important social patterns, widening existing inequalities in walking behavior. COVID-19 response measures have a larger impact on walking behavior for those from low-income areas and high use of public transportation. Provision of equal opportunities to support walking is key to opening up our society and economy.