Monoclonal antibodies are a focus in vaccine and therapeutic design to counteract SARS-CoV-2. Here, we combined SARS-CoV-2-specific B cell sorting with single-cell VDJ and RNA-seq and antibody structures to characterize B cell responses against SARS-CoV-2. We show that the SARS-CoV-2-specific B cell repertoire consists of transcriptionally distinct B cell populations with potently neutralizing antibodies localized in two clusters that resemble memory and activated B cells. Of 92 isolated antibodies, 27 neutralize SARS-CoV-2 with IC80 values as low as 7 ng/ml. Cryo-electron microscopy structures of potently neutralizing Fabs complexed with SARS-CoV-2 spike trimers show recognition of various receptor-binding domain (RBD) epitopes, including one antibody (BG10-19) that locks the spike trimer in a closed conformation to potently neutralize SARS-CoV-2, including lineages B.1.1.7 and B.1.351, SARS-CoV and cross-reacts with heterologous RBDs. Together, our results characterize transcriptional differences among SARS-CoV-2-specific B cells and will inform design of immunogens to elicit cross-reactive protection.