How Do You Digitize Hundreds of Thousands of Records? Crowd-Sourcing

On March 21, 2018, the African-American Civil War Soldiers Project was launched.  Their goal was to transcribe documents from the National Archives for over 200,000 African-American soldiers who served in the Civil War in order to “restore these freedom fighters to their proper place in history.”

First Tweet

The project seems simple enough, it is a crowd-sourcing database transcription project between historians, social scientists, and the African-American Civil War Museum.  They hope to increase the amount of digitized data that can be easily accessed and searched in order for this information to be utilized by teachers, students, and the general public to increase their awareness of African-Americans in the Civil War.  Another interesting goal is they want this to also be a resource for people to look up if they have descendants who fought in the war.  They are transcribing images of the soldiers’ military service records, which have been photographed and scanned by the National Archives and Records Administration, which holds millions of documents, and Fold 3, which is a part of, that is used to “discover your family’s military past.”  These service records indicate that there are an estimated 200,000 soldiers who formed the United States Colored Troops (USCT).  When the database is complete (as of this writing, it was 38% complete), it will be presented on the African-American Civil War Museum website.  The African-American Civil War Soldiers project is supported by the African-American Civil War Museum (Washington D.C.), the National Archives and Records Administration (Washington D.C.), Fold 3 (Salt Lake City, UT) and the Willson Center Digital Humanities Lab at the University of Georgia (Atlanta, GA).  They do not disclose how much funding they are provided with or if they are receiving any other grants.


The project is rather ambitious.  The project’s team includes only five individuals, John Clegg, a historical sociologist; Danny Colligan, a Sociology PhD student; Aaliyah Muhammad, an undergraduate at the University of Chicago majoring in anthropology; Scott Nesbit, an assistant professor of digital humanities; and Kate Thomas, an independent scholar researching Frederick Douglass who has a PhD in music history.  Their backgrounds vary, which that brings an interesting approach to the project.  I am not sure that the project is innovative, because similar projects like it have been undertaken before, such as the Freedman’s Bureau Project at the National Museum of African-American history and Culture in DC, which involved thousands of volunteers from across the country as well as visitors to the museum who could go in a dedicated room to transcribe information from primary sources.  So this idea is not new, but the goal of the project is new and I think the digitized documents will be useful to educators, students, and those genuinely curious about this history.

At the bottom of their homepage, they have a status bar that shows the completion percentage and number of volunteers.  At the time of this writing, they have had 2,841 volunteers help contribute to the project, which is really impressive.  The website also has a way for the contributors to talk to each other on a forum page.  There are a few hundred people involved in these discussion threads.  This is a great tool to have so people who are interested in the project can connect with each other and also ask for help if they are having an issue.  An even greater tool is their Twitter page.  There, they have almost 1,000 followers and nearly 300 tweets.  They update it rather frequently. Here, they share interesting finds from the transcription process.  In each of these Tweets, they include the volunteer who transcribed the document, such as this one with a soldier named Betsey (who they assume to be a man since women were not allowed to enlist).  I think the Twitter page is a great tool to use to give people involved with and those following the project to interact with the process and with other individuals helping out.

Betsey transcription tweet.jpg

The project itself is hosted on a platform called Zooniverse.  All of the projects on this site use the same framework as the African-American Civil War Soldiers Project in that it allows hundreds of thousands of people from around the world to contribute to projects and assist professional researchers who otherwise would never be able to have a workforce that large.  It allows projects that before seemed impossible to tackle now within the realm of possibility.  I think this a great tool since funding, especially on a large scale, is hard to come by and has become more politicized over time.  This platform is a democratizing force that allows any researcher to tap into a massive pool of ready volunteers.  This is also good for the African-American Civil War Soldiers Project because one does not need to be an expert in order to participate. All they have to do is read the document and type it into their form.  They have even provided a tutorial explaining how to transcribe the documents.  The project team did not state when they expect or hope to complete the transcribing, but after only seven months, being 38% percent of the way to digitizing the records of 200,000 soldiers is pretty impressive and I hope many others join this project to help them achieve their goal.

Once the project is complete, it will live on the African-American Civil War Museum website, it will be easily searchable, and it will sit alongside educational resources for teachers and students.  I can definitely see this project being used in middle schools, high schools, and colleges to not only to teach students about digitally transcribing documents but also for learning about African-Americans who fought in the Civil War.  I am excited to see the maps, charts, and other visual resources that will come with the completion of this project.  Below is one of the first charts they have published.

aa38febd-8249-487c-bfe2-4eb62455218c.jpegAfrican-American soldiers and sailors by date of enlistment

Overall, I really like the project.  I think it has an ambitious goal, undertaking such a large number of documents, but they have found and utilized Zooniverse, which I think is going to allow them to succeed with the help of thousands of people who would otherwise never have the opportunity to be involved with a such a project.

748be9a7-768e-416f-9924-2173c60e335d.jpegOriginal recruitment poster for African-American Soldiers

This entry was posted in Digital Project Reviews. Bookmark the permalink.

2 Responses to How Do You Digitize Hundreds of Thousands of Records? Crowd-Sourcing

  1. jenansbach says:

    I had a chance to visit a Civil War African American cemetery in Mississippi, and what was most fascinating was that the scholar who came with us said that there are few records left for that particular cemetery and that they had used part of it as a cemetery for enslaved people (I think it was a place the enslaved people could bury their own, rather than being a place where a slave holder buried his enslaved people). They were not even certain where the enslaved had been buried on the land, and it was poorly kept. That those who visit that cemetery could potentially use a site like this to find out about the people buried there, who seem to have been forgotten, is an amazing leap in technology. Thank you for pointing out that they have a Twitter–that’s always my favorite social media.

    Liked by 1 person

  2. larrydurst says:

    I agree that while this project might not be wholly novel, it really does a great job of “recovery,” to use the term we first got from Kim Gallon, but that has become a working theme for the class. I found it to be a good model for what a digital archive can be. The site is both simple and ambitious, which might be the key to its usability and to its potential for other researchers. I can see this site used for both serious research and by casual, curious browsers.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s