Plan a pilot box
Menu

What digitizing a historical collection actually involves

Reader question: We want to digitize our collection. What happens from first assessment to files online and preserved, what do we have to decide, and what does each step ask of our staff?

By Dastok Editorial TeamIssue: Published

Digitizing a historical collection is a chain of about ten decisions, not a scanning job: why you are doing it and for whom, what you hold and what state it is in, what you select and may legally share, a pilot, the capture specification, text recognition, description, quality control, publishing, and preservation of both the files and the originals. Scanning is one link in that chain. Most projects that disappoint do so because a decision upstream of the scanner was skipped, such as an unread donor agreement, a capture standard nobody wrote down, or files with no plan for where they will live in ten years.

This guide walks through those decisions in the order you will meet them. For each one it says what you decide, who on your side should own it, which published standard applies, what commonly goes wrong and which Reading Room article goes deeper. It is written from the institution’s side, whoever does the hands-on work. If you want to see how one firm runs the work, our own method is on how we work; this article is about your decisions, and they are the same whether you scan in-house, hire a vendor or both.

The decisions, in the order you make them

The Federal Agencies Digitization Guidelines Initiative (FADGI) describes digitization as “a complete process” that “covers all project components from content selection through delivery of digitized objects into a repository environment,” in its project planning and management outline, approved in November 2009 and still current. That definition is the right frame. The capture itself is the middle of the work, not the whole of it.

#DecisionUsually owned byStandard or tool that appliesGo deeper
1Purpose and audienceDirector and boardYour mission and collections policyThis article
2Inventory and conditionCollections managerCondition survey, box-level inventoryCollection assessment
3Selection and rightsCurator or archivistNEDCC selection questions, copyright term chartRights review worksheet
4PilotProject leadYour own measured resultsRunning a digitization pilot
5Capture specificationProject lead with the imaging teamFADGI Technical Guidelines, 3rd ed. (2023)FADGI star ratings explained
6Text recognitionArchivist or librarianCharacter error rate on a reference sample; ALTO XMLAI handwriting recognition accuracy
7Description and AI suggestionsCatalogerDACS, Dublin Core, MARC 21, EAD; labeling machine fieldsLabeling AI-generated metadata
8Quality controlProject leadRandom sampling, image QC checklistImage QC checklist
9Publishing and accessibilityDirector with IT or web hostIIIF, WCAG 2.1 AA, ADA Title II ruleIIIF for small institutions
10Preservation and the originalsDirector and records officerNDSA Levels, BagIt (RFC 8493), records schedulesNDSA Levels for small archives
The ten decisions in a digitization project, the usual owner on the institution's side, and the standard that applies. Compiled by the Dastok Editorial Team from the FADGI planning outline and Technical Guidelines, NEDCC Leaflet 6.6, the NDSA Levels and the ADA Title II rule, checked November 2026.

Three things follow from the table. First, most owners are on your staff even when a vendor does the capture, because the decisions are about your collection, your audience and your legal position. Second, the order matters: rights come before capture because there is no point imaging at high cost what you cannot publish, and preservation planning comes before delivery because the files need somewhere to land. Third, several steps can overlap once the pilot is done, so the order is a sequence of decisions, not a strict calendar.

Purpose and audience: decide why before what

The first decision is the reason for the project, written in a sentence or two that a board member could repeat. “Make the Okerlund letters searchable for family historians” leads to different choices than “reduce handling of the fragile day books” or “publish everything for the county’s anniversary.” Purpose drives every later trade-off: how much to select, how high a capture level to pay for, whether handwriting needs full transcription, and whether files go online at all.

Write down three things at this stage:

  1. Who will use the results. Researchers, genealogists, students, staff answering reference questions, or the public browsing on phones. Each group searches differently and needs different description.
  2. What they will look for. Names, places, dates, events or subjects. This decides which fields matter most in the catalog and whether full-text search of handwriting is worth the cost.
  3. What success looks like in a year. A number of items online, a reduction in requests to handle originals, or a finding aid linked to images. A measurable target makes the pilot easier to judge.

What commonly goes wrong is a purpose so broad that nothing can be ruled out (“digitize the archive”). That turns selection into an argument and the budget into a guess. A narrow first phase with a clear audience is easier to fund, finish and evaluate, and it leaves you with a working pipeline for the next phase.

Phasing a large collection

Most collections are too large to digitize in one project, and they do not need to be. Split the work into phases that each deliver something usable on their own: one series, one format or one theme tied to an anniversary or a research need. Each phase should end with images online or in your system, a preservation copy stored, and a short note of what you learned. The second phase then starts from a tested specification and a known cost per item, which makes it far easier to fund. A phase plan also protects you if funding stops partway: what is finished stays finished.

Ownership sits with the director and, for larger projects, the board, because purpose sets the budget and the institution’s public commitments. Staff time at this stage is a few meetings, but it saves weeks later.

Inventory and condition: know what you hold

You cannot plan what you have not counted. An inventory for digitization does not need item-level detail; box and folder level is usually enough at first. For each box or series, record the format (letters, bound volumes, prints, negatives, glass plates, maps), the date range, a rough item or page count, and condition. A collection inventory worksheet gives a usable column set, and a condition assessment checklist lists what to look for.

Condition matters because it sets the speed and method of capture and, sometimes, whether capture should happen before conservation. Examples of findings that change a plan:

  • Brittle paper or iron gall ink corrosion. Pages may crack when turned, so handling time per page rises and some items need a conservator first.
  • Tight bindings. Bound volumes that open only part of the way need cradles and slower capture; they should never be forced flat.
  • Film decay. Acetate negatives that smell of vinegar or curl, and any cellulose nitrate film, need separate handling and storage decisions.
  • Mold or pests. These stop the project for that material until it is treated, and they are a health and safety issue for staff.

The output of this step is a list with counts. Those counts feed the budget and the schedule, so measure rather than guess where you can: count pages in a sample of boxes and multiply. The digitization budget calculator and the project timeline calculator turn counts and measured times into hours and weeks.

A worked count

Counting does not mean opening every folder. Take a random sample of boxes from each series, count the pages or items in those boxes, and scale up by the number of boxes. Count images, not items, because capture time follows images: a letter with an envelope may be three images, a cabinet card with writing on the back is two, and a bound volume is every page plus covers. Note anything that changes speed, such as oversize sheets, tightly bound volumes or items in sleeves that must be removed.

Example (fictional). The Aster County Historical Society holds 60 boxes of family papers. A sample of 6 boxes averages 160 items and 3.1 images per item, so the society estimates about 9,600 items and roughly 30,000 images for the series, with one shelf of bound ledgers counted separately because each page is its own image and capture is slower.

The number is an estimate, and you should say so in any budget. The pilot replaces it with measured rates.

Ownership sits with the collections manager or archivist. Staff time is real here: expect days to weeks for a mid-sized collection, which is why many institutions start with a formal collection assessment as a separate, smaller piece of work.

Selection and rights: should, may and can

Selection decides what goes first, and what never goes. The Northeast Document Conservation Center puts the problem plainly in its leaflet on preservation and selection for digitization: “No institution can afford to preserve or to digitize everything it owns.” The leaflet frames selection as three questions, whether materials “should be digitized, whether they may be digitized, and whether they can be digitized.”

  • Should? Is the material important to your purpose, in demand, and better served by a digital copy? Material that is heavily used, fragile, or impossible to consult otherwise usually rises to the top.
  • May? Do you have the right to copy and share it? This is the rights question, and it often decides what goes online even when capture is allowed.
  • Can? Can it be captured safely and well with the methods and budget available? Oversize maps, glass plates and fragile bindings may need specialist handling.

Rights in practice

For US material, the published or unpublished status of an item usually decides its copyright term. Cornell University Library’s copyright term chart, updated January 1, 2026, shows that works first published in the United States before 1931 are in the public domain due to copyright expiration. Most items in a historical-society box are unpublished: letters, diaries, ledgers and many photographs. For those the chart gives a term of the author’s life plus 70 years, and 120 years from creation when the author’s death date is unknown, so the published-before-1931 rule often does not decide the question.

Rights review also covers what copyright does not: donor agreements, deed restrictions, privacy of living people, and culturally sensitive material that a community, not the holding institution, should decide about. Record a working assessment per item or series, using the standardized statements from RightsStatements.org so that later systems can read them. The rights review worksheet has the columns and all twelve statements. None of this is legal advice; when a decision carries real risk, ask counsel.

Sensitive and community materials

Some material raises questions that copyright law does not answer. Records about living people, medical or court material, and cultural heritage that belongs to a community all need a decision about who may see what, and sometimes about whether a machine should process them at all. For Indigenous materials, the Protocols for Native American Archival Materials, endorsed by the Society of American Archivists in 2018, and tools such as Local Contexts labels exist so that communities set the terms. Our article on digitizing Native American materials explains how those frameworks work, and our page on community archives describes how we approach shared control. The practical step for any project is to flag such material during selection and settle access rules before capture, not after publication.

What commonly goes wrong is leaving rights to the end, then discovering that the most interesting material cannot go online. Ownership sits with the curator or archivist, with the director deciding on risk.

The pilot: test the whole chain before you commit

A pilot runs a small, representative batch of your real material through every step: capture, text recognition, description, quality control and delivery into the system where it will live. It is the single best way to replace estimates with measurements. Choose items that include your hardest cases, such as the faded ledger, the worst handwriting and the oversize map, not only the clean ones.

Measure at least these in the pilot:

  1. Capture rate. Images per hour for each kind of original, including handling time.
  2. Defect rate. The share of images that fail quality control and need recapture.
  3. Recognition accuracy. Character error rate for print and for each hand, on a reference transcription.
  4. Description time. Minutes per item to catalog, and how many machine suggestions survive review.
  5. Fit with your system. Whether the records and files import cleanly, with fields in the right places.

Example (fictional). The Aster County Historical Society pilots one box: 41 letters, 18 cabinet cards and a 212-page store ledger from Wren Gulch, 1893 to 1896. The pilot shows the ledger captures at about half the rate of the loose letters because the binding opens only partway, that recognition on Ida Okerlund’s hand needs targeted correction of names and places, and that subject headings suggested by a model need more review than names and dates. The society changes its full plan in three ways before committing: it budgets the ledgers separately, it chooses targeted rather than full correction for the letters, and it asks catalogers to review every subject heading. The whole fictional box is followed step by step on our example collection page.

How big should a pilot be? Big enough to include every kind of original and every hand you care about, and small enough to finish and review in a few weeks. For many collections that is one or two boxes, or a few hundred images, chosen deliberately rather than taken from the top of the shelf. The point is not volume but coverage: a pilot that skips the ledgers tells you nothing about the ledgers.

A pilot also tests the people side: who answers questions, who approves the capture specification, how long reviews take. Ownership sits with a named project lead. Our article on running a digitization pilot covers sample design and what to write down, and the pilot is the first thing to ask any vendor for, as our guide to writing a digitization RFP explains.

Capture: write the specification down

The capture specification is a short written document that says, for each series, what image quality you are paying for. In the United States the reference is FADGI’s Technical Guidelines for Digitizing Cultural Heritage Materials, third edition, 2023. It sets four star levels, from one star, “appropriate for applications where the intent is to provide a reference to locate the original,” to four stars, “the best imaging practical today.” FADGI “does not recommend digitization to less than a three-star level when possible,” and cautions against four-star imaging for all but the most sophisticated programs.

A specification should state, per series:

  • The FADGI category and target star level. For example, unbound manuscripts and other rare documents call for a minimum of 300 ppi at three stars and 400 ppi at four stars.
  • Resolution, bit depth and color. Resolution is set relative to the size of the original, which is why the scanning resolution calculator asks for the original’s dimensions.
  • File formats. Preservation masters (FADGI lists TIFF and JPEG 2000 among master formats) and smaller access copies made from them.
  • What gets captured. Fronts and backs, blank pages, envelopes, mounts and covers.
  • How quality is checked. Targets, sampling and tolerances.

A star rating is earned by measuring images against FADGI’s evaluation parameters through image analysis, not by resolution alone. So the honest way to describe a project is “captured to the level agreed in writing and checked against targets,” not a blanket claim. Our article on FADGI star ratings explains how to choose a level for each kind of material.

Handling and where the work happens

The specification should also say how originals are handled: whether fasteners may be removed and replaced, whether folded items may be relaxed, how far bound volumes may open, and what happens if an item starts to tear or flake. A good rule is that capture stops and a conservator is consulted whenever an original may be harmed. Decide too where the work happens. Some collections can travel to a studio with a signed receipt at each handover; others, such as public records that may not leave custody or items too fragile to move, are better captured on site. Our collection care page lists what to agree in writing either way, and the packing and transport checklist covers the trip itself.

Ownership sits with the project lead, who approves the specification, and the imaging team, who meets it. What commonly goes wrong is an unwritten specification, which leaves you unable to reject poor images later because nobody agreed what good meant.

Text recognition and description: make the images findable

Images become useful when people can find them, and that depends on two things: the text inside the images and the description around them.

Text recognition

Optical character recognition (OCR) reads printed and typed text; handwritten text recognition (HTR) reads manuscripts. Both make mistakes, so the useful question is not “can it read this?” but “how often is it wrong on our pages, and does that matter for our users?” The standard measure is the character error rate: substituted, missed and extra characters divided by the characters in a reference transcription. The Transkribus platform’s guidance on character error rate says a good rate “for printed text is between 0.5 and 2%, while for handwritten text, it ranges between 2 and 8%,” depending on whether a model was trained on one hand or many. Measure your own rate in the pilot; the OCR accuracy calculator turns error counts into rates and correction hours.

Who corrects the text is its own decision. Staff know the names and places; volunteers can cover large volumes if they are trained and their work is checked; a vendor can do it under a written accuracy target. Many projects combine them, with machines producing a first draft and people correcting the parts that matter most for search. Our article on volunteers, AI or both compares those models.

Decide a correction level per batch: full correction for text that will be quoted, targeted correction of names, places and dates for large search-driven collections, or none, clearly labeled, where raw recognition is good enough to search. The text should come with word positions, usually in ALTO XML, the Library of Congress layout standard, so viewers can highlight hits on the page image. Our article on whether AI can read old handwriting goes into what current tools get right and wrong.

Description

Description means catalog records: titles, dates, names, places, subjects, extent and rights. Choose the standard your systems and users already expect. Archives often describe at the collection and series level using DACS (Describing Archives: A Content Standard), maintained by the Society of American Archivists, whose required elements include reference code, repository, title, date, extent, creator if known, scope and content, conditions governing access and languages. Item records for online collections often use Dublin Core’s fifteen elements; libraries use MARC 21; finding aids are encoded in EAD. Whether to describe item by item or at a higher level is a cost decision, discussed in our article on item-level or collection-level description.

Machine suggestions in catalog records

Language and vision models can now suggest names, places, dates and subjects from recognized text and images. The Library of Congress tested this in its Exploring Computational Description experiments and concluded that “no current AI tools are currently good enough to run fully automatically for this task,” finding identifiers, author and title extracted “at around 85% accuracy, while others like subject headings, genre, and dates were only generated accurately at or under 50% of the time.” The practical response is to treat suggestions as drafts, have a cataloger review them, and keep machine-made fields labeled until they are reviewed, so researchers know what a person has checked. Our guide to labeling AI-generated metadata shows field-level patterns, and our AI policy describes how we apply this ourselves.

Ownership sits with an archivist, librarian or cataloger. Staff time here is often the largest internal cost of the whole project, which is why the pilot’s description-time measurement matters.

Quality control: check by sampling, record what you find

Quality control confirms that the work matches the specification before you accept it. It covers completeness (every page, in order, nothing duplicated), image quality (focus, framing, glare, color targets within tolerance), files (formats, embedded metadata, checksums) and records (fields present, machine suggestions labeled, identifiers matching file names).

Checking every image by eye is rarely possible, so check a random sample of each batch and decide in advance what defect rate triggers a recheck or recapture. The QC sample size calculator gives a defensible sample size for a batch, and the image QC checklist lists what to look at. Record defects with file names and actions taken, so the next batch improves.

When a batch fails

Decide in advance what happens when a sample shows more defects than you agreed to accept. The usual sequence is to look for a cause (a wrong setting, a dirty glass, a misread specification), fix it, recapture the affected images, and check a fresh sample from the corrected batch. Keep the failed sample and the decision in the project log. If the same defect returns, change the process rather than adding more inspection, because inspection finds problems but does not prevent them.

What commonly goes wrong is accepting everything at the end in a rush. Batch-by-batch acceptance spreads the work and catches systematic problems, such as a mis-set color profile, early. Ownership sits with the project lead, with spot checks by whoever knows the material best.

Publishing and accessibility: decide where the collection lives online

Publishing is a set of choices about platform, viewer and access rules. You can add records and images to the collection system you already use, publish through a state or regional aggregator, run a hosted search site, or do several of these. Before choosing, check that you can export everything, images, text and records, if you leave a platform later.

Viewers and IIIF

The International Image Interoperability Framework (IIIF) is a set of open specifications for delivering and describing images on the web. Its Image API and Presentation API are at version 3.0. IIIF lets any compatible viewer zoom deep into a high-resolution image and lets other institutions show your images without copying them. Whether a small institution needs it depends on its platform and goals; our article on IIIF for small institutions works through the decision.

Accessibility and the ADA Title II rule

If you are a state or local government body, such as a county records office or a public library, the Department of Justice’s ADA Title II rule applies to your web content. The 2024 rule requires web content to meet WCAG 2.1 Level AA. An interim final rule published April 20, 2026 moved compliance to April 26, 2027 for entities serving 50,000 or more people and April 26, 2028 for smaller entities and special districts, as of November 2026. The rule exempts archived web content that meets specific conditions, including that it is kept only for reference, research or recordkeeping and sits in a clearly identified archive area. How those conditions apply to digitized historical collections is covered in our article on ADA Title II and digitized collections; it is not legal advice.

Questions to ask of any platform

  • Can you export every image, every text file and every record in standard formats, without a fee, if you leave?
  • Does it display recognized text and highlight search hits on the page image?
  • Can records show which fields were machine-suggested and which were reviewed?
  • Does it meet WCAG 2.1 AA for its viewer and search pages, and has that been tested with a keyboard and a screen reader?
  • Who pays for hosting after the project, and what happens to the collection if that payment stops?

The last question matters more than it looks. A collection that goes offline when a subscription lapses has to be published again, which is why many institutions keep a complete copy of everything they publish under their own control.

Even where no rule applies, accessible publishing reaches more people: alt text for images, transcripts for handwritten pages, and viewers that work with a keyboard and a screen reader. Ownership sits with the director and whoever runs your website or platform.

Preservation and the originals: plan for the next twenty years

Digital files are fragile in a different way from paper. They fail silently when a drive dies, a format goes out of use, or nobody pays the storage bill. Preservation planning answers three questions: how many copies, where, and how you will know they are intact.

The National Digital Stewardship Alliance’s Levels of Digital Preservation give a practical scale. Version 2.0 (2019) arranges actions into five areas, storage, integrity, control, metadata and content, across four levels. Its first level asks you to “have two complete copies in separate locations”; the second, to “have three complete copies with at least one copy in a separate geographic location.” Version 2.1, released in March 2026, adds a focus on environmental sustainability. Integrity means checksums: a fingerprint of each file, created when files arrive and checked on a schedule. BagIt, published as RFC 8493 in October 2018, is a simple packaging format that keeps files and their checksum manifests together. The preservation readiness checklist turns these into tasks, and our article on NDSA Levels for small archives shows a realistic plan.

Formats, names and documentation

Preservation is easier when the files explain themselves. Keep masters in a format chosen for longevity, keep access copies separate, and give every file a name that ties it to its box, folder and item and to its catalog record. The file naming builder produces fixed-width, lowercase names that sort correctly and survive moves between systems. Store a short project document with the files: the capture specification, the description standard, the correction level used for text, and the date of each batch. Twenty years from now, that document is how someone else will know what the files are and how they were made.

The originals

Digitization rarely replaces the originals, and it is not a substitute for caring for them. For public records, whether a scanned original may ever be destroyed is governed by law and records schedules: in Colorado, the State Archives prepares retention and disposition schedules for state agencies, counties and municipalities, which advise which records must be kept permanently. Never assume a digital copy permits disposal; our article on destroying originals after scanning sets out what to check. For everyone else, plan the return: originals back in their original order, in good enclosures, with a signed receipt.

Ownership sits with the director and, for public bodies, the records officer. Storage is a continuing cost, so budget for it beyond the project end date.

What the project asks of your staff, and where to start

Even when a vendor does capture and much of the description, a digitization project draws on staff time at every step. The table below shows typical roles and the work each does; actual hours depend on your collection and should come from the pilot.

Role Main work in the project When
Director or board Purpose, budget, risk decisions on rights and publishing, long-term storage commitment Start, after the pilot, at launch
Project lead Plan, pilot, capture specification, schedule, batch acceptance Throughout
Collections manager or archivist Inventory, condition, selection, preparing and returning originals Before and during capture
Cataloger or librarian Description standard, review of machine suggestions, record checks During and after capture
Records officer (public bodies) Retention rules, public access and privacy obligations Start and end
IT or web host Storage, platform, imports, accessibility of the viewer Planning and delivery

Budget lines to expect

Labor is usually the largest cost, whether it is your staff’s time or a vendor’s. Beyond capture, plan for preparation, quality control, text correction and cataloging, which together can exceed capture time; the digitization budget calculator separates these stages so you can see which one drives the total. Add storage for every copy for several years, conservation treatment for items the condition survey flagged, archival enclosures for rehousing, transport and insurance if originals travel, and platform or hosting fees if you publish online. If you apply for a grant, the grant budget worksheet lays these out as budget lines with cost share.

Small institutions often have one or two people covering all of these. That is workable if the first phase is small and the decisions are written down, because each phase then reuses the specification, the description template and the storage plan from the last one.

Funding shapes the pace for many institutions. As of November 2026, NEH’s Collections Stewardship program lists eligible activities that include “digitizing or reformatting collections and making them available for public use,” with awards of up to $350,000 for one organization and a December 15, 2026 deadline. Program details change, so check each funder’s current notice; our article on grants for digitization surveys federal and Colorado options with their verified status.

If you are deciding who should do the hands-on work, our article on in-house or outsourced digitization compares the two honestly; Dastok sells digitization, so we say so there and give both sides.

A practical starting sequence:

  1. Write the purpose and audience in two sentences and get them agreed.
  2. Inventory one series at box and folder level, with counts and condition.
  3. Run a rights check on that series and mark what may go online.
  4. Pilot a small, representative batch through every step, including your hardest items.
  5. Turn the pilot’s measurements into a budget and schedule with the digitization budget calculator, then write the capture specification and the preservation plan before full capture starts.

If you would like to see the whole chain on real material before deciding anything, a pilot box is how we start; the options are described on engagements.

Working through this with a real collection?

If this article describes a decision you are facing, tell us about the collection. A pilot box answers most questions with your own material.

Plan a pilot box

Studio and contacts

24 South Weber Street, Suite 225
Colorado Springs, CO 80903
(719) 244-8450sales@dastok.com