To load a vendor’s images and metadata without breaking anything, agree the delivery package before scanning starts: one row per object keyed by the identifier your system matches on, that identifier in every file name, one date format, UTF-8 text, and no line breaks inside fields. Then back up, run a test import of about 20 hard records, check every join, and only then load the rest. The details differ by system: the PastPerfect import “only adds new records” and images are attached separately, CONTENTdm and CollectiveAccess match files by file name, and Omeka’s CSV importers decide where files come from and whether existing items can be updated.
Everything below comes from each vendor’s own documentation, read in November 2026 and cited where it is used. We do not rank these products. The aim is to show where each one’s import breaks the link between a file and its record, so you can write a delivery specification that avoids it. Field-by-field mapping lives in our metadata mapping worksheet, and the choice of standard is covered in Dublin Core, MARC or EAD.
The three joins an import has to keep
Every import of digitized material has to hold three connections. When records come out wrong, one of these joins has failed, so it helps to name them before looking at any product.
- Row to record. Each row in the delivery spreadsheet becomes, or updates, one catalog record. The join is the identifier: an Object ID, accession number or local identifier that the system stores in a known field.
- File to row. Each image, PDF or transcript has to land on the right record. Most systems make this join through a file name column in the spreadsheet, and several require the file name to match exactly.
- New to existing. If a record for the item already exists, the import must either skip it, add to it, overwrite it or create a duplicate. Some systems let you choose; some only ever add.
The rest of this article goes through each system’s documented behavior on those three joins, then turns it into a delivery specification and a test procedure.
Decide what your system matches on before the vendor delivers
Choose the matching identifier before capture starts and write it into the contract. Every other rule in the delivery depends on it, and changing it after thousands of files are named is expensive.
The documented matching fields differ by system:
| System | What identifies a record | What attaches a file |
|---|---|---|
| PastPerfect 5.0 | Object ID (required for catalog imports; PastPerfect “will not assign these numbers”) | Images are attached by hand, record by record |
| CONTENTdm (classic) | Records are created from rows; later edits by script use CONTENTdm’s own item ID | A file name field in the tab-delimited text file |
| CONTENTdm (new version) | Records are created as drafts from rows | A column you select as the file name column |
| Omeka Classic | Records are created from rows | Public, permanent file URLs in a column |
| Omeka S | An identifier column matched to a property you choose, for updates | File URLs, or file names in a sideload folder on the server |
| CollectiveAccess | idno, preferred label or internal ID, by the policy you set | A media file name matched exactly to files in the import directory |
Two consequences follow. First, if your system matches on an identifier, every row the vendor delivers must carry it in exactly the form the system holds, punctuation included. Second, the file name stem and the catalog identifier do not have to be the same string, but the connection between them must be written down in the record. The file naming builder produces box, folder and item stems; the mapping worksheet has a row for recording how they relate to your accession numbers.
Put the identifier rule, the file naming pattern and the import layout into your request for proposals, so bids are priced against them. Our guide to writing a digitization RFP shows where those requirements sit in the document.
PastPerfect import: what it does and does not do
The PastPerfect import adds new catalog records from a file. It does not update records, and it does not attach images. Those two facts shape every PastPerfect project.
The desktop import in version 5.0
PastPerfect 5.0’s Import Data screen sits under Utilities. Its first warning is the one to plan around: “Importing data into PastPerfect only adds new records. It does not update existing records.” The guide also says, “Please create as many fields in your file as possible. You will not be able to import more data into these records later.”
The rules that matter for a vendor delivery:
- Identifiers and names are required. Catalog imports need an Object ID for every record, and PastPerfect “will not assign these numbers.” Each item also needs an Object Name; the guide suggests “Unknown” when you are not sure.
- No duplicates. Object IDs must not repeat within the file or match IDs already in PastPerfect. The pre-import screen lets you exclude duplicates and blank IDs, and the guide recommends doing so.
- One table per import. You import into Objects, Photos, Archives, Library, Accession or Contacts, one at a time, so a mixed delivery has to be split into one file per catalog.
- Field length. From Excel and ASCII files, “a limit of 254 characters per field is imposed. Fields with text longer than 254 characters will be truncated.” Long descriptions and transcriptions need another route.
- Columns and order. For Excel and ASCII files you choose the target fields and must keep them in the same order as the file. The guide warns that “Empty columns will result in data going into the wrong fields.”
- Dates. In delimited files, dates must be mm/dd/yy or mm/dd/yyyy. A two-digit year is assumed to be in the 20th century.
- Afterwards. You must reindex after the import. If the data did not come in as expected, the documented remedy is to “restore the backup you made before importing and try the import again.”
Images are attached separately
The import fills the catalog tables only. The guide says these “do not include any of the related tables, such as Condition Reports, Media (for images)”, and that “images must be manually attached.” Image handling needs the optional MultiMedia Upgrade, which PastPerfect sells for an additional charge.
When you attach an image, PastPerfect copies it into numbered subfolders and renames it from the Object ID “with periods, dashes or other punctuation removed”: an image on record 2003.2.2 becomes 200322.jpg, and a second image becomes 200322-2.jpg. The vendor’s file name does not survive inside PastPerfect. Its Image Metadata screen, however, has fields for the “Location of High-Resolution Archival Image” and the “High-Resolution Image File Name (including path)”. Record the master’s file name there, so the record can always be joined back to the preservation file.
PastPerfect also treats attached images as discovery images of limited size. Its guide suggests that files of about 100 KB for JPG are adequate, and an optional setting turns any image over a size threshold (2,000 KB by default) into a small attached JPG plus a link to the large file. Keep your masters outside the application in any case; the attached copy is for finding the record.
Web Edition
PastPerfect’s cloud product has its own user guide. Its knowledge base was not readable by our tools, so we rely on the Web Edition User Guide. Images are added one at a time from a record’s Image Management screen, in JPEG, TIFF, PNG, GIF or BMP; HEIC and WebP are not supported. The guide’s Utilities menu lists backups, a DPLA export, analytics, a recycle bin and login history, and we found no catalog data import described there. PastPerfect offers a paid data conversion service that asks for a data map and, if images are included, a sample of 10 to 15 image files. Ask PastPerfect support how a vendor’s delivery should reach Web Edition before you fix the layout.
What this means for a PastPerfect delivery
Because records cannot be updated by import, finish description before you load. Load metadata once, check it, and plan the time to attach images record by record, or ask PastPerfect about its conversion service. Keep long text, such as full transcriptions, in files linked to the record rather than in an imported field that will be cut at 254 characters.
CONTENTdm: the classic Project Client and the new version
CONTENTdm builds records from a delimited text file and matches files by name. OCLC’s help site, as of November 2026, documents two products: CONTENTdm (classic version), loaded through the desktop Project Client, and CONTENTdm (new version), loaded in the browser. They behave differently.
Classic: tab-delimited files and exact file names
The Project Client’s Add Multiple Items wizard can take files from a directory or with a tab-delimited text file. The file’s first record holds the field names, and “one field in each subsequent record must contain the file name of the item that you are adding.” You then map the imported fields to the collection’s fields.
OCLC’s rules for tab-delimited files are strict, and most of them protect the file join:
- “The file names must exactly match the contents of the upload directory for the files. One single mismatch causes the entire upload to fail.”
- File names include the extension, without a path, and must be unique, and case does not make them unique: “item.jpg is treated the same as ITEM.jpg and ITEM.JPG.”
- Remove the special characters \ / : * ? “ < > | from file names and field names.
- Put the file name field last “for best results”, end each record with a carriage return, and use no tabs or carriage returns inside a field.
- At least one field must map to Title.
- Date fields accept forms such as yyyy-mm-dd, yyyy-mm, yyyy, mm/dd/yyyy and ranges such as 2008-2009.
From Excel, OCLC advises turning off text wrap, which can add a character that makes the import fail, and replacing quotation marks inside fields with apostrophes. When a tab-delimited import fails with “File Does Not Exist”, OCLC suggests checking for extra carriage returns or tabs and confirming that the file name field is mapped to Object File Name.
Classic: compound objects from a vendor
Letters with several pages, ledgers and scrapbooks are usually compound objects. OCLC describes three ways to add many at once. The directory structure method expects a separate directory for each compound object, each with its own tab-delimited file, all of one type, and OCLC notes that “some scanning vendors provide your data with this structure.” The object list method uses one text file for all the objects but cannot carry page-level metadata.
In a compound object’s text file, the second record holds the object-level metadata and later records hold the pages and their file names. OCLC recommends no more than 10,000 items in a project, and notes that the original files stay where they were; the Project Client adds copies.
New version: bulk uploads of up to 500 items
In the new version, Add multiple digital items takes up to 500 files at a time with an optional metadata file in CSV, TSV or TXT format. The metadata file must be one file for the whole upload, “with one row per item”, no larger than 1 MB. You select the column that holds the file name and map the other columns to collection fields, including every property the collection’s template requires.
Optional transcript files are matched “to the item with the same file name”, and unmatched transcripts are skipped and logged. Items arrive as drafts, “individual errors do not prevent the batch from completing”, and errors go to an error log. A compound item can hold up to 500 pages.
The 500-item limit sets the batch size for a large delivery, and the 1 MB limit on the metadata file affects how much text one batch can carry. Ask the vendor to split deliveries into batches that fit both.
Omeka Classic and Omeka S: two different CSV importers
Both versions of Omeka import from spreadsheets, but they fetch files differently and only Omeka S can update existing items by identifier.
Omeka Classic CSV Import
The CSV Import plugin treats each row as “metadata for a single item.” Its documented constraints:
- Files come from the web. “File import only works for files hosted and publicly available online; you cannot import files from a local source (that is, your computer), or from a password-protected location.” The column must hold permanent URLs, and the manual warns that storage services such as Google Drive or Dropbox may not work without a stable URL to each file.
- One item type and one collection per import. If you map to item type fields, every row in the file must be the same item type.
- Clean structure. One header row, unique column names, and the same number of columns in every row.
- Encoding. The file “should be UTF-8,” and Omeka “cannot accept metadata entries that include emojis.”
- Automapping. Column names in the form “Dublin Core:Title” map automatically.
- Undo. An “Undo Import” link deletes all the items and files an import added.
The URL rule is the one that catches vendor deliveries. A drive of JPEGs is not enough for Omeka Classic; the access files must first be placed somewhere they can be reached by stable public URLs, and those URLs go in the file column.
Omeka S CSV Import
The Omeka S CSV Import module accepts CSV, TSV or ODS files, “must be encoded in UTF-8,” needs a header row, and automaps headers written as vocabulary terms such as dcterms:title. Only Global Administrator and Supervisor users can run it. Media can come from URLs or, with the File Sideload module, from file names in a folder on the server.
Its main difference from the Classic plugin is the Action setting. Beyond creating new items, an import can append, revise, update or replace data on existing items, or delete them. For those actions you choose a resource identifier column and the property it matches. The manual warns that matching “will only work with exact matches,” and that if more than one resource matches, “it will only take action on the oldest resource.” Each of these actions is marked “(Cannot be undone.)” A plain create import can be undone from the Past Imports page.
If any columns feed fields that come from modules, such as map coordinates from the Mapping module, the manual says to install and configure those modules before importing, and warns that “data may be lost if you uninstall those modules later.”
Two further cautions from the manual are worth copying into your procedure. When using File Sideload with CSV Import, disable the module’s file deletion setting, because “CSV Import may delete files from your Sideload directory even if an import job has failed or stalled.” And the default batch of 20 rows can be set to 5 or even 1 to find the row that causes an error. Never sideload from the only copy of your access files, and never from your preservation masters.
CollectiveAccess: mappings, record policies and media by file name
CollectiveAccess imports through a mapping spreadsheet that you write once and reuse. It gives detailed control over how new rows meet existing records, and that control has to be set deliberately.
The mapping and the existing record policy
An import mapping lists each source column and its destination. For spreadsheets the source is given by column number (A = 1, B = 2). The setting called existingRecordPolicy “determines how existing records in the CollectiveAccess system are checked for and handled.” Its options include none, and skip, merge or overwrite matched on idno (the identifier field), on preferred label (the title or name), on both, or on internal record IDs, with “with_replace” variants of merge. A mergeOnly setting updates matching records and creates none.
The errorPolicy setting decides whether an error is ignored, stops the import or prompts you, and the documentation says the default is to ignore. For a vendor delivery, choose matching on idno and set an explicit error policy, so a bad row is noticed rather than skipped in silence.
Media matched by exact file name
Media is attached through the object representation splitter. Its media setting names the column holding the file name, “which is what will match on the actual media file in the import directory.” The documentation adds that “filenames in source data may or may not include the file extension, but source data must match filename exactly.” A special source, row, can write each record’s spreadsheet row number into a field such as internal notes, which makes it easy to trace a record back to its row during testing.
Testing and rerunning
The Running an Import page offers a “dry run” that writes a log “without actually creating any records in the system.” It advises backing up the database before every import, “as you will likely have to tweak imports multiple times,” and recommends the command line for large imports. To rerun after fixing a mapping, the documented route is to restore the database dump and run the import again.
How the four systems compare on the joins that break
The table below gathers the documented behavior in one place. It compares mechanics, not quality, and the right system for you depends on much more than its import.
| System | Metadata file | How files attach | Updates existing records by import? | Test or reverse | Documented limits |
|---|---|---|---|---|---|
| PastPerfect 5.0 | Excel, ASCII (comma or tab), dBase, FoxPro | Attached by hand; renamed from Object ID | No: "only adds new records" | Back up first; restore the backup | 254 characters per field from Excel or ASCII; one table per import |
| PastPerfect Web Edition | No import tool found in the user guide; vendor conversion service | Attached one at a time | Not described | Ask PastPerfect | JPEG, TIFF, PNG, GIF, BMP; no HEIC or WebP |
| CONTENTdm (classic) | Tab-delimited text | File name field, exact match, case-insensitive | Not by the Project Client; the Catcher web service edits metadata by script | Review in project spreadsheet before upload | One mismatch fails the upload; 10,000 items per project recommended |
| CONTENTdm (new version) | CSV, TSV or TXT, one row per item | Chosen file name column | Not by this import | Items arrive as drafts; error log | 500 items per batch; 1 MB metadata file |
| Omeka Classic | CSV, UTF-8 | Public, permanent URLs | Not described | Undo Import | One item type and one collection per import; no emojis |
| Omeka S | CSV, TSV or ODS, UTF-8 | URLs, or sideloaded file names | Yes: append, revise, update, replace by identifier | Undo for create imports; other actions cannot be undone | Exact identifier match; oldest record wins |
| CollectiveAccess | Excel, CSV, XML and others via a mapping | Media file name, exact match in import directory | Yes: skip, merge or overwrite by policy | Dry run; restore database dump | Error policy defaults to ignore |
Where the link usually breaks, and how to spot it
Most failed imports show one of a handful of symptoms, and each points back to one of the three joins. The table pairs what you see after a test load with the documented behavior that usually explains it.
| What you see | Likely cause | Where to check |
|---|---|---|
| Values in the wrong fields, shifted one column over | An empty column, or field order in the file different from the order chosen on screen | PastPerfect warns that “empty columns will result in data going into the wrong fields” |
| Descriptions cut off mid-sentence | Text over 254 characters in a PastPerfect Excel or ASCII import | Measure the longest value in each column before import |
| The same item appears twice | A second PastPerfect import of records already loaded, since import only adds | Search for the identifier; exclude duplicates on the pre-import screen |
| “File Does Not Exist” in CONTENTdm classic | A stray carriage return or tab, a misspelled file name, or the file name field not mapped to Object File Name | Open the text file in a plain text editor and look for short rows |
| The whole CONTENTdm classic upload fails | One file name in the text file not matching the upload directory | Compare the file name column with a directory listing |
| One image shows on the wrong record | Two files whose names differ only by capital letters | CONTENTdm treats item.jpg and ITEM.jpg as the same name |
| Omeka Classic items created without files | File URLs that are local, private or not permanent | Paste a few URLs into a browser that is not logged in |
| An Omeka S update changes the wrong item | An identifier that is not unique, so the oldest matching item was changed | Check the identifier property for duplicates first |
| Rows missing from CollectiveAccess with no message | The error policy left at its default of ignore | Set the error policy and read the import log |
| Accented names turned into odd symbols | A file saved in an encoding other than UTF-8 | Re-save as UTF-8 and re-test the record with the most diacritics |
The first three rows reflect PastPerfect’s documented import. The identifier and encoding problems further down can occur in any of these systems, because all of them match on text.
Corrections after the load
Plan how you will correct records once they are live, because the import that created them may not be able to fix them. PastPerfect 5.0’s import cannot update records, so corrections are made in the records themselves. In CONTENTdm’s new version, metadata can be changed from My drafts or Published items. For the classic version, OCLC documents a web service called Catcher that “supports batch requests to add metadata-only records,” to “edit existing metadata fields in any record, or delete a record.” It is scripted, works in XML and needs each item’s CONTENTdm ID, so it suits a technically confident user. Omeka S can revise or update items by identifier, and CollectiveAccess can merge or overwrite on idno. In both, test a correction import on a few records first, because a mistaken update spreads as quickly as a mistaken load.
A delivery specification your vendor can follow
A delivery specification turns the rules above into instructions a vendor, a volunteer or your own staff can follow without knowing your software. The items below satisfy the strictest documented rule in each area, so the same package can serve your current system and a future migration.
| Rule | What to require | Why (documented constraint) |
|---|---|---|
| Identifier | Your catalog identifier, exactly as stored, in its own column on every row | PastPerfect needs Object IDs it will not assign; Omeka S updates only on exact matches; CollectiveAccess matches on idno by policy |
| File names | The file name stem in every file name; lowercase; no spaces or \ / : * ? “ < > |; unique regardless of case; extension included | CONTENTdm fails the whole upload on one mismatch and ignores case; CollectiveAccess matches exactly |
| File name column | One column per row naming the file, without a path; placed last | CONTENTdm classic’s guidance |
| Rows | One row per object; separate files per catalog or item type where the system requires it | PastPerfect imports one table at a time; Omeka Classic one item type per import |
| Compound objects | One directory per object with its own text file, or object rows followed by page rows | CONTENTdm classic’s directory structure method |
| Dates | A sortable ISO date column (yyyy-mm-dd), plus a display date; produce the system’s format from it | CONTENTdm accepts yyyy-mm-dd; PastPerfect’s delimited import needs mm/dd/yyyy |
| Encoding | UTF-8, no emoji | Omeka Classic and Omeka S require UTF-8 |
| Field content | No tabs or line breaks inside fields; long text in separate files | CONTENTdm classic; PastPerfect truncates at 254 characters |
| Provenance | A review status column beside every machine-suggested field | Keeps labels through import and export |
| Batches | Batches sized to the target, with a manifest listing every file and checksum | CONTENTdm new version takes 500 items per batch |
| Masters | Preservation masters delivered separately from access files, never placed in an import folder | Sideload and attachment steps copy, rename or delete files |
Two rules deserve explanation. The date rule asks for two columns because no single format suits every system: a year-first date sorts correctly and travels well, and a script or spreadsheet formula can produce mm/dd/yyyy for PastPerfect from it. The provenance rule matters if any titles, subjects or names were suggested by software. Our article on labeling AI-generated metadata explains the paired status columns; whether they survive depends on your system having a field to receive them, which is what the test import checks.
What the vendor should hand over with the files
The delivery is more than images and a spreadsheet. Ask for these items alongside them, so that whoever runs the import, now or after a migration, can check it without calling the vendor:
- A data dictionary. One line per column: its name, the field it maps to in your system, the allowed values or format, and whether it may be empty.
- A file manifest. Every file name with its size and checksum, grouped by batch, so you can confirm nothing went missing between the vendor’s drive and your import folder.
- A list of exceptions. Items not captured, pages skipped as blank, files renamed after capture, and any rows that break the rules above, with the reason.
- The review status key. What each status value means, who reviewed which fields and when, if any values were machine-suggested.
- A test load result. If the vendor has access to a test copy of your system, or to the same product and version, the result of loading the first 20 records there.
None of these is specific to one product, and all of them outlast it. When you change systems in ten years, the data dictionary and manifest are what let the next import reconnect files and records.
Example (fictional). The Aster County Historical Society loads Box 3 into PastPerfect 5.0 and publishes the same items in Omeka Classic. Its specification asks the vendor for two delivery files built from one master spreadsheet:
- For PastPerfect: an Archives catalog file and a Photos catalog file, each a comma-delimited file with Object ID (for example ACHS-3-012 and ACHS-3-C04), Object Name, title, dates as mm/dd/yyyy, and descriptions under 254 characters. Full transcriptions of the 41 letters go in text files named with each letter’s file stem, linked to the records later.
- For Omeka Classic: one CSV per item type in UTF-8, with “Dublin Core:Title” style headers, the same identifiers in a Dublin Core:Identifier column, and a file column of permanent URLs to the access JPEGs on the society’s web host.
- For both: access files named like achs_c003_b03_f001_i012_0001_a.jpg, masters as TIFF on a separate drive with a checksum manifest, and a status column beside the subject field, since those subjects were machine-suggested.
When a volunteer attaches images in PastPerfect, she types the master’s file name into “High-Resolution Image File Name”, so record ACHS-3-012 can always be traced to its TIFF even after PastPerfect renames the attached copy.
Test with 20 records, then load the rest
A test import of about 20 records, chosen to include the hardest cases, finds most problems before they are copied a thousand times. Twenty is a working number, not a vendor rule: enough to include every awkward case once, small enough to check every field by hand.
Choose the 20 on purpose
Pick records that break things, not typical ones:
- the longest description and the longest title in the delivery;
- names with diacritics or other non-ASCII characters;
- a record with several values in one field, such as three subjects;
- a compound object with page-level data, such as a multi-page letter or ledger section;
- a record with no date, and one with an approximate date;
- an item whose identifier already exists in your system;
- a record with machine-suggested fields and their status columns;
- two files whose names differ only by case, if any exist.
Run it
- Back up. PastPerfect’s and CollectiveAccess’s documentation tell you to back up before importing; do the same for every system. Note where the backup is and how to restore it.
- Use a test space if you have one. A test collection in CONTENTdm, a test site or item set in Omeka, or a dry run in CollectiveAccess keeps trial records out of the public catalog.
- Import the 20 rows with the exact settings you intend to use for the full load, and record those settings.
- Count. The number of records created, files attached and errors reported should match what you expected.
- Open every record. Compare each field with the spreadsheet: truncation, characters, date conversion, multi-value fields split correctly, labels in the right place.
- Check the file joins. Open each image from its record and confirm it shows that item, in page order.
- Check the existing-record case. Confirm the duplicate identifier was skipped, merged or rejected as you intended, not added twice.
- Export and compare. Export the 20 records and compare them with the delivery. This round trip shows whether review status columns and identifiers survive leaving the system as well as entering it.
Reverse it, fix it, run it again
Write down how you will undo a bad load before you run the full import:
- PastPerfect: restore the backup made before the import.
- CONTENTdm classic: review in the project spreadsheet before uploading; the original files remain where they were.
- CONTENTdm new version: items arrive as drafts, so check them before publishing.
- Omeka Classic: use Undo Import on the Status tab.
- Omeka S: undo create imports from Past Imports; back up the database before any append, revise, update, replace or delete action, since those cannot be undone.
- CollectiveAccess: dry run first; to rerun, restore the database dump.
When the 20 pass, load in batches sized to the system, check counts after each batch, and keep a log that pairs each batch with its delivery manifest. Use the QC sample size calculator to decide how many records from each loaded batch to open by hand.
What to do before your next delivery
Most broken imports trace back to a decision nobody wrote down: which identifier the system matches on, where files come from, or what happens to records that already exist. Settle those three in writing, then check them with a test load.
- Confirm which product and version you run, and read its import documentation yourself; versions change behavior.
- Write down the identifier field your system matches on and the exact form of its values.
- Fill in the metadata mapping worksheet for your fields, including a row for file name stems.
- Generate the naming pattern with the file naming builder and give it to whoever scans.
- Put the delivery specification above into your contract or request for proposals.
- Keep preservation masters out of every import folder, with checksums, as part of a digital preservation plan.
- Run the 20-record test, write down the rollback, then load in batches.
If a vendor is preparing your records, ask for the delivery to be built against your system’s documented import, not a generic spreadsheet. Our AI-assisted cataloging service keeps machine-suggested fields labeled in the delivered records, and our collection search and publishing service covers import into a client’s own system or a search site. If you want to see how a package behaves in your own software before committing a collection, a pilot box runs a small batch of your material through every stage first.