Calisphere is based on a harvest model, where we pull in metadata that points to content hosted on your own digital collections system. Most contributors have found the harvest process to be quick and easy. Although it requires some set-up at the beginning, once the process is in place, we can easily grab new collections (and pick up changes and additions to existing collections).
In 2021, we started an active development project called Rikolti, to replace our current Calisphere harvesting infrastructure. Please note that through 2023 we will temporarily pause harvesting from any new platforms not already in use by current contributors. For more information about the temporary pause to harvesting activities, please see the Rikolti update and temporary pauses to Calisphere harvesting announcement.
Here is a summary of the workflow for our existing harvesting infrastructure. We anticipate many of these steps will carry over to the new Rikolti harvesting workflow, though some details may be updated.
1. Establishing a Process
The first step to contribute to Calisphere is to simply let us know you're interested in having your collection(s) harvested! Contact us if you're ready to start. Please let us know who will be the key contact(s) at your institution, and who is primarily responsible for managing or coordinating processes on your digital collections system. We'll also need to obtain a Terms of Service Agreement from your institution, if you haven't already submitted one to us.
We'll set up a time to talk about your digital content and systems, and to map out the best method for harvesting data.
2. Defining Collections
Calisphere's harvest works on a collection-by-collection basis. By "collection," we mean a group digital objects which are hosted in the same source, which can be harvested according to the same protocol, and which you’d like to appear together as a set within the Calisphere interface. Some institutions have a single collection, and some have many.
We'll work with you to confirm which collection(s) you'd like to publish in Calisphere, and establish entries for them in our Collection Registry.
This includes basic information about the collection that will be used on Calisphere (such as title and description).
3. Metadata Mapping
Once we've determined the collections you'd like us to harvest, we'll review the source metadata to confirm required metadata is present, and that it is mapped correctly to our metadata scheme.
Some key bits of metadata are required -- but if it's not already in your records, no problem! We can globally "inject" the metadata into every object in the collection at the time it is harvested. We'll just need to confer with you on what information should be added.
4. Harvest to Test Calisphere Environment, QA, and Publication
Each time we run a harvest of your collections, we'll first put the records in a test Calisphere environment where you can preview and perform quality assurance (QA) on them.
If everything looks OK in the testing environment, we'll ask for your approval to publish the records in Calisphere.
That's it! Your collections will immediately be available on Calisphere, and they'll appear on the Digital Public Library of America (DPLA) soon thereafter (DPLA harvests from Calisphere on a bimonthly schedule).
5. Updating or Adding to Collections
Updating or adding to collections? We understand that your digital collections evolve; items are added and deleted and metadata changes. Our aim is to reflect changes to your collections as quickly as possible in Calisphere. Learn more about the process...
⚠ NOTE: Temporary pause in the Calisphere publication service during our transition to the new Calisphere harvesting system:
As mentioned in our recent progress update to develop the new Calisphere harvester, we are planning to transition all harvesting operations to the new harvesting system early this year. To prepare for this transition period, we’d like to share a few anticipated service updates:
- Our current harvesting system will conduct a final publication run on Thursday, January 11.
- After January 11, we will briefly be unable to publish new or updated collections as we transition Calisphere to work with the new index.
- Once we fully transition, we anticipate the new system will support a more frequent and nimble publication schedule for new collections and updates.
In the meantime, please feel free to submit requests to harvest and re-harvest collections; we will queue up your requests to publish your collections to the new index. Our intent is to have the new harvesting system in place by early this year; we will be in touch with a specific date to officially launch the new harvesting system.
Bonus Step: Add Custom Filters to your Published Digital Collection
Digital collections on Calisphere are published with two default filter options: Type of Item (e.g., image, text, audio, etc.) and Decade (e.g., 1910s, 1920s). Interested in adding custom filter selections to your collection? Learn more about Defining Custom Filters in Digital Collections...