Calisphere is based on a harvest model, where we pull in item records (metadata and associated thumbnails) from your digital collections platform. These harvested item records point users to the content published through your own digital collections platform.
Setting up the harvesting process requires some initial setup at the beginning, so that we can establish a connection with your digital collections platform. Most contributors have found the harvest process to be quick and easy. Although it requires some set-up at the beginning, once the process is in place, we can easily grab new collections (and pick up changes and additions to existing collections).
This page includes information about:
What is harvesting?
Calisphere uses a harvesting model for contributing your collections into this aggregation. This strategy allows us to programmatically “fetch” collections from your local digital asset management system – specifically, descriptive metadata and thumbnails for items in the collections. Once fetched, we “map” the metadata into a central index underlying Calisphere, to support searching and browsing of the items.
Harvest methods
We can harvest metadata from a range of digital collections platforms, listed below. We use the term “harvest” loosely to apply to a range of approaches, for example:
- Using the OAI-PMH protocol: Many platforms support options to enable an OAI-PMH feed, which provides metadata outputs for your selected collections or sets.
- Using APIs: Many platforms support an API (Application Programming Interface) response that will provide metadata outputs for your selected collections.
- Obtaining a metadata export: Some platforms support exporting metadata files, such as MARC21, MARCXML, or customized XML exports; please share the URL to the exported file and we can work with you to test out your export.
We currently work with a number of institutions using a range of different platforms to expose their metadata for harvesting. In most cases, there are just a few simple steps, requiring just a few minutes of time, that will allow you to share your collections with Calisphere. We can walk you through the process.
Our objective is to find the method that is easiest for you. We'll work with you to discuss your platform and figure out what that is; please contact us to get started!
It is important that your digital items records have stable -- and ideally, persistent -- URLs. When we harvest your digital item records, we create Calisphere-based URLs that are based on your source digital item records' URLs (e.g., in CONTENTdm, Omeka, DSpace, BePress, etc.). If you change those source URLs -- e.g., by migrating the collections to a different platform, changing the domain name, etc. -- the links from Calisphere to your source digital item records will break. If users have cited your items (e.g., in Wikipedia, articles, etc.), those links will also break.
Hence, we recommend using persistent URLs such as ARK or DOI whenever possible, to minimize broken links. If you do migrate collections from one platform to another or change domain names, you should also consider redirecting the old forms of URLs to the new URLs, so the older links do not break.
The list below includes platforms Calisphere currently harvests from; this list will continue to be updated as Calisphere establishes connections to additional platforms. Please feel free to let us know if you are interested in working with a platform that is not currently listed, and we can explore the harvest method strategies together.
See the “harvest methods” section above for additional details about the different ways Calisphere connects with source platforms.
Update in Progress: As part of our upgrade to the new Calisphere harvester, we have been working on updating previously established harvest methods (to programmatically fetch and map your data). Any indicated with an Update in Progress label is currently in progress. (Those without labels are completed.)
Digital Collections Platform | Digital Collection example in Calisphere | Method to establish the connection |
|---|
Alma/Primo
Update in Progress [Development Started] | California Section History Picture Catalog [Contributed by: California State Library] | MARC XML export: https://knowledge.exlibrisgroup.com/Alma/Product_Documentation/010Alma_Online_Help_(English)/040Resource_Management/075Publishing_Profiles/030Export_and_Publishing |
BePress / Digital Commons | Women's Studies, Feminist Zine Archive [Contributed by: Chapman University]
John Muir Journals [Contributed by: University of the Pacific] | OAI-PMH: https://digitalcommons.elsevier.com/en_US/digital-commons-and-oai-pmh |
Biblionix
Update in Progress [Development Started]
| Sierra Madre Historical Archives
[Contributed by: Sierra Madre Public Library] | MARC XML export |
CollectionSpace
Update in Progress [To Do] | Art Collection [Contributed by: UC Berkeley, Berkeley Art Museum and Pacific Film Archive (BAMPFA)]
| Solr API |
CONTENTdm | Travel Poster Collection [Contributed by: Los Angeles Public Library]
Jay T. Last Collection of Graphic Arts and Social History [Contributed by: Huntington Library]
OC Stories Photo [Contributed by: OC Public Libraries]
| OAI-PMH: https://help.oclc.org/Metadata_Services/CONTENTdm/CONTENTdm_Administration/Server_Administration/020Harvesting?sl=en
Notes: - Compound objects: In the ContentDM admin view, there will be an option to set up OAI harvesting. Please enable OAI harvesting, set “Enable compound object pages” to “No”, and relay the “Base URL for the OAI Repository” to us; this will give us a way to fetch the metadata records and images for all items–everything we need to pull into Calisphere.
- Transcriptions: When sharing collections, please ensure that any full-text transcript fields are not mapped to Dublin Core "Description", but enabled as a "Full Text Search" data type field.
- Dates: Additionally when sharing collections, please ensure that any date field intended for holding specific years (for years within a date span) are not mapped to Dublin Core “Date”, but are enabled as “searchable” and “hidden”; this will prevent the enumerated years from appearing in the OAI feed, but will still ensure that they are searchable within your CONTENTdm site.
|
Drupal | The Interviews: An Oral History of Television [Contributed by: Television Academy Foundation] | OAI-PMH: https://www.drupal.org/project/rest_oai_pmh |
Fedora | Eastman's Originals Collection [Contributed by: UC Davis, University Library, Special Collections] | OAI-PMH: https://wiki.lyrasis.org/pages/viewpage.action?pageId=66585896 |
Flickr | Cawton Ostrich Farm [Contributed by: South Pasadena Public Library]
Redwood City History [Contributed by: Redwood City Public Library] | Calisphere has an established connection with the Flickr API.
Organize your images into a Flickr album; Calisphere can harvest this set, which will display as a digital collection with item records for each image in your album. |
Internet Archive
Update in Progress [To Do] | Theater Guest Recordings Collection [Contributed by: UC Berkeley, Berkeley Art Museum and Pacific Film Archive (BAMPFA)] | Calisphere has an established connection with the Internet Archive API.
Organize your images into an Internet Archive collection; Calisphere can harvest this set, which will display as a digital collection with item records for each record in your collection. |
Islandora
Update in Progress [Nearly Finalized] | Burbank Parks and Recreation [Contributed by: Burbank Public Library]
Kinsey Sicks Collection, 2015-16 [Contributed by: GLBT Historical Society] | OAI-PMH: https://islandora.github.io/documentation/user-documentation/oai/
|
Luna Imaging
| Adobes in San Luis Obispo County Library Collection [Contributed by: Black Gold Cooperative Library System]
| OAI-PMH: https://lunaimaging.atlassian.net/wiki/spaces/V63Doc/pages/1707202/OAI-PMH+in+LUNA
|
NPGallery | Panoramic Fire Lookout Photographs, 1934-1935, 1937 [Contributed by: Yosemite National Park Archives]
Historic Images [Contributed by: Sequoia and Kings Canyon National Parks] | OAI-PMH (custom-generated)
NPGallery is a system developed by the National Parks Service: https://npgallery.nps.gov/About |
Omeka Classic | A.E. Hanson (1893-1986): Landscape Designs [Contributed by: UC Santa Barbara, Architecture and Design Collection, Art, Design and Architecture Museum]
Snapshot Stories [Contributed by: Santa Cruz Public Libraries] | OAI-PMH
Omeka plugin: https://omeka.org/classic/docs/Plugins/OaipmhHarvester/
Omeka S module: https://omeka.org/s/modules/OaiPmhRepository/ |
openEQUELLA | Hamaguchi Study Print Collection [Contributed by: California College of the Arts Libraries] | OAI-PMH: https://openequella.github.io/guides/AdditionalGeneralInfoHelp.html#turning-an-openequella-institution-into-an-oai-endpoint |
PastPerfect Desktop
Update in Progress [Development Started] | Center for Sacramento History Photo Collection [Contributed by: Center for Sacramento History]
History San Jose Online Catalog [Contributed by: History San Jose Research Library] | XML export: https://museumsoftware.com/WebHelp/Chapters/PP5-25e.htm Note: Please contact the PastPerfect/MuseumSoftware vendors ([email protected]) to make sure your software is up to date and that the XML export is configured. PastPerfect staff will also be able to help get everything set up, including which metadata fields and object types to share, and troubleshooting with generating XML files. We can work directly with PastPerfect staff as needed once we are ready to test harvest. |
Preservica
Update in Progress [Development Started] | Flatlands Newspaper Collection [Contributed by: Oakland Public Library, African American Museum and Library at Oakland]
Oakland Theaters Collection [Contributed by: Oakland Public Library, Oakland History Center] | API: https://developers.preservica.com/api-reference
Note: Although Preservica does offer an OAI-PMH feature>, it does not enable subsetting collections into sets [as of 2024]. If Preservica begins to support subsetting of collections into sets in their OAI-PMH, we are interested in exploring that strategy as a preferred harvest method over the API. |
Quartex | Changing Face of Southern California [Contributed by: Loyola Marymount University, Department of Archives and Special Collections, William H. Hannon Library]
Guerneville School Children's Book Project, 2020-2021 (SPV.0004) [Contributed by: Sonoma County Library] | OAI-PMH: https://community.quartexcollections.com/search?executeSearch=true&SearchTerm=oai-pmh&l=1 |
Samvera | Los Angeles Times Photographic Archives [Contributed by: UCLA, Library Special Collections, Charles E. Young Research Library]
Harry Mayo Surfing Photography Collection [Contributed by: UC Santa Cruz Library, Special Collections and Archives] | OAI-PMH |
TIND | Bonney (Thérèse) Photograph Collection [Contributed by: UC Berkeley, Bancroft Library] | OAI-PMH |
UC San Diego Library DAMS | Dr. Seuss Political Cartoons [Contributed by: UC San Diego, Special Collections and Archives] | Solr API: https://library.ucsd.edu/research-and-collections/digital-collections/dams.html
Note: This is a locally developed platform. |
UCSF Industry Documents Library DAMS
Update in Progress [To Do] | Pollay Advertising Collection
[Contributed by: UCSF Industry Documents Library] | Solr API: https://www.industrydocuments.ucsf.edu/research-tools/api/
Note: This is a locally developed platform. |
Veridian | Desert Circus Issues [Contributed by: Palm Springs Public Library] | OAI-PMH: https://veridiansoftware.com/knowledge-base/veridian-features/ |
YouTube
Update in Progress [Development Started] | Analog to Digital [Contributed by: San Francisco Public Library]
Local History Shorts [Contributed by: Harrison Memorial Library] | Calisphere has an established connection with the YouTube API.
Organize your videos into a YouTube playlist; Calisphere can harvest this set, which will display as a digital collection with item records for each video in your playlist. |
This is a running list of platforms we would like to establish harvesting pipelines with, in consultation with partner contributors using these platforms.
- Archipelago
- AtoM
- Axiell or Piction
- CalTech DAMS (locally developed)
- MontageDC
- Omeka S
- OrangeLogic
- Open Access
- PastPerfect Web Edition
- Recollect
Partner contributors have migrated off these systems; we do not have currently active collections from these sources. If this is a system you are using or considering, please let us know and we can discuss strategies to get started.
Please let us know if there are additional platforms that are not listed on this page.