Johns Hopkins Blacklight Implementation

Write a non-interactive 'batch' routine to do a bulk index, probably via rake task

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: Milestone 4
  • Component/s: None
  • Description:
    May avoid writing scratch files to disk by piping export stdout to solrmarc stdin

    (12 hours estimated development time)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Jonathan Rochkind added a comment - 20/Oct/10 3:52 PM
Mostly working. The development of this has of course been a back and forth process with bug-finding with john craig.

Adding several hundred thousand docs to solr index and only then committing seems to be somewhat undesirable. Am going to add auto-commit settings to our solrconfig.xml.
Show
Jonathan Rochkind added a comment - 20/Oct/10 3:52 PM Mostly working. The development of this has of course been a back and forth process with bug-finding with john craig. Adding several hundred thousand docs to solr index and only then committing seems to be somewhat undesirable. Am going to add auto-commit settings to our solrconfig.xml.
Hide
Permalink
Jonathan Rochkind added a comment - 10/Nov/10 5:46 PM
Done, we have a rake task to pipe an export using Alpha-G utility directly to SolrMarc, to Solr.

Documented at https://wiki.library.jhu.edu/display/PHWG/Indexing
Show
Jonathan Rochkind added a comment - 10/Nov/10 5:46 PM Done, we have a rake task to pipe an export using Alpha-G utility directly to SolrMarc, to Solr. Documented at https://wiki.library.jhu.edu/display/PHWG/Indexing

People

Dates

  • Created:
    17/Jun/10 5:12 PM
    Updated:
    10/Nov/10 5:46 PM
    Resolved:
    10/Nov/10 5:46 PM
Atlassian JIRA (v4.1.2#531) | Report a problem | Request a feature | Contact administrators
Powered by a free Atlassian JIRA open source license for Blacklight. Try JIRA - bug tracking software for your team.