Hide
So RefWorks i really wonky with importing non-ascii UTF-8. This is really RefWorks bug, not Blacklights (it effected by legacy OPAC too), but a workaround is a good idea, since RefWorks sure as hell isn't going to fix their bugs, or even acknolwedge them.
While RefWorks is documented to accept utf-8, I discovered through painful trial and error that at least for the "Marc Format" import fitler we use, RefWorks can NOT handle certain "decomposed" UTF-8 characters (combining diacritics and such). But for all the test/demonstration/problem records I found, first normalizing them in Unicode Normalization Format C (NFC) would make RefWorks happy.
So I have committed a change where the new export_as_refworks_marc_txt method will force a normalization to NFC before outputting it's file. Test included to verify export_as_refworks_marc_txt NFC normalizes now. This requires the ruby 'unicode' gem, added the gem dependency in blacklight environment.rb, but you may still need to manually install it do to general plugin/gem weirdness. "sudo gem install unicode".
But then I wasn't out of the woods yet. My particular institution's Blacklight is deployed via https. Which means the refworks export URLs generated had callbacks to blacklight with https. And while normally Refworks is fine with https callbacks... for certain test UTF-8 records, normalized to NFC -- for some reason they work fine with RefWorks when the callback is http, but not when it is https. This makes NO SENSE at all, I don't even understand how RefWorks could possibly have a bug like that. But I verified it with isolation cases using simply static files served by apache, without even getting Blacklight involved.
Since this is so weird, and only effects people that have https access to their BL, and could result in mysterious problems for someone that has ONLY https but not http access to their BL -- I have not YET committed a change to BL to force all Blacklight Refworks export URLs to use http callbacks.
But what I HAVE done is refactored the code that creates the refworks export url -- moved it from hard-coded in the view to a helper method. This seems like just better cleaner design, and it also allows a local installer like me to easily over-ride it locally to, for instance, force http connections. So THAT refactor is committed. No tests for this I'm afraid, because we still haven't figured out how to test helper methods that generate paths -- and because this does not change the actual output at all anyway, it's just a refactor (so all existing cucumber tests are still valid).
So that's where we are. Closing this ticket as soon as I've opened it. But if anyone wants core BL to _insist_ on http callbacks in RefWorks export URLs, I can easily make that change in BL core, jsut not sure if that will end up being more or less confusing with terribly buggy awful RefWorks.
Show
So RefWorks i really wonky with importing non-ascii UTF-8. This is really RefWorks bug, not Blacklights (it effected by legacy OPAC too), but a workaround is a good idea, since RefWorks sure as hell isn't going to fix their bugs, or even acknolwedge them.
While RefWorks is documented to accept utf-8, I discovered through painful trial and error that at least for the "Marc Format" import fitler we use, RefWorks can NOT handle certain "decomposed" UTF-8 characters (combining diacritics and such). But for all the test/demonstration/problem records I found, first normalizing them in Unicode Normalization Format C (NFC) would make RefWorks happy.
So I have committed a change where the new export_as_refworks_marc_txt method will force a normalization to NFC before outputting it's file. Test included to verify export_as_refworks_marc_txt NFC normalizes now. This requires the ruby 'unicode' gem, added the gem dependency in blacklight environment.rb, but you may still need to manually install it do to general plugin/gem weirdness. "sudo gem install unicode".
But then I wasn't out of the woods yet. My particular institution's Blacklight is deployed via https. Which means the refworks export URLs generated had callbacks to blacklight with https. And while normally Refworks is fine with https callbacks... for certain test UTF-8 records, normalized to NFC -- for some reason they work fine with RefWorks when the callback is http, but not when it is https. This makes NO SENSE at all, I don't even understand how RefWorks could possibly have a bug like that. But I verified it with isolation cases using simply static files served by apache, without even getting Blacklight involved.
Since this is so weird, and only effects people that have https access to their BL, and could result in mysterious problems for someone that has ONLY https but not http access to their BL -- I have not YET committed a change to BL to force all Blacklight Refworks export URLs to use http callbacks.
But what I HAVE done is refactored the code that creates the refworks export url -- moved it from hard-coded in the view to a helper method. This seems like just better cleaner design, and it also allows a local installer like me to easily over-ride it locally to, for instance, force http connections. So THAT refactor is committed. No tests for this I'm afraid, because we still haven't figured out how to test helper methods that generate paths -- and because this does not change the actual output at all anyway, it's just a refactor (so all existing cucumber tests are still valid).
So that's where we are. Closing this ticket as soon as I've opened it. But if anyone wants core BL to _insist_ on http callbacks in RefWorks export URLs, I can easily make that change in BL core, jsut not sure if that will end up being more or less confusing with terribly buggy awful RefWorks.
Bug
Closed
Major