Fixed TitleNormalizer for Chinese characters

[damien-git]

This PR for TitleNormalizer fixes the order for Chinese characters by using ICUCollatorNormalizer. Without it, a search using Chinese characters will return a position before A but these titles are appearing after Z. The current issue might also affect other unicode characters.

[todolson]

It seems we might want a TitleNormalizerTest to demonstrate the desired sort order with respect to how Roman, CJK, and potentially other code points sort. It could be based closely on or extended from ICUCollatorNormalizerTest.

[damien-git]

@todolson The current ICUCollatorNormalizerTest does not seem hard enough to properly test this particular issue (the current implementation for TitleNormalizer works fine with most scripts), but I don't feel qualified to write good tests for that.

[demiankatz]

@damien-git, would it be better than nothing to copy ICUCollatorNormalizerTest to TitleNormalizerTest and include something similar to the current diacritics test, only include a CJK string in the middle of the list and ensure that it sorts to the correct end? I probably don't know my way around the test framework here much better than you do, but if you need help, let me know and I'm willing to take a stab at it when time permits.

[todolson]

@damien-git, along those lines, it would be useful to see an example of maybe a half-dozen strings that illustrate the desired sorting. I'm far removed from the test framework these days myself, but between the three of us we can probably figure it out without too much effort.

[damien-git]

Yes, I guess the ICUCollatorNormalizerTest tests with an added example using Chinese characters would be better than nothing. I will prepare something.

[demiankatz]

Thanks, @damien-git. This seems reasonable to me, but I won't merge immediately just in case @todolson has any further suggestions!

[todolson]

I think this looks good. It gives a baseline expectation, and can be build upon should the need arise.

[demiankatz]

Thanks, everyone; merging now! @damien-git, do you have any more browse handler changes coming soon? If not, let me know if you'd like me to build this and update the dev branch of the main VuFind repo to use it. (Or you can open a PR to do that if you like!).

[damien-git]

@demiankatz I don't have additional changes coming. I was planning to prepare a VuFind PR with the jars once this PR is merged. I will do it this week.

[demiankatz]

@demiankatz I don't have additional changes coming. I was planning to prepare a VuFind PR with the jars once this PR is merged. I will do it this week.

Thanks, @damien-git, sounds good -- please let me know if I can do anything more to help; otherwise, I'll watch for your PR.