* FIX Fixes the order of search results when computing the
  next and previous search links on the record page by not
  reversing the initial search results.

Signed-off-by: Ludmila Marian <[email protected]>

* NEW Record recommender loads and displays recommendations on
  each record page. The recommendations are calculated with the speared
  python package the invenio-record-recommender.

* Loads the calculated recommendations from Redis.

* Creates /record/*id*/recommendations endpoint.

* Introduces CFG_RECOMMENDER_REDIS and CFG_RECOMMENDER_PREFIX config
  used to set the Redis instance from where the recommendations should
  be loaded.

* Renders recommended records in JavaScript after the record is loaded.

* Logs clicks on recommended records to webstat.

Signed-off-by: David Zerulla <[email protected]>

* Makes test suite independent from PDF extraction process.
  (closes inveniosoftware#1590)

* Simplifies test suite.

Signed-off-by: Patrick Glauner <[email protected]>

…ncho/1590-fix-regresion-test-solrutils

  SolrUtils: reliable regression test suite

 * Adds checkbox to search terms also in to fulltext to the add
   to search interface.

Signed-off-by: Patrick Glauner <[email protected]>

 * Ranks fulltext and metadata properly

Signed-off-by: Patrick Glauner <[email protected]>

 * Ceils ranking scores to avoid anomalies

Signed-off-by: Patrick Glauner <[email protected]>

…ncho/master-fulltext-by-default

  Solrutils: more accurate ranking scores
  BibRank: Solr: better ranking for fulltext by default
  WebSearch: new fulltext by default search

* Filters a record based on MARC code.

* Supports wildcards in identifiers and subfield code.

Signed-off-by: Patrick Glauner <[email protected]>

* Supports filtering per MARC tag, identifier and subfield.

Signed-off-by: Patrick Glauner <[email protected]>

…ncho/master-xm-field

  BibFormat: new 'xmf' format
  BibRecord: new MARC code filter

…lamarian/master-weblayout-cds

  WebStyle: add CERN specific URL handlers

Conflicts:
	modules/webstyle/lib/webinterface_layout.py

  BibEncode: Video: Push video to youtube

Conflicts:
	modules/webstyle/css/Makefile.am
	modules/webstyle/lib/webinterface_layout.py

* Added disabled-by-default config options for Elasticsearch logging.

* When Elasticsearch logging is configured, don't populate rnkPAGEVIEWS and
  rnkDOWNLOADS.

Signed-off-by: Joe MacMahon <[email protected]>

…ncho/elasticsearch_logging

  stats: Elasticsearch logging with Lumberjack.

* Adds `loanrequest` schema for `Lumberjack`.

* Adds custom events elastic search logging.

Signed-off-by: Harris Tzovanakis <[email protected]>

  WebStat: register custom events on es

* Back ports lazy dictionaries from next.

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

…ncho/master-backport-pidstore

  BibFormat: DataCite3 export addition
  pidstore: initial release backport
  xmlDict: initial release backport
  containerutils: new lazy data structures

…owski/1918-forbidden-characters-fix

  BibDocFile: disallow percent in filenames

…ilamarian/master-bibconvert-arxiv-license

  BibConvert: retrieve arXiv license information

Conflicts:
	modules/bibconvert/etc/oaiarxiv2marcxml.xsl

…ilamarian/master-bibedit-fix-recid

  BibEdit: fix recID type

…ancho/master-bibfield-bug-fixes

  BibField: bug fixes

…y-missing

  BibReformat: new --only-missing option

…oumis/legacy-webcomment-improvements

  WebComment: cosmetic updates
  WebComment: `collapse` and `expand` all comments
  WebComment: fix collapse and review issues
  WebComment: Enable a custom  checkbox for comments
  WebComment: display comment submission deadline
  WebComment: bibdoc relations & filtering
  WebComment: general comment format improvements

Conflicts:
	Makefile.am
	modules/webcomment/lib/webcomment.py
	modules/webstyle/css/invenio.css
	modules/webstyle/img/Makefile.am
	requirements.txt

* Moves page "Your Searches" from WebAlert to WebSearch and
  under the /yoursearches/ URL. Updates all the respective URLs
  and function calls. Moves function account_list_searches()
  from WebAlert to WebSearch.

* Separates the popular alerts page from the user searches page
  and creates new separate function for popular alerts the user
  can choose from to set up new alerts.

* Fixes various doc strings, variable and function names.

  (closes inveniosoftware#880)

* Redesigns the "Your Searches" page.

* Adds the "Search into your searches" functionality.

* Fixes a few small bugs.

  (closes inveniosoftware#881)

* Redesigns the "Your Alerts" display interface.

* Adds new functionality to display alerts based on
  a specific search query.

  (closes inveniosoftware#882)

* Fixes a small bug that caused special characters that
  had been replaced by the urllib.quote() function to crash
  during the string formatting.

  (closes inveniosoftware#883)

* Changes some CSS attributes to match yoursearches and
  youralerts style to the default Invenio style.

* Changes the order of the Personalize menu to be in alphabetical
  order again after the addition of Your Searches.

* Deletes some previously added unneeded images.

  (addresses inveniosoftware#884)

* Changes the arrows used for page navigation in the
  Your Searches page to match Invenio's default arrows.

* Deletes the respective previously added images files.

  (closes inveniosoftware#884)

* Introduces page navigation support for the
  "Your Alerts" page.

  (closes inveniosoftware#885)

* Introduces searching in the user's defined alerts based on
  the alerts' names and arguments.

* Fixes small bug in the paging navigation that caused a crash
  when no alerts were to be displayed.

* Adds user confirm step before permanently deleting an alert.

  (closes inveniosoftware#886)

* Fixes small bug in the paging navigation that caused a crash
  when no searches were to be displayed.

* Introduces fix that skips querying the database again when there
  are no matching user searches.

  (closes inveniosoftware#887)

* Fixes various warnings and errors reported by kwalitee.

* Adds new images for yoursearches and youralerts to the Makefile

* Adds the option to pause and resume alerts. The option can be
  configured either in when editing a specific alert or when
  displaying all the alerts.

(closes inveniosoftware#1227)

* Prettifies the display list header for Your Alerts and Your Searches
  by adding the paging navigation information.

* Hides the link to the popular alerts if there are none already defined.

* Improves the appearence of the "Store results in basket?" menu.

* Links the basket name to the basket display when displaying alerts.

* Harmonizes and prettifies the display of "Your Searches" and "Your Alerts"

* Revamps the "Your Account" page by arranging items to be
  displayed in a 3-column grid, trying to keep all columns
  equal in size.

* Re-arranges code for consistency and readability.

* Fixes various warnings and errors reported by kwalitee.

* Fixes the string checked in get_authenticated_mechanize_browser to make
  sure that the user was logged in correctly.

* Removes mentions of the number of total searches the user has performed
  in favor of only the unique searches to avoid possible confusion.

…oumis/legacy-personalization-fixes-and-improvements

  General: fix failing tests
  Websearch: remove mentions of total searches
  Miscutil: fix get_authenticated_mechanize_browser
  General: fix kwalitee reported issues
  Websession: Revamp the "Your Account" page
  Webalert & Websearch: harmonize display
  Webalert: improve store results in basket menu
  Webalert: smart display of popular alerts link
  Webalert & Websearch: prettify display list header
  WebAlert: option to pause and resume alerts
  WebStyle: add new images to the Makefile
  WebAlert & WebSearch: fix kwalitee reported issues
  WebSearch: Fix paging bug in yoursearches
  WebAlert: Searching for youralerts
  WebAlert: paging for youralerts
  WebSearch: more CSS fallback to default
  WebSearch & WebAlert: CSS fallback to default
  WebSearch: fix yoursearches string format bug
  WebAlert: youralerts interface improvements
  WebSearch: yoursearches interface improvements
  ...

Conflicts:
	modules/webcomment/lib/webcomment.py
	modules/webcomment/lib/webcomment_templates.py
	modules/webstyle/img/Makefile.am

* BETTER Cleans the port from the user's IP preventing issues in
  future manipulations of it.

Reviewed-by: Samuele Kaplun <[email protected]>
Signed-off-by: Esteban J. G. Gabancho <[email protected]>

* Improves `webinterface_handler_wsgi` PEP8 code style.

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

…ancho/legacy-enhancement-wsgi-ip

  WebStyle: kwalitee fix
  WebStyle: wsgi handler IP parser enhancement

…va/tab-lan

  WebSearch: tabs miss language parameter

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

Signed-off-by: Esteban J. G. Gabancho <[email protected]>

…ancho/bibindex-better-external-files

  BibRank: allow ranking external files using solr
  BibIndex: external file index enhancement
  WebSubmit: subtitle file addition to converter

* Changes in main navigation menu: Training->Learning

…l_main_navigation

  WebJournal: navigation menu category replacement

* The WebNews module that will be responsible for presenting the site's news
  to the users and keeping a history of them.

* This first release includes the tooltip functionality and the database
  structure.

  (addresses inveniosoftware#1288)

  WebNews: New module to display and manage news

Conflicts:
	modules/webstyle/lib/webinterface_layout.py

…umb_hack

  WebSearch: guess primary collection for CERN Site

…owski/copyrights_license

  BibDocFile: Copyright and license for files

* Adds `loanrequest` schema for `Lumberjack`.

* Adds custom events elastic search logging.

Signed-off-by: Harris Tzovanakis <[email protected]>

* NEW Adds a new ip_field column to staEVENT

* Uses this column to run GeoIP lookup on logged events

Signed-off-by: Joe MacMahon <[email protected]>

…eoip_logging

  elasticsearch: GeoIP in Elasticsearch logging
  WebStat: register custom events on es

Conflicts:
	modules/webstat/lib/webstat.py

* FIX Attaches `typeahead` autocomplete plugin to more specific element.

Signed-off-by: Harris Tzovanakis <[email protected]>

…ssing-authors

  WebSubmit: fix authors autocomplete

* FIX Checks only for error code in bibconvert of authorlist, because
  stdoutput warnings caused autholist to be ignored.

Signed-off-by: Harris Tzovanakis <[email protected]>

…horlist

  oaiharvest: authorlist missing

…owski/ckeditor

  WebStyle: CKEditor update

Conflicts:
	modules/webstyle/etc/invenio-ckeditor-config.js

…un/bibcirculation-bibedit

  BibCirculation: BibEdit redirect

…_mm_cds

  WebHelp: new multimedia help page for CDS

…sorting_washer

  BibSort: new washer for journal number

* Adds a new predefined keyword for year generation.

Co-Authored-By: Ludmila Marian <[email protected]>
Signed-off-by: Ludmila Marian <[email protected]>

…er_generation-enhancements

  WebSubmit: report number generation enhacement

…er_view_restr

  WebAccess: improve viewer rights access

…search_latest

  WebSearch: consider sort order when searching

* Adds `CFG_ELASTICSEARCH_FALLBACK_DIRECTORY` configuration variable
  to specify where the fallback file to be stored.

* Creates a fallback file with the hostname as file name.

Signed-off-by: Harris Tzovanakis <[email protected]>

…fallback

  elasticsearch: fallback file per hostname

* IMPROVEMENT Adds `downloads` and `pagesviews` as custom events.

* Custom events will be registered only in elasticsearch if
  `CFG_ELASTICSEARCH_LOGGING` is enabled.

Signed-off-by: Harris Tzovanakis <[email protected]>

…-option

  webstat: downloads and pageviews as events

* A ILL is currently extended by manually updating the *due_date* of
  that ILL, this leads to the following issue:
  A user receives an ILL recall letter, therefore gets the ILL extended
  at the library.
  The next ILL recall letter will be of the second category, since the
  *overdue_letter_number* never gets reseted.

* Introduces the new function *update_ill_request_letter_number*

* Changes the *overdue_letter_number* in case the updated due_date is a
  later date than the current due_date in
  *ill_request_details_step2*.

Signed-off-by: Martin Vesper <[email protected]>

…per/invenio_ill_recall_mail

  BibCirculation: reset ILL overdue_letter_number

  Notes:
    Update accordingly to Tibor's comments in inveniosoftware#3592

…e/legacy-obelix-pr

  WebSearch: optional Obelix integration

Conflicts:
	modules/miscutil/lib/Makefile.am
	modules/websearch/lib/websearch_webinterface.py
	requirements.txt

* Fixes creation of the Invenio config with the tool inveniocfg.
  The `elasticsearch_logging` module tries to use the invenio.config
  before the config is created. To reproduce remove the invenio.config
  and run `inveniocfg --update-all`.

Signed-off-by: David Zerulla <[email protected]>

  elasticsearch: Fix invenio.config import

…e/legacy-record-recommender

  BibFormat: optional record recommender

Conflicts:
	config/invenio.conf
	modules/miscutil/lib/Makefile.am
	modules/websearch/lib/websearch_webinterface.py
	modules/webstyle/css/invenio.css

* FFT__y subfield contains language for files uploaded to the
  committee documents collections.

Signed-off-by: Sebastian Witowski <[email protected]>

* If CFG_CERN_SITE, consider the possibility of the viewer
  of a record being an egroup, defined as '[email protected]'
  insted of 'foo [CERN]'.

Signed-off-by: Ludmila Marian <[email protected]>

…_record_viewer

WebSearch: improve detection of record viewers

…prev_next_order

Legacy fix prev next order

…button-disable

websubmit: disable submit button correction