RFC: new circulation module

[mvesper]

The circulation module is going to be rewritten from scratch in Invenio 2, making it more flexible and covering more library use cases. The purpose of this RFC is to discuss current development stages.

Current stage: Data structure

It's about what to store in a database.

Two ideas:

    1+-----------------+                                          
+--->|Record           |                                          
|    +-----------------+                                          
|    |Won't change this|                                          
|    +-----------------+                                          
|                                                                 
|            +---------------------------------+                  
|            v1                                |                  
|    +--------------+        n+------------+   |                  
|    |ItemManager   |    +--->|Item        |   |                  
|    +--------------+    |    +------------+   |                  
+----+record        |    |    |item_manager+---+                  
     |items: []     +----+ +--+infos       |                      
     |requesters: []+--+   |  |status: []  |                      
     +--------------+  |   |  |history: [] +---------------+      
                       |   |  +------------+               |      
+----------------------+   |           ^n                  |      
|                          |           |                   vn     
|   n+-----------+1        |           |             +-----------+
+---^+Borrower   +^--------+           |             |Event      |
     +-----------+                     |             +-----------+
     |           |                     |             |date       |
     |loans: []  +---------------------+             |description|
     |history: []+---------------------------------->|borrower   |
     ------------+                                  n|item       |
                                                     +-----------+

    1+-----------------+                       
+--->|Record           |                       
|    +-----------------+                       
|    |Won't change this|                       
|    +-----------------+                       
|                                              
|           +---------------------------------+
|           vn                                |
|    +--------------+        1+-----------+   |
|    |Item          |    +--->|Borrower   |   |
|    +--------------+    |    +-----------+   |
+----+record        |    |    |loans: []  +---+
     |borrower      +----+    |history: []+---+
     |status: []    |         +-----------+   |
+----+history: []   |               ^n        |
|    |requesters: []+---------------+         |
|    +--------------+                         |
|                                             |
|                                             |
|   n+-----------+n                           |
+--->|Event      |<---------------------------+
     +-----------+                             
     |date       |                             
     |description|                             
     |borrower   |                             
     |item       |                             
     +-----------+                             

• I know calling any class '*Manager' is a bad idea, but well ^^
• Both outlines should be able to handle the current use cases of
  • Simple loans
  • Simple loans, with unavailable item(s)
  • inter library loan
  • purchase request
• ItemManager?
  • The ItemManager would just handle the steps required for items which are not immediately available
  • The same functionality could be provided using some more elaborate (maybe not?) database queries
• The Event objects are proposed to keep track of loans and item statuses in order to satisfy the need to generate meaningful statistics
• The attributes are dummies

Personally, I suggest the second approach.

Criticism and ideas are highly encouraged :)

[jirikuncar]

Simple (first iteration)

Objects

• records
• items
• users

Relations

• record has many items
• user can borrow many items

States

• items - on shelf, borrowed, requested, ...

Second iteration

Objects

• records
• items
• users
• requests

Relations

• user has many requests
• request has (item | many items)
• record has many items

States

• items - on shelf, borrowed, requested, ...
• requests - new, pending, running, expired, done

Actors

• user can create new requests
• "desk manager" can process requests

Can we start with proper state machine? Can we meet and create a model in e.g. http://www.uppaal.org/ (please note the uppal requires licence!) ?

[aw-bib]

Some further iteration:

• Items commonly have an accession number as primary key. This can but has not to be the same as the barcode of the item. Both are imporant values.
• Other item related values can be found in LoC holdings tag 852: Location as well as LoC holdings tag 876/878: Item Information-General Information. It seems clever to follow them, they have quite an experience in item information at the Library of Congress.
• Items can hold supplemental material (ie. refer to other records) like a CD-ROM or DVD or a map... If you borrow such an item you usually (but not always) borrow the supplement as well.
• States need to be configurable. Additional states would be at least
  • missing
  • reference work (== not for loan)
  • permanent loan (this could also be handled as a sublocation)
• It would be nice if states can be configured by a librarian not a programmer
• The "on shelf" (= shelfing and location) state needs to be more explicit (probably not in this part of the model?): on what shelf?
  • Items might be in several departments
  • several libraries
  • several sublocations within a library (common is: one reference work not for loan, one or several for loan)
• It can well be, that holding a history of all borrowers of a given item runs into legal issues (privacy). In Germany you may hold the last borrower (sure) but not those before => clipping of borrower history needs to be configurable.

Pinging some of our fellow librarians in join2 for additional input: @katringrosse, @CLSi, @mahncm, @shesselbach, @martinkoehler

[kaplun]

• States need to be configurable. Additional states would be at least
  • missing
  • reference work (== not for loan)
  • permanent loan (this could also be handled as a sublocation)
• It would be nice if states can be configured by a librarian not a programmer

Agree on the configurability, up to a certain limit, since some states are expected to exists in order for workflows to correctly work (e.g. from on-shelf to on-loan etc.)

• The "on shelf" (= shelfing and location) state needs to be more explicit (probably not in this part of the model?): on what shelf?
  • Items might be in several departments
  • several libraries
  • several sublocations within a library (common is: one reference work not for loan, one or several for loan)

A single item can be in a single position (shelf/department/library), and typically it's a stable property for the item. So either the book is on loan, or when it's on shelf it's in its declared position. (which can be of course amended).

• It can well be, that holding a history of all borrowers of a given item runs into legal issues (privacy). In Germany you may hold the last borrower (sure) but not those before => clipping of borrower history needs to be configurable.

Good point.

[tiborsimko]

Some additional musings:

• In theory, an item may be associated with more than one record. Think book proceedings where each chapter lives as an independent bibilographic record. In practice, does this happen, or would there always be a "parent record" for book? Check with CERN library use case and other CC-ed libraries.
• Item borrowing times may differ for various items depending on their state. This is akin to reference books never leaving the room.
• Item availability may differ depending on item location. Think deposition store in the basement, or other building. We need a "libraries" class as @aw-bib already commented on above.
  • It is good to study the current table structure to see what was deemed needed, such as crcLIBRARY. Even if the code etc will be replaced fully, and even if the table structure will be changed fully.
• Borrowers are being sent recall email, if they are overdue. This is another core feature that may need to be represented and made configurable. Different libraries may have different workflows.
• In theory, borrowing times could differ for different class of users. E.g. students, staff, professors, retirees. In practice, does this have to happen? Check with CERN library use case and other libraries.
• In Invenio 1, the acquisition and ILL were part of the module, so to speak. In Invenio 2, it will be good to separate them away. Anyway, it seems profitable to discuss these use cases at the same time, so that the whole picture could be attacked globally. Can you start thinking about and preparing sister RFCs for these?
• I like the state-machine-like approach with rebrushing our use cases from blank slate. Instead of starting from scratch, have you got inspiration from existing state diagrams describing circulation processes and event diagrams and whatnot? Here is one example for illustration; our system librarian colleagues will be able to provide the best domain knowledge pointers.
  • Another inspiration can be obtained from similar applications, e.g. see Koha's workflows and table structure.

In addition to @aw-bib's CC list above, I'm CC-ing also @Kennethhole @PXke who may have other requirements coming from @tind clients, and @anastass @jgarcial who may offer insight from IAEA workflows.

[aw-bib]

@kaplun: for configurability, I agree that there's a defined set. Probably, it needs elaboration, but this might depend on the model further down the road. One can treat permanent loans like a collection of a library, maybe this is a better approach.

For the defined shelf: My musing was in the direction of item = one bibliographic item (=record), which holds several of your items. I think now it's clear to me what your item is. Still more status info (thinking in 876 $j in Marc) is locked even to your items: you're right , that "on shelf" is the status, however, Marc 876 always goes with 852 defining which shelf.

@tiborsimko: for individual articles in a proceedings volume, if you want to make them available for lending, you'd have individual bibliographic records and add holdings to them, not the parent. You'd need this anyway for retrieval on the users side. I agree that I'd also have a parent record as a common bracket around them (requires record linking), but the holdings, status and such would live on the articles, not on the parent entity. Very similar are multi volume works which are very common. E.g. you'd catalogue every volume of the Britannica Biographical encyclopedia of artists and not just Encyclopedia Britannica, 4 Volumes A-Z (E.g. http://gso.gbv.de/DB=2.1/PPNSET?PPN=523994168 for the parent, http://gso.gbv.de/DB=2.1/SET=7/TTL=1/FAM?PPN=523994168 for the children). Another very similar thing where it is quite obvious, would be a book series. Surely, you'd catalogue every book and the series would only be used as the bracket.

• Besides the reference works @tiborsimko mentioned, borrowing times would better not live on the item, but on the borrower(s' role) as also commented by you below. E.g. if you have a lot of students at your lab it might well be that you give them a lending period of only a week (probably they are only there for the 4 weeks of your regular lending period...) while a regular scientist of the lab gets 4 weeks. Or in a university setting: students get 4 weeks while university staff gets "unlimited till claimed by someone."
• For the LIBRARIES class there is definitely some sub location (named collection in current bibcirculation, if I understood it correctly). This could handle the above mentioned reference works: if in sub location "reference" it follows that nobody can borrow it, unless role allows to (probably staff can get it anyway as they are assumed "on campus and available all the time").
• For additional inspiration I think Koha is probably indeed a pretty good starting point as AFAIK it models most use cases.

Still, even if you might not love it (you don't have to), I'd check into the Marc Holdings (852)[http://www.loc.gov/marc/holdings/hd852.html] and (876/878)[http://www.loc.gov/marc/holdings/hd876878.html]. I discussed the issue of cataloging with @katringrosse, @CLSi and also our local man at the front @mahncm. It crystallized that the current procedure of doing bibedit the click through the holdings stuff is not very convenient. For our case I thus implemented a simple one way sync from Marc 852/876 to the internal tables as this is a much more streamlined work flow for the cataloguers. I added some field-templates to ease up things (here it would be nice if a field template could add two fields at once) and currently try to integrate a few simple kbs to keep typing to a minimum. The general comment was: "I'm in bibedit anyway for cataloguing, no need to leave it. Make addition of holdings to happen there and with just adding fields." My current implementation uses (indicator 1 == 7 signifies: source in $2 as usual) uses:

• 8527_

subfield	value	example
2	Source	DESY
a	Library	DESY Hamburg Library
b	collection	Permanent Loan
c	shelfmark	C Loe
h	classification	C
p	barcode	082932
t	item number	1
z	public note

• 8767_

At DESY NSK is a number similar to an accession number. But the real accession number is the the barcode (in general accession number and barcode differ). I don't like the $h yet as it should actually live on the users role but I fear I can't do that in invenio 1.x:

subfield	value	example
2	Source	DESY
a	NSK	47538
d	Accession date	20141120000000.0
p	barcode	082932
h	loan period	4 weeks / 1 week / reference
j	availability	available/missing
t	item number	1

I'll have to extend this to model the book of acquisitions completely (adding prices and seller), that's already ticketized at join2.

[lnielsen]

Have you compared the proposed data model from other library systems, e.g. https://github.com/liblime/LibLime-Koha/tree/4_18/t/data/db_schemas

[lnielsen]

or e.g. http://docs.evergreen-ils.org/2.8/schema/

[mvesper]

As discussed with @jirikuncar and @Kennethhole:
Maybe we can agree on a name for this module and start a new repository so we can discuss the different aspects of the design in separate RFCs and host some design documents there.
We could then use this RFC to summarize the process.

[tiborsimko]

https://github.com/liblime/LibLime-Koha/tree/4_18/t/data/db_schemas

FWIW I posted nicer URL above 😜

• http://schema.koha-community.org/relationships.html
• http://schema.koha-community.org/index.html

[tiborsimko]

Maybe we can agree on a name for this module

I think invenio-circulation would be a good name. This is how the facility is usually called, see for example ILS@Wikipedia:

• acquisitions (ordering, receiving, and invoicing materials)
• cataloging (classifying and indexing materials)
• circulation (lending materials to patrons and receiving them back)
• serials (tracking magazine and newspaper holdings)
• the OPAC (public interface for users)

[PXke]

Some other points :

• We need signals !
• We need to be able to create message template easily, for different situation, for example a librarian with no prior knowledge in python should be able to create a message that will be sent on different action.
• It needs to be possible to chose when to change the message either by plugging it on some signals or on some workflow steps.
• It needs to be possible to add some type of variable in the message like dates, barcode or other information about the operation.
• It needs to be possible to choose how to send the message (Ie SMS, Mails etc) I think only mail should be implmented by default but the send message interface should allow to plug module.
• Message should be able to be associate in which way it needs to be sent.
• Data is important for Librarian, it will be cool to be able to keep history of operations, which record has been lend and when.
• Data must be able to be mine to extract tendencies. Establish estimation data, forecast data.
• It would be cool if it contains a part to do intercommunication between different invenio instances, for ILL or other IL operations.
• It should be possible to configure which type of delay and periodicity of message for different type of object and people category. Default situation should be possible.

That's all at the moment.

[Kennethhole]

Regarding message templates: as Invenio support multiple libraries, the signature should reflect the library which lend out the item.

[aw-bib]

@Kennethhole I'd add sublocation (collection in LoC speak) to the list to manage things like:

Dear Kenneth,
You have item 823.914: "The book of strange new things" by Michel Faber from the Reference collection of DESY Library, Hamburg (Barcode 01234) on loan. We have a request for this book and would kindly ask...

Reference collection would be such a sublocation. Similar departments local shelfs or the like. (I put the variables of the above in italics.) BTW: perfectly agreed with @PXke that this has to be easily configurable by a non-programmer.

[martinkoehler]

A short comment to other library systems
IHMO: Besides KOHA, ALPEH has a lots of use cases as well
see http://lib.haifa.ac.il/extprojects/scill/images/pdf/Aleph%2021%20Circ%20guide.pdf

@kaplun With respect to the on-helf:
When an item has been requested it is usually put on a request shelf (same can be true if it is returned and still waits for further processing, then its on a return shelf. It might from there be shipped to another sublibrary (e.g. an item from DESY Zeuthen (Berlin) is at DESY Hamburg, it must be shipped back to Zeuthen (Btw1: with the status "in-transit" during shipping :-). BTW2: "in transit" belongs in MARC 876$l (temporary location))
Another shelf is the new item shelf. Items are on-display on that shelf for e.g. one week for users to see them more prominently. Then the item is moved to the "final" loaction (btw. during that stage this item can not be borowed (This is an example for @tiborsimko variable borrowing times)

With respect to the checkout. A self checkout is important for 7/7 opening hours
There is a module from KOHA for SIP which might be usefull in this connection.

[mvesper]

Even though there is still a lot of information to process, I split this RFC into 3 RFCs in another repo inveniosoftware/invenio-circulation#2, inveniosoftware/invenio-circulation#3, inveniosoftware/invenio-circulation#4.

This will hopefully lead to a more structured discussion, while this RFC is used for some kind of summary :)

[jirikuncar]

Discussion continues in Invenio-Circulation package.