To: IUAG
From: CCAC and OPACCTF
Date: May 3, 2006
Re: Indexing initial articles in Voyager
CCACC (the Consortial Cataloging and Authority Control Committee) and OPACCTF (the OPAC Customization Task Force) recommend that ten new indexes be created in Voyager as part of the upgrade to Voyager V6.1. We think the indexes will improve access to titles that begin with non-significant words such as "a," "an," "the," and their equivalents in other languages. Because all Voyager indexes have to be rebuilt during the upgrade, this request is timely. Once the indexes have been built, OPACCTF and CCAC can test further and decide whether to proceed with incorporating them into WebVoyage.
Details about this recommendation follow:
The TALL ("Title") and JALL ("Journal / Magazine Title") indexes that appear on the Quick Search page in WebVoyage (and also in Voyager staff clients) are "composite" indexes, composed of multiple "left-anchored" indexes of individual title fields in the MARC record. Several of these MARC title fields support non-filing indicators, and the existing left-anchored indexes of these fields respect those indicators. A non-filing indicator tells the system to ignore the initial article on a title, and to begin indexing the title with the first significant word. For instance, a non-filing indicator of 4 on the title The Evolution of Jane causes the title to be indexed under "evolution" rather than under "the." The effect on the user is that to search for this title the user must enter the search "evolution of jane" rather than "the evolution. . . . " This is "The Initial Article Problem" that has plagued library users since the first online search systems became available. Usability tests in ILCSO/I-Share and elsewhere show that a very high proportion of the "no hits" messages users see can be explained by the users' inclusion of initial articles in their search arguments, and that even when faced with these messages, users do not realize they need to omit initial articles when they try again. (Indeed, usability tests show that users do not know what "initial article" even means, and so libraries have devoted significant effort both to instruction and to interface redesign, to try to improve users' search success rate.)
CCAC and OPACCTF recommend that in addition to the existing left-anchored indexes that respect non-filing indicators we create left-anchored indexes that ignore non-filing indicators. If both kinds of index were combined in the TALL and JALL composite definitions the effect on the user would be that either the search "evolution of jane" or the search "the evolution of jane" would retrieve the record desired. (Of course, if the user searches for "the jane eyre" and the title is truly Jane Eyre, the search will still fail. Regardless of what indexes we build, what the user enters is still what the system tries to find.)
Indexing titles both with and without their initial articles solves a number of problems:
- First and foremost, it puts the data (the title and other data linked to it) in the place where the user is looking for them, whether the user is "looking" with an initial article or not.
- In cases where catalogers have inadvertently set the non-filing indicator incorrectly, the record can still be retrievable by title.
- Attempts to extract initial articles from the user's search input, an alternative strategy CARLI has considered for solving The Initial Article Problem, are fraught with complications. (Is the word "die" at the beginning of a search argument an initial article or not?)
- Attempts to design a search interface that discourages users from typing initial articles have failed signally over the years.
- Likewise, efforts to instruct users not to type initial articles have failed, and doubtless will continue to fail.
Note that even if non-filing-indicator-ignoring indexes are incorporated into the TALL and JALL composite definitions, TALL and JALL search results will still be sorted with respect for non-filing indicators. If more than one record meets their search criteria users will see results sorted first by the title-as-retrieved (the indexed form that matched the search criteria), then by full title without initial article (assuming non-filing indicators have been set correctly), then by date of publication.
Example: User searches for "the evolution of j":
Line# |
Title |
Full Title |
Author |
Date |
1 |
The evolution of Jane / |
Evolution of Jane / Cathleen Schine. |
Schine, Cathleen. |
1999 |
2 |
The evolution of Jane / |
Evolution of Jane: a Novel / Cathleen Schine. |
Schine, Cathleen. |
1998 |
3 |
The evolution of Japanese security policy / |
Evolution of Japanese security policy / by Yukio Satoh. |
Satoh, Yukio. |
1982 |
4 |
The evolution of Javanese gamelan / |
Beginners Guide to Evolution of Javanese gamelan / Mantle Hood. |
Hood, Mantle. |
1980 |
5 |
The evolution of Javanese gamelan / |
Evolution of Javanese gamelan / Mantle Hood. |
Hood, Mantle. |
1980 |
6 |
The evolution of jazz dance from folk origins to concept stage / |
Evolution of jazz dance from folk origins to concept stage / by Russella Brandman. |
Brandman, Russella, 1945- |
1992 |
7 |
The evolution of jazz dance from folk origins to concert stage |
Evolution of jazz dance from folk origins to concert stage [microform] / by Russella Brandman. |
Brandman, Russella, 1945- |
1992 |
8 |
The evolution of jazz dance from folk origins to concert stage |
Evolution of jazz dance from folk origins to concert stage [microform] / by Russella Brandman. |
Brandman, Russella, 1945- |
1979 |
CARLI staff have already done a proof-of-concept test. They built one left-anchored title index that ignored the non-filing indicator, for one field, in one database, on a test server. That one sample index is no longer available for testing, because the test server has since been refreshed with a new copy of I-Share libraries' production data. The next step, after the refreshed test server has been re-upgraded to Voyager 6.1, will be to build non-filing-indicator-ignoring versions of all the left-anchored indexes that use non-filing indicators and that currently are included in I-Share libraries' TALL or JALL composites. Those indexes are:
- N130 (adapted from 1300)
- N211 (adapted from 2110)
- N222 (adapted from 2220)
- N240 (adapted from 2400)
- N245 (adapted from 2450)
- M245 (adapted from 2451)
- N440 (adapted from 4400)
- N730 (adapted from 7300)
- N740 (adapted from 7400)
- N830 (adapted from 8300)
For the next phase in testing, CCAC and OPACCTF recommend that CARLI build these ten left-anchored, non-filing-indicator-ignoring indexes still for just one test database. If that round of tests goes well, then CARLI staff will build the same ten indexes for every database, at the time we do the real upgrade of our production databases. Finally, if post-upgrade testing continues to go well, OPACCTF can decide whether to specify a permanent modification to the TALL and JALL composite definitions in each I-Share library database. (The TALL and JALL composites are among the core indexes that OPACCTF includes in the default WebVoyage configuration and that OPACCTF has, in the past, specified should be defined the same way in every database.)
Adding the ten new left-anchored indexes to the TALL and JALL composite definitions is a change that can be made through the Voyager SysAdmin client, so it does not have to coincide with the upgrade. Only the left-anchored index creation must be done at the time of the upgrade. Once the left-anchored indexes are created for each database, at the time of the upgrade, OPACCTF can, at its post-upgrade leisure, add them to the composites and continue its testing. CCAC and OPACCTF are very excited by the potential for this indexing change to solve, or at least greatly reduce, The Initial Article Problem. We hope IUAG will approve our recommendation for index creation and continued testing.